Machine learning - Bayesian optimization and multi-armed bandits
Вставка
- Опубліковано 11 чер 2024
- Bayesian optimization, Thompson sampling and multi-armed bandits. Applications to algorithm configuration, intelligent user interfaces, advertising, control and other decision problems.
Slides available at: www.cs.ubc.ca/~nando/540-2013/...
Course taught in 2013 at UBC by Nando de Freitas
He is a hidden gem .. I have gone through a lot of his videos , they are great in terms of removing jargon .. and bringing clarity
Learning advanced ml concepts for free! What a time to be alive. Thanks a lot for the vid!
wow, this professor is such a great teacher. Model for all profs!
please explain the first colourful plot he illustrates with
You gotta watch his previous videos on Gaussian processes to understand this lecture. Basically you need to understand the Gaussian processes first.
@@always-stay-positive5187 using the acquisition function to locate the point we want to optimize I guess
"heuristics" -> "terrorist sex"
youtube automatic captions is getting better :D
Very elaborate and simple presentation. Thank you.
Thanks prof. Freitas, this is extremely helpful
finally found a nice class. thank you very much
Thank you for providing excellent resources
This is the best video for understanding the Bayesian Optimization. It would be a great help if you could you post a video on multi objective Bayesian optimization specifically on expected hyper volume improvement. Thank you
Thankyou! Great lecture. Really enjoyed
Too good in explaining, I never took such detailed notes.
Great explaination!
Thank you! Really great explanation!!!
Hello, professor Nando, I have som questions about the smoke simulation auto optimization.Since Bayesian optimization is about better places at next round, I see user can choose many places, so I think it should be many good candidates, and also,
the objective function is what is still not clear, in user's mind? So you mean we using Bayesian optimization to approach the function in user's mind? And also, I check main open-source library out there, optunity seems quite nice and have a very easier api for end user like me. It suggest using Particle Swarm Optimization or Tree-structured Parzen Estimator to optimize,so can PSO and TPE also can do things like in the video, let user choose many candidate? Is it possbile? Thanks
great teacher...
Hello Sir (Nando de Freitas) , A Really Great lecture on this optimization method, I am a Computer Science student, and i have gone through your other lectures on Random Forests as well, sir i am working on a research project based on automatic tuning of the hyperparameters in Random Forests. This method is really great for that but i was wondering like are there any other optimization methods available to tune the hyperparameters ? It will be a really great help from your side Sir.
refering to GP-UCB (around 57 min). Do you discretize the x domain and then search for the min (argmin GP-UCB(x)) or do you use a gradient-based optimizer on GP-UCB(x)? In case you use the second option, how do you calcuate the gradient of GP-UCB(x)? This should be analytically trackable.
change playback speed to 1:1.25. btw great lecture!
Hi,
If we have A=0,1 , B =0,1 and C =0,1,2 total combination is 12 ..How can we reduce using Bayesian optimization
Thanks!
It seems there were some contents presented after 1:20:30.
a stupid question: how to do x_{n+1} = argmax u(x | D)? just randomly choose x and see if it generate biggest u(x | D)? how essentially to generate the curve of the acquisition function? Thanks!
Say we have n bandits labelled from 1 to n. And if on the x-axis, I take 1 to n and on y-axis I take their corresponding rewards. Then I don't think it would be safe to say that my function is smooth. What are alternatives when you don't have smooth functions?
I think when your actions are discrete and your utility function over actions f(a) is not be smooth, GP might not be your best choice.
Instead, you can model a distribution for every action, e.g. using beta distribution for each bandit.
@@zhouxinning7284 though beta distribution is as far as I understand only a good choice for Bernoulli-bandits, where you either win or lose, isn't it?
in the PI acquisition function, the Phi function has the variance as the denominator. So to maximise Phi(x), mean needs to increase and variance need to decrease right? But the explanation in 40.31 the prof says high variance is needed since we are trying to maximise the area under the curve. Can someone clear this to me and show a way to connect the equation with the graph explanation please.
Nirandika Wanigasekara There's an error in the slides, which is that the probability for a right-tail should be 1 - the CDF. This corresponds to wanting the CDF to be close to zero, i.e., mean close to (mu^+ + epsilon) and variance as large as possible.
+Dustin Tran +Nirandika Wanigasekara I believe the equation is correct as 1-CDF(x) = CDF(-x) for Gaussian distribution. Higher variance is justified as long as mu(x) is less than (mu^+ + eps), but a lower variance might be preferred when mu(x) > (mu^+ + eps) which is counterintuitive.
Awesome!!!
Thompson sampling @59:00
The use of notation at 23:00 is confusing for me
Professor de Freitas has neat paper on this very topic. Do yourself a favor and grab the pdf at arXiv:1012.2599
thanks for sharing!
Good to know slot machines always pay out the same amount and there's no variability.
I am becoming smarter ...muahaha
i dont understand those plots. they dont look lie Gaussians at all.
Always-Stay-Positive Actually, they aren’t supposed to be! The Gaussian process induces a gaussian prior over all possible functions. What’s being plotted is the mean function (the function that gives the mean at a particular point). Does that make sense?