Ancestral Sampling & Likelihood Calculation | Directed Graphical Models
Вставка
- Опубліковано 20 тра 2024
- How do you sample Directed Graphical Models by following the edges? Welcome to Ancestral sampling. You can find the notes here: raw.githubusercontent.com/Cey...
In this video, we use ancestral sampling to draw random variates from joint distribution. This technique exploits the graph structure of the directed graphical model. Similarly, we can also use this structure to calculate the likelihood.
-------
📝 : Check out the GitHub Repository of the channel, where I upload all the handwritten notes and source-code files (contributions are very welcome): github.com/Ceyron/machine-lea...
📢 : Follow me on LinkedIn or Twitter for updates on the channel and other cool Machine Learning & Simulation stuff: / felix-koehler and / felix_m_koehler
💸 : If you want to support my work on the channel, you can become a Patreon here: / mlsim
-------
⚙️ My Gear:
(Below are affiliate links to Amazon. If you decide to purchase the product or something else on Amazon through this link, I earn a small commission.)
- 🎙️ Microphone: Blue Yeti: amzn.to/3NU7OAs
- ⌨️ Logitech TKL Mechanical Keyboard: amzn.to/3JhEtwp
- 🎨 Gaomon Drawing Tablet (similar to a WACOM Tablet, but cheaper, works flawlessly under Linux): amzn.to/37katmf
- 🔌 Laptop Charger: amzn.to/3ja0imP
- 💻 My Laptop (generally I like the Dell XPS series): amzn.to/38xrABL
- 📱 My Phone: Fairphone 4 (I love the sustainability and repairability aspect of it): amzn.to/3Jr4ZmV
If I had to purchase these items again, I would probably change the following:
- 🎙️ Rode NT: amzn.to/3NUIGtw
- 💻 Framework Laptop (I do not get a commission here, but I love the vision of Framework. It will definitely be my next Ultrabook): frame.work
As an Amazon Associate I earn from qualifying purchases.
-------
Timestamps:
0:00 Opening
0:11 The example Graphical Model
5:23 Ancestral Sampling
06:15 Drawing Samples with NumPy
08:42 Repetition: Factorizing the joint
09:35 Likelihood Calculation
12:37 Summary
👏👏👏👏👏this is awesome! You're truly an elite teacher.
You're welcome, thanks for the kind feedback 😊
These videos are great, keep up the good work! Are you planning on any code-focused tutorials on graph. models?
Thanks for your feedback :) I appreciate it a lot.
Yes, I am definitely planning for more code-focused videos on this topic. Soon, there will be videos on the EM-Algorithm and Variational Inference.
Do you have any video topics that you would like to see covered? This channel is in its early stages and I would love for it to be shaped by the community.
Awesome videos. Seriously awesome job. I subscribed.
Thanks a lot :)
Thank you very much.
You're welcome 😊
Great!
Is tensorflow probability able to handle such a more complex DGM, like the in the first video? For example, we can sample by using tensorflow probability as well, as oppose to the way you did it here by using Python's uniform random variable generator?
Good question: Simply put, you can always use ancestral sampling on DGMs given you have a way to sample from all the involved distributions.
However, there are some interesting things to consider:
1) There are distributions that are not easy to sample. In this video we only looked at Bernoulli and Categorical distribution which are extremely simple. Things are getting more complicated for stuff like Gamma, Beta etc. which take a noticeable amount of computational resources in order to create random variates (= samples)
2) Auto-Correlation due to imperfect pseudo-random number generators: Almost all sampling algorithms for various distributions require a "source of randomness". Usually, this is a pseudo-random number generator. These techniques are a science for itself as you have to mimic randomness on a computer which is purely deterministic. Take a look at the Wikipedia article: en.wikipedia.org/wiki/Pseudorandom_number_generator
Due to these limitations, you usually use more advanced sampling techniques than Ancestral Sampling. This is in particular important if you have to create high-quality samples in order to approximate expectations. There are some really cool algorithms, most importantly there are "Markov-Chain Monte-Carlo"-techniques. They are also an essential part of TensorFlow Probability. I will also be covering them in a future video. They base on the idea that the DGM easily factorizes our joint, and we query the probability (or log-probability) of a proposal sample.
Let me know if something was unclear :)
Thanks for the content!
I have one confusion though.
The way you are drawing the samples, it is actually based on interval between [0,1] right?
For instance for Season S (timestamp 06:50), what you did was you draw the number ~ 0.6257. Now you picked "Summer" because you assumed "Winter" € [0, 0.25], "Spring" € (0.25, 0.50], "Summer" € (0.50, 0.75] and "Fall" € (0.75, 1].
So the main question is the numbers that you draw [0,1] are no probabilities in that sense, but rather kind of continuous indices, which helps you assigning the value right?
Hopefully you understood my confusion and could explain it briefly :)
Hi,
thanks a lot for the comment and the kind words. :)
You are right, the number I drew uniformly between 0 and 1 is just a helper to then get one of the 4 discrete classes. Feel free to also check out the corresponding video on sampling the Categorical distribution: ua-cam.com/video/MYHxW-MIld0/v-deo.html
Hope that helped :). Feel free to ask a follow-up question if sth is still unclear.
I think it’s important to note that random.rand() is a guassian not a uniform distribution… random.uniform is the uniform distribution. Since he is just whimsically picking certain states you can use random or whatever… but when he is calculating the the probability he is actually using the correct probabilities to compute the complete probability of the sampled information
However, for correct sampling he should have utilized rand.uniform instead of rand.rand() but it’s forgivable since it is not a video about sampling just showing given a sampled information whats the probability of getting that. But bigger question is Elon Musk’s sleep is gone and nobody said a thing.
First off, thank you very much for excellent videos. BTW, I have a question: Should not you say: because the probability is above 0.2 we pick good Night Sleep and not because it is below 0.8, should you? Could you please shed more light on this aspect? What state would you pick up if you get a random probability value above 0.8, below 0.2? And why please.
Hi, thanks for the feedback and the question :). Can you give a timestamp to the point in the video you are referring to? It has been some time since I uploaded the video.
@@MachineLearningSimulation 6:38 - 8:40. Many thanks in advance.
@@todianmishtaku6249 I think I now understand the question. :) We are assigning categories, depending on whether a number is in the interval (0.0, 0.8) or in the other interval (0.8, 1.0). Hence, we can say, we assign one category if the random variate is below 0.8 and the other one if it is above 0.8.
However, we could also take the intervals the other way around, i.e. we assign the latter category if the random variate is in (0.0, 0.2) and the other one if it is in (0.2, 1.0), as long as we use the larger interval for this outcome of the Bernoulli with the higher probability.
I would say, it does not matter, both approaches are correct ways to sample Bernoulli distributions (maybe this video of mine also helps ua-cam.com/video/jqBc32NGKn8/v-deo.html ).
You can also try it yourself (e.g. with Python): Let's say you draw 1000 random numbers (uniformly distributed between 0 and 1). Then, you first assign labels with one interval scheme and the another time with the other. You will see that you will end up with approximately the same number of random variates for each category. (And if you increase to 10000 or higher the relative differences will become more and more negligible).
Hope that helped :)
@@MachineLearningSimulation, pretty cool clarification. it confirmed my suspicion: we project a second variable- a uniform variable in order to randomly generate values of it such that when all the values are generated, we can notice that (roughly speaking) theta percent of them are
That's correct 👍
To my knowledge uniformly distributed numbers are the base for all sampling methods to distributions, e.g., also to the Normal distribution: ua-cam.com/video/4fVQrH65aWU/v-deo.html
I want Elon musk’s sleep to come back
😅
Sorry but your exaplanation went over my head
Hi, I'm not familiar with that phrase. 😅 Did you like the explanation?
@@MachineLearningSimulation Nah your lectures are great but too overwhelming for someone who's starting into this field..I had a hard time understanding and had to watch 5-6 times in order to understand it a little. Lectures are great It's why I'm watching It like 5-6 times but It's just feels too hard for a beginner like me
Thanks for the feedback 😊
its not easy for beginners, I have come back to this playlist 1 year later and its starting to make sense now!