Errata: 14:00 The rule is noted down incorrectly. I mistook B & C. Correct would be p (A, B) = p (A, B). Thanks to @Ngoc Anh Nguyen for pointing this out. The file on GitHub has been updated accordingly: github.com/Ceyron/machine-learning-and-simulation/blob/main/english/probabilistic_machine_learning/directed_graphical_models_d_separated.pdf 17:13 The result should, of course, be that N&O are d-separated given W. (I wrote (and also said) that N&P were d-separated given W, which is not true!). Thanks to the anonymous user who spotted this.
Thank for the useful video! But still have a small question in this part: in basic rule 3, do we say A and B is d-separated by C or A and C is d-separate by B? Get a bit confused at this part.
Thanks so much :) It was the same for me. Examples really helped me a lot in getting the full understanding. I also did a video on how to check for d-separation in Python using NetworkX: ua-cam.com/video/1wMZiejmGWU/v-deo.html I always find using libraries or coding it down yourself particularly valuable.
hi, i am a bit confused, in a previous video you talked about how if the arrow -> is from W->H the joint p(W,H) should be p(H|W)p(W) then now in at 08:55 we have the something similar where W->H->P so shouldn't the p(W,P,|H) = p(H|W)p(P|H). Thank you
Thanks for the question :) Here, we introduced an observed variable. That changed the game a bit. The goal of these simple rules is no longer to just factor the joint (these rules will always hold in directed Graphical models), but to find how observed variables change the relation between other variables. The rule I noted down was purely based on arguing.
@@MachineLearningSimulation thank you for your clarification. so based on what i can understand this is due to the fact that in this specific problem we had an observed variable that depends on two unobserved ones. since only "H" is given so we would say that given H the "W" and "P" are independent. additionally i also watched some other videos related to active and inactive paths between triples in a graph. that also made this concept somewhat clear. additionally, could you please share the reference material ? I am trying to read some papers on variational autoencoders and every paper introduces some notations that throw me off. i am trying to get to the bottom of this. haha
Of course :). Probably takes some practice to internalize these rules. I can recommend doing some examples with "networkx" (a graph theory library in python). I also have a video on this (should be the next one in the probabilistic ml series). A general reference is bishops "pattern recognition and machine learning".
Great content, thanks a lot! Just to make sure, you meant N & O, right? Because N & P don't seem to be d-separated (H is not observed so you can go from N to P through H). Thanks again :)
Hi, If P is observed, is N and O conditionally independent given P? My thinking is that W is not blocked but I am confused if H is blocked or not. H looks like scenario 3 which would mean it is blocked but am unsure haha... great video btw
Hi, Thanks a lot for the kind feedback :). Sorry for the delayed response; I hope it is still helpful 😊. Correct is: "N & O are NOT conditionally independent given P". I missed one detail for rule (3) in that it also holds if a descendant of that node is observed. So, in your scenario, "P" is observed, and "P" is a descendant of "H". As such, rule (3) applies to the triplet N -> H
Fair question. I think I wasn't too precise on this one. My initial goal was to show that "N" and "O" are d-separated given "W". In this case "H" is blocking because rule 3 applies (the case with Simpson's paradoxon). In essence, we would then have two nodes that are blocking on our way from "N" to "O". But one important point I missed: "H" is only blocking when looking at the conditional independence from "N" to "O". When we would look at the relation from "N" to "P", H is not blocking anymore (if it is latent; if it was observed, it would of course be again because of rule two) I hope this makes sense. Let me know if it is still confusing.
Hey, thanks for the amazing feedback :) I am super happy, I could help to that extent. There is no publication of mine in that regard. It is not my primary field of research. However, if it is an informal article, you could cite the GitHub Repo: github.com/ceyron/machine-learning-and-simulation (on the right side of that page you will find the button "Cite this repository" which produces a bibtex file for you). If that is not appropriate for the publication you plan, then I am equally happy if you could spread the word about the channel and promote it in your environment, or maybe give it a shoutout on social media (if applicable). Thanks again
What are you referring to? The playlist should still be available: 🎲 Probabilistic Machine Learning: ua-cam.com/play/PLISXH-iEM4JlFsAp7trKCWyxeO3M70QyJ.html
Hey, thanks for the reply :) To which part of the video are you referring? (Maybe a time stamp) If I remember correctly, this should be how I presented the rule. The third basic rule should: marginal independence, but conditional dependence.
@@MachineLearningSimulation Yes, if A-> C and B->C and C is unobservable, then A and B are independent, but A and B are conditional dependent if we control C. I think you just mistook B & C.
@@NgocAnhNguyen-si5rq You are right :) Good catch. Indeed, there is a mistake in the presentation. (I switched B & C) I will leave a pinned comment. Thanks a lot for figuring this out. :) Unfortunately, I do not have access to my written files from this older video. I will try to correct the PDF on GitHub as soon as possible.
I think that Berkson's paradox is related to sampling bias, therefore it should be Simpson's, but I could be wrong. Maybe that link could be resource: stats.stackexchange.com/questions/445341/simpsons-paradox-vs-berksons-paradox What do you think?
Errata:
14:00 The rule is noted down incorrectly. I mistook B & C. Correct would be p (A, B) = p (A, B). Thanks to @Ngoc Anh Nguyen for pointing this out. The file on GitHub has been updated accordingly: github.com/Ceyron/machine-learning-and-simulation/blob/main/english/probabilistic_machine_learning/directed_graphical_models_d_separated.pdf
17:13 The result should, of course, be that N&O are d-separated given W. (I wrote (and also said) that N&P were d-separated given W, which is not true!). Thanks to the anonymous user who spotted this.
Thank for the useful video! But still have a small question in this part: in basic rule 3, do we say A and B is d-separated by C or A and C is d-separate by B? Get a bit confused at this part.
Hello, great video!
Why are N&P not d-separated given W if the path is indeed blocked? (at H)
Best D-seperation explanation out there. Thank you so much
Thanks a lot for the feedback :)
This is by far the best explanation of d-separation. These concepts are hard to grasp. Illustrating with examples really clears a lot of grey areas
Thanks so much :)
It was the same for me. Examples really helped me a lot in getting the full understanding. I also did a video on how to check for d-separation in Python using NetworkX: ua-cam.com/video/1wMZiejmGWU/v-deo.html I always find using libraries or coding it down yourself particularly valuable.
@@MachineLearningSimulation checked it out. As again, clear and crisp. Thank you so much.
hi, i am a bit confused, in a previous video you talked about how if the arrow -> is from W->H the joint p(W,H) should be p(H|W)p(W) then now in at 08:55 we have the something similar where
W->H->P so shouldn't the p(W,P,|H) = p(H|W)p(P|H). Thank you
Thanks for the question :)
Here, we introduced an observed variable. That changed the game a bit. The goal of these simple rules is no longer to just factor the joint (these rules will always hold in directed Graphical models), but to find how observed variables change the relation between other variables. The rule I noted down was purely based on arguing.
@@MachineLearningSimulation thank you for your clarification. so based on what i can understand this is due to the fact that in this specific problem we had an observed variable that depends on two unobserved ones. since only "H" is given so we would say that given H the "W" and "P" are independent. additionally i also watched some other videos related to active and inactive paths between triples in a graph. that also made this concept somewhat clear.
additionally, could you please share the reference material ? I am trying to read some papers on variational autoencoders and every paper introduces some notations that throw me off. i am trying to get to the bottom of this. haha
Of course :).
Probably takes some practice to internalize these rules. I can recommend doing some examples with "networkx" (a graph theory library in python). I also have a video on this (should be the next one in the probabilistic ml series).
A general reference is bishops "pattern recognition and machine learning".
Great content, thanks a lot!
Just to make sure, you meant N & O, right?
Because N & P don't seem to be d-separated (H is not observed so you can go from N to P through H).
Thanks again :)
Hey, thanks a lot :)
You're absolutely right, I meant N & O.
Hi, If P is observed, is N and O conditionally independent given P? My thinking is that W is not blocked but I am confused if H is blocked or not. H looks like scenario 3 which would mean it is blocked but am unsure haha... great video btw
Hi,
Thanks a lot for the kind feedback :). Sorry for the delayed response; I hope it is still helpful 😊.
Correct is: "N & O are NOT conditionally independent given P". I missed one detail for rule (3) in that it also holds if a descendant of that node is observed. So, in your scenario, "P" is observed, and "P" is a descendant of "H". As such, rule (3) applies to the triplet N -> H
In your last example, why is H a block given that H is not observed.
Fair question.
I think I wasn't too precise on this one. My initial goal was to show that "N" and "O" are d-separated given "W". In this case "H" is blocking because rule 3 applies (the case with Simpson's paradoxon). In essence, we would then have two nodes that are blocking on our way from "N" to "O".
But one important point I missed: "H" is only blocking when looking at the conditional independence from "N" to "O". When we would look at the relation from "N" to "P", H is not blocking anymore (if it is latent; if it was observed, it would of course be again because of rule two)
I hope this makes sense. Let me know if it is still confusing.
Thanks for the lecture and since I am working on a article about DAG, if you have any paper published about this, I would love to cite them.
Hey, thanks for the amazing feedback :)
I am super happy, I could help to that extent. There is no publication of mine in that regard. It is not my primary field of research. However, if it is an informal article, you could cite the GitHub Repo: github.com/ceyron/machine-learning-and-simulation (on the right side of that page you will find the button "Cite this repository" which produces a bibtex file for you). If that is not appropriate for the publication you plan, then I am equally happy if you could spread the word about the channel and promote it in your environment, or maybe give it a shoutout on social media (if applicable).
Thanks again
@@MachineLearningSimulation Thanks for the reply!
Why did you remove the playlists :’(
What are you referring to?
The playlist should still be available: 🎲 Probabilistic Machine Learning: ua-cam.com/play/PLISXH-iEM4JlFsAp7trKCWyxeO3M70QyJ.html
Hi! Great content! But I think the third rule should be P(A)P(B) = P(A,B) with no conditional on C
Hey, thanks for the reply :)
To which part of the video are you referring? (Maybe a time stamp) If I remember correctly, this should be how I presented the rule. The third basic rule should: marginal independence, but conditional dependence.
@@MachineLearningSimulation It's 14:00. Correct me if I'm wrong ^^.
@@MachineLearningSimulation Yes, if A-> C and B->C and C is unobservable, then A and B are independent, but A and B are conditional dependent if we control C. I think you just mistook B & C.
@@NgocAnhNguyen-si5rq You are right :)
Good catch. Indeed, there is a mistake in the presentation. (I switched B & C)
I will leave a pinned comment. Thanks a lot for figuring this out. :)
Unfortunately, I do not have access to my written files from this older video. I will try to correct the PDF on GitHub as soon as possible.
@@MachineLearningSimulation You're welcome. Keep up with your good work ^^.
Thank you so much!
Glad it helped!
@11:05: should it be Berkson's paradox instead?
I think that Berkson's paradox is related to sampling bias, therefore it should be Simpson's, but I could be wrong. Maybe that link could be resource: stats.stackexchange.com/questions/445341/simpsons-paradox-vs-berksons-paradox
What do you think?
perfection!
Nice streak of comments, love it :)
Let me know if you have any additional topic proposals or things you would want to see covered.
Thank you so much.....
You're very welcome 😊
Exam in 20 minutes, thanks haha
Best of luck! 😉
Das english
Ich hoffe, es hat einen guten Eindruck hinterlassen ;)
Lass mich wissen, falls was unverständlich ist