I am a bit troubled about the result of the tickets lottery. I would expect to buy from all tickets from the all-0s ticket up to the (1000 choose 123) tickets to have 99% of winning (so it would rather be => sum of (N choose n) with n goes from 0 to 123). But here, the teacher only considers the (1000 choose 123) tickets which does not represent 99% chance of winning the game. I can't see where I failed to understand this result ... @Jakob Foerster ?
Hi @Jakob or anyone, I have a wired question that for the submarine. if instead of using the probability that each box is not the one = log64/63+log63/62..and things do cancel out, we using the probability that each is the one which would use the probability of 1/64+1/63...and so on. Therefore, log64+log63.... I am assuming each guesses on the box will give the same information. But why is these two different? Could anyone help with this thanks .
The log64/63+log63/62.. formula assumes there's only one submarine. The 1/64+1/63 formula you proposed would another game where the submarine relocates (without revisiting any location it has stayed at) each time you hit it and you hit it every time (which is extremely lucky).
These lectures are really nice! I would like to know why the string 00000....0000 was the most probable in the 20 bits string? Is this string 00000...00000 the most probable when N=1000? Thanks in advance
0 has a probability of 0.9, while 1 has a probability of 0.1. Therefore the 0^N string you mention is the single most probable string, independent of N.
Another way of thinking is that even thought the probability P(x=00...0)P(x=S, S is one specific string | S has a "1"). Sorry for reviving the post after such a long time haha
As @Jakob was saying, any ticket t has chances P(t) of being selected, where P(t) = 0.9^N_0 * 0.1^N_1 constrained to N_0 + N_1 = N. To maximize P(t), then you should take N_0 = N, and that is why the ticket t = (00...0) is the best ticket for N = 20, N = 1000 or any other value.
As @Gabriel pointed out, it is interesting that even though a particular ticket with N_1 > 0 is a priori more unlikely to be selected, there are so many tickets with N_1 ones, that choosing one of these finishes being more likely than choosing the single one with N_0 = N.
@@julianmartindelfiore7420 I believe I had assumed the equal probability of getting a 0 or a 1. Maybe that was my mistake. I will watch the video again. Thanks!
@@lkcastelano Maybe! Hehe but in any case, your question pointed to an important fact: the reason why this was the correct answer was vaguely discussed, but not formally shown in the video...so i just tried to provide the More detailed explanation in case someone was looking for that :) hehe
It's because the fist question is basically asking what the MSB is. For example, if the answer is yes, we know the binary number is: "1xxxxx", where 'x' is still unknown. The next question determines the next digit: "10xxxxx". The last question determines the LSB: "101010". I think the LSB (even or odd) is easiest to think about intuitively. If you ask if the number is odd, and the answer is "Yes", you know that the number in binary has a '1' as the LSB.
If what you mean is all tickets t such that t has N_1 ones, where 77 1/(1-P_0) or, to say something N * (1 - P_0) = N * P_1 >= 100. Since we have N = 1000 and P_1 = 0.1, then we can use the CLT, thus the gaussian distribution approximation is valid for the example, for any N_0 or N_1 you consider. I think that the answer is more related to what I was saying above.
@@JakobFoerster This is wrong, the limit of the binomial distribution is normal for any p. The asymmetry comes from the fact that buying the tickets from 0-"1s" to ~77-"1s" is drastically cheaper than the rest, and it saves you buying the tickets around 2^~123.
Well, he is approximating the log(1000 choose 123) term, not the log( 1000 choose 100) term. You are calculating the binary entropy of NH(0.9, 0.1), rather than NH(0.877, 0.123).
if the 0^N string you mention is the single most probable string, independent of N, shouldn't i get it most of the time. And thus the mode is 0 for the outcome 1. Then the mode and mean are different in this case implying it isnt a gaussian distribution
@@jonathanho5026 the binomial distribution (N, p) can always be thought as a sum of N i.i.d. Bernoulli(p), and since N*p >> 1, then you can apply the central limit theorem, giving you the Gaussian distribution. Btw, i think that if you actually simulate this many times, you would actually see that the ticket 00...0 is the one that gets chosen more often...but N = 1000 is too big, you would requiere many simulations to see something that converges to that result...that Is why the professor speeds up the process taking N=20, it Is faster to see the trend like that.
NH(0.9, 0.1) = 469 is indeed correct, but it is a rough approximation here. You arrive at 531, if you approximate more properly. Look at 41:52 - 42:16, where the lecturer shows, which terms to consider in the general case. The dominant term is the Binomial-Coefficient with highest k, i.e., Bin(N, k = N*f + 3 * sqrt(N*f*(1-f)). (The 3 stands for considering 3 standard deviations). We also learned, that the logarithm of a Binomial-Coefficient can be approximated as follows: log(Bin(N, k)) = N*H(k/N)). [*] So, by using *, we obtain the following approximation: log(Bin(N, k = N*f + sqrt(N*f*(1-f)) = N*H(f + sqrt((f*(1-f))/N). -> Explicitly, we have: N*H(0.1 + 3*sqrt((0.1 * 0.9)/N) = N*H(0.1 + 3*0.095) = 1000 * H(0.1285) = 553 bits Note: You arrive at 531 bits, if you consider a different factor for the standard deviation, e.g. 2 instead of 3.
Revision: Upto 4:00
Sixty Three- 4:00
Submarine- 13:23
Bent Coin Lottery- 21:11
Source Coding Theorem- 48:42
"and we go all the way into the book to page..... 1" Amazing.
I see what you did there :)
Wow, Great Lecture! Thank You!
Please could we get a link to the submarine game? or maybe the source code?
I am a bit troubled about the result of the tickets lottery. I would expect to buy from all tickets from the all-0s ticket up to the (1000 choose 123) tickets to have 99% of winning (so it would rather be => sum of (N choose n) with n goes from 0 to 123). But here, the teacher only considers the (1000 choose 123) tickets which does not represent 99% chance of winning the game. I can't see where I failed to understand this result ... @Jakob Foerster ?
Mean of the binomial = N×p = 100 -> P(X
The answer is 42. #douglasadams
Hi @Jakob or anyone, I have a wired question that for the submarine. if instead of using the probability that each box is not the one = log64/63+log63/62..and things do cancel out, we using the probability that each is the one which would use the probability of 1/64+1/63...and so on. Therefore, log64+log63.... I am assuming each guesses on the box will give the same information. But why is these two different? Could anyone help with this thanks
.
The log64/63+log63/62.. formula assumes there's only one submarine. The 1/64+1/63 formula you proposed would another game where the submarine relocates (without revisiting any location it has stayed at) each time you hit it and you hit it every time (which is extremely lucky).
wow, one project .. some additional exercises and (1-6) chapters to read :)))) Cambridge it is
These lectures are really nice! I would like to know why the string 00000....0000 was the most probable in the 20 bits string? Is this string 00000...00000 the most probable when N=1000? Thanks in advance
0 has a probability of 0.9, while 1 has a probability of 0.1. Therefore the 0^N string you mention is the single most probable string, independent of N.
Another way of thinking is that even thought the probability P(x=00...0)P(x=S, S is one specific string | S has a "1").
Sorry for reviving the post after such a long time haha
As @Jakob was saying, any ticket t has chances P(t) of being selected, where P(t) = 0.9^N_0 * 0.1^N_1 constrained to N_0 + N_1 = N. To maximize P(t), then you should take N_0 = N, and that is why the ticket t = (00...0) is the best ticket for N = 20, N = 1000 or any other value.
As @Gabriel pointed out, it is interesting that even though a particular ticket with N_1 > 0 is a priori more unlikely to be selected, there are so many tickets with N_1 ones, that choosing one of these finishes being more likely than choosing the single one with N_0 = N.
@@julianmartindelfiore7420 I believe I had assumed the equal probability of getting a 0 or a 1. Maybe that was my mistake. I will watch the video again. Thanks!
@@lkcastelano Maybe! Hehe but in any case, your question pointed to an important fact: the reason why this was the correct answer was vaguely discussed, but not formally shown in the video...so i just tried to provide the More detailed explanation in case someone was looking for that :) hehe
I wish he explained why the answers to the sequence of yes or no question turned out to be the number sought (42) in binary.
It's because the fist question is basically asking what the MSB is. For example, if the answer is yes, we know the binary number is: "1xxxxx", where 'x' is still unknown. The next question determines the next digit: "10xxxxx". The last question determines the LSB: "101010".
I think the LSB (even or odd) is easiest to think about intuitively. If you ask if the number is odd, and the answer is "Yes", you know that the number in binary has a '1' as the LSB.
If I bought exclusively the tickets from 2^77 ones to 2^123 I would still get a 99% chance of winning, right?
I expect the approximation of a gaussian distribution to break down since the 0's are more likely than the 1's.
You wouldn't have spent the minimal amount of money so the mafia boss would kill you.
If what you mean is all tickets t such that t has N_1 ones, where 77 1/(1-P_0) or, to say something N * (1 - P_0) = N * P_1 >= 100. Since we have N = 1000 and P_1 = 0.1, then we can use the CLT, thus the gaussian distribution approximation is valid for the example, for any N_0 or N_1 you consider. I think that the answer is more related to what I was saying above.
@@JakobFoerster This is wrong, the limit of the binomial distribution is normal for any p. The asymmetry comes from the fact that buying the tickets from 0-"1s" to ~77-"1s" is drastically cheaper than the rest, and it saves you buying the tickets around 2^~123.
But NH(0.9,0.1)=1000*[log2(10/9) *0.9 + log2(10) *0.1]=469, not 531
What is wrong?
Is 469+531=1000 a coincidence?
Well, he is approximating the log(1000 choose 123) term, not the log( 1000 choose 100) term. You are calculating the binary entropy of NH(0.9, 0.1), rather than NH(0.877, 0.123).
@@bunnywu9726 it is a coin-cidence
That mean i need to buy more than half amounts of tickets to be in the zone of winners
Isnt this the same as lecture 2?
i think there may be an oversufficient entropy issue with the potato you are using to record this. computer science's best minds 👍
@37:52 zishhhh
why the mean is 100 at 0:33:16
You are flipping 1000 biased coins, with a probability of the outcome 1 of 10% for each flip. Average total is 0.1 * 1000 = 100. Does that help?
Jakob Foerster thanks.
if the 0^N string you mention is the single most probable string, independent of N, shouldn't i get it most of the time. And thus the mode is 0 for the outcome 1. Then the mode and mean are different in this case implying it isnt a gaussian distribution
@@jonathanho5026 the binomial distribution (N, p) can always be thought as a sum of N i.i.d. Bernoulli(p), and since N*p >> 1, then you can apply the central limit theorem, giving you the Gaussian distribution. Btw, i think that if you actually simulate this many times, you would actually see that the ticket 00...0 is the one that gets chosen more often...but N = 1000 is too big, you would requiere many simulations to see something that converges to that result...that Is why the professor speeds up the process taking N=20, it Is faster to see the trend like that.
Cambridge chalk
in 63 game, I can ask what is the x mod 64 , and that would be the answer.
Otto Dvalishvili LOL he meant the answer of the question can be only "yes＂ or "no＂.
excellent question LOLOL, hope @Bunny Wu gave a suitable answer to clarify your ques
You could also straight up ask what the number is, but that's not the point of the game.
But NH(0.9,0.1)=1000*[log2(10/9) *0.9 + log2(10) *0.1]=469, not 531
What is wrong?
NH(0.9, 0.1) = 469 is indeed correct, but it is a rough approximation here. You arrive at 531, if you approximate more properly.
Look at 41:52 - 42:16, where the lecturer shows, which terms to consider in the general case. The dominant term is the Binomial-Coefficient with highest k, i.e., Bin(N, k = N*f + 3 * sqrt(N*f*(1-f)). (The 3 stands for considering 3 standard deviations).
We also learned, that the logarithm of a Binomial-Coefficient can be approximated as follows: log(Bin(N, k)) = N*H(k/N)). [*]
So, by using *, we obtain the following approximation:
log(Bin(N, k = N*f + sqrt(N*f*(1-f)) = N*H(f + sqrt((f*(1-f))/N).
-> Explicitly, we have:
N*H(0.1 + 3*sqrt((0.1 * 0.9)/N) = N*H(0.1 + 3*0.095) = 1000 * H(0.1285) = 553 bits
Note: You arrive at 531 bits, if you consider a different factor for the standard deviation, e.g. 2 instead of 3.