The last puzzle has an amazing story IIRC. A composer set it up at either the world championship or just some tournament, and the best players in the world at the time (Botvinik, Kasparov etc.) couldn't solve it. Then the magician from Riga took a look at the board and after some time couldn't solve it. So he went for a walk to ponder the position, When he came back he produced this solution. No wonder engines can't solve it, the best players in the world needed a magician! Edit: Upon rewatching the video I sourced this knowledge from (It's an Agadmator video titled "Grandmasters and Engines couldn't solve it, then came the Magician from Riga") I realized Botvinik was _not_ present at the tournament, however it was still a strong tournament with players like Kasparov, Karpov, Larson and obviously Mikhail Tal. Check out Agadmator's video as it goes through the puzzle in much greater detail as he dedicated an entire video to it :)
All positions analyzed with Stockfish 15.1 to satisfy my own curiosity about whether an engine could solve the puzzles on a better computer. 1. Mate in 8 found at depth 31, starting with Qh5+. 2. Engine still thought this was a draw at depth 72 and over 12 billion nodes searched, regardless of variations. Once Bc3 is played, it found the right continuation immediately before I could record the depth. FAIL 3. Qf6+ found at depth 27, but only by asking for 2+ variations. Not found by depth 60 with one variation. However, it found a defense you didn't mention in the video which is sacrificing the black queen back with Qxf6, exf6, Kxf6 where black barely survives at a +5 evaluation. It immediately notices the mate in 8 after Qf6+, as sacrificing back the queen is the only defense given. 4. Mate in 16 found at depth 126 starting with Kxe1. Not found before depth 126, but it only took about 10 seconds to get there. Much sooner than expected, as mentioned in the video. 5. Forcing line found at depth 25 starting with a +3.7 evaluation, creeping up to +10.13 evaluation by depth 42 and 15 billion nodes. TL;DR most of these puzzles just need a stronger computer or multiple variations to solve. #2 is the only one that could not be solved by the computer without being given the first move.
Tips get the strongest possible analysis, use: - Chess Engine: Stockfish 16 (45MB download) The "lite" version doesn't have NNUE, so it is basically Stockfish 11. - Number of Lines: 1 The more lines you have, the more resources Stockfish has to spend computing the 2nd, 3rd, 4th... best moves, instead of focusing on trying to find just the best one.
actually I found for the second I think it was if I put the lines to a very high number it would spot it instantly but at 1 or any low number it would never find it
Bishop is on the same diagonal as black king, if he moves the pawn white can move king to g7 which is mate, it's check from bishop and king can't move because of white pawn and white king
And for a very intuitive reason: knight jumps go between opposite colored squares. So a knight would always need an even number of jumps to go from a dark square to a dark square.
The 3rd puzzle is ridiculous. There may be 0.001% chance that Stockfish missed the initial Queen sac, but there is 0 chance that it will miss the mate after 6:46! That position is so simple even I, a merely 1200 elo player, can figure it out.
That's the trade off of the pruning. The stockfish is following move branches that look more promising. If you give it more time, it is also more likely to look deeper into good branches than the ones that at first seemed bad. The other problem is that he's using browser version, it runs really slow, which exacerbates the above problem.
Not really. From the video it was quite obvious they had mulitple lines enabled and still the critical lines weren’t found. These lines (branches) are simply removed (pruned) from the tree after a certain depth of being significantly worse.
I've heard that the reason chess engines struggle to find certain check mates is because the calculate one move at a time, places a value on each move and continues down the path of different move orders until they find which path has less risk and more value. When one move initially is very bad in terms of value, like sacrificing your queen, the engine will persue other paths first until they eventually calculates the queen sacrifice. So they will eventually find it, but humans will sometimes find a queen sacrifice faster because of our own interest in calculating a queen sac before other lines.
Stockfish 16.1 NNUE says that this position at 10:38 is +4.0 for white and if Nf7+ is played as in 11:28, Stockfish (at low depth) says that there is mate in 11 (+M11).
Stockfish dev finds the first and third positions near instantly. And the second after a couple minutes. Leela is also very fast on the 5th one. And some more puzzle focused engines like crystal can get all of them. Also you should get off of the lite version. That uses the old HCE (handcrafted eval) that hasn't been updated since before SF12. So it will have less positional understanding than NNUE.
@@JohnDoe-ti2np #5 seems to be the toughest for dev. Ran dev for 15 minutes and 16 for 8 before giving up. #2 seems a lot faster on dev, probably from the removal of HCE.
@@许玄清 You mean #2? With 4GiB hash 1 thread dev-20240211-c115e517 takes 379M to switch to Bc7 and 387M to announce M12. Takes about 3G with 12 threads, though maybe that was unlucky. Previous dev happened to be better here IIRC.
you gotta tell the amount of nodes the engine searched through (or depth), these kind of "run for X minutes" comments are pointless, for all we know you could be running on a 10 year old laptop that costed $200 brand new 10 years ago (and thus would be nearly worthless in 2024)
Latest version of Stockfish WAS able to find the solution to the last one with about 10 minutes of searching. The second longest puzzle was the second one, which it found in a minute or two. All the others were instantaneous.
@@snekkers1609 No, that's not how engines work, and that's what he says in the video. An engine cannot physically check every possible line and every possible move to see if it is good or bad, so it starts by first checking every move, and figuring out what black's likely best response is, and if the move lost a piece or is bad for white in some other way, and white doesn't gain (or doesn't to gain) anything from it, then it will move on and find a better move. So, moves that literally just blunder a queen will be glanced at, then ignored by then engine. A move that sacrifices a queen to expose the enemy king will not be ignored, though, since white does gain something: they now have a safer king than black does. But if the move loses a queen, but does not actually gain anything until like 8 moves later, the engine will probably never see it.
@@justinbombach9873 yes, its pruning some moves, but the moves it prunes are worse then the move your are ending up with, you have move ordering and a bunch of pruning algos for that, it might lose some tactical moves, but it for sure not missing a checkmate
@@justinbombach9873 "But if the move loses a queen, but does not actually gain anything until like 8 moves later, the engine will probably never see it." thats not true, engine studies each depth separatly so it will find the move, it just needs to get to that depth, and remember, its checkmate, so it can get there even in quiescence search. Stockfish get to that depth in less than a second.
The latest Stockfish dev version solves #1-#4 instantly and requires a depth of 49 ply for #5, which could be 10 minutes or more on slower hardware. Stockfish Crystal solves all puzzles instantly (e.g. depth 20 for #5).
The version "Date: Wed Feb 14 21:39:31 2024 +0100 Timestamp: 1707943171" finds Bc7 @ depth 71 on Apple Silicon with 4 parallel threads in about 3 seconds. Maybe the parallel search makes the result indeterministic. With one thread it takes depth 75 and about 52 seconds.
@@thebcwonder4850 If it didn't know them before, and it does know them now, then what does that tell you of whether or not it learned them? Naturally, one might deduce that it is quite likely it did train on those puzzles, wouldn't you say?
The puzzle that mikhail tal solved in the last one the engines cant figure out they can only see when we show them the 3 moves. Mikhail tal was truly an genius
the fact that the chess engine can't even find every possible move pretty much shows that there might still be the most beautiful lines out there that have never been discovered in history, akin to whole continents that are yet to be explored
1) real stockfish solves in 10 seconds 2) real stockfish can't solve, looks like a bug, not a "horizon effect", stockfish does not evaluate this move even on depth 100. But with Syzygy endgame tablebases, it should be able. 3) real stockfish finds in 40s. 6:33 Kg8 is a human mistake, causing M+8. Instead, Qxf6 and game continues (not great for black, but not mate) 4) real stockfish solves in 1 second, even though stockfish is not designed to solve illegal positions 5) real stockfish can't solve Note that "in-browser stockfish lite" is like a "book of good chess ideas", not a full-blown engine. Web-browsers are limited in terms of calculations (not enough memory, limited threading, limited instruction set running in vm).
For number 2, try and set MultiPV higher and higher, I'm on my phone rn so can't try it. My guess is that something like "setoption name MultiPV value 10" or "14" should get it.
latest stockfish dev, nearly 20 mins gone, depth 76, 1 gb hashfile and 5 man syzygy tablebases, still hasn't found #2. No1 uses (or should use) more than MultiPV 1 in a real game, don't be ridiculous.
@@prgnify interesting, with MultiPV=10 Stockfish indeed can see the mate for number 2. But it is kind of cheating, it is a direct hint from human to estimate more first moves instead of relying to solutions tree.
@@kerkertrandov459 I mean, you are correct, but if we are talking about finding "creative" plays, breadth can be interesting, and #2 is a great example.
2 positions that Stockfish 16 can't find on a cheap laptop in a reasonable amount of time, one that takes over 3 minutes, one that takes 12 seconds, and one that takes milliseconds: 1) rnbq3r/pppp4/3b1pkp/5p2/2BP4/4P3/PPP2PPP/RN1QK2R w KQ - Mate in 8 (12 seconds) 2) n1QBq1k1/5p1p/5KP1/p7/8/8/8/8 w - - Still draw after 9 minutes @depth 74! 3) r2q1r2/1b2bpkp/n3p1p1/2ppP1P1/p6R/1PN1BQR1/NPP2P1P/4K3 w - - +3.84 after 42 seconds 4) 8/8/8/2p5/1pp5/brpp4/qpprp2P/1nkbK3 b - - Checkmate in 0.00 seconds! 5) 8/3P3k/n2K3p/2p3n1/1b4N1/2p1p1P1/8/3B4 w - - It takes 3:10 minutes @depth 43
What do you mean two positions? If you mean position 3 then rewatch 5:50. It isn't imminent forced mate because black can sac the queen back instead of Kg8, but you have to see that line to justify Qf6+. So +3.84 means SF saw it. Also I've never seen 5 solved that fast by SF. 2 is solved in a couple minutes by SFdev, probably improved by the removal of handcrafted eval.
do you have 5 piece tablebase (around 1 gb i think)? those are crucial to find the wins. 6 and 7 tablebase are too big and don't help much anyway in terms of positions calculated per sec because they distract the engine, but 5 piece is mandatory.
Maybe someone said this but puzzle 3 isn't a forced mate at the starting position. At 6:40 - 2... Qxf6 leave white slightly ahead, but definitely not mating (following 3. Bh6+ Kg8 4. exf6. And stockfish picks Qxf6 over Kh8 or Kg8 because both are mate in 8. Hardly a "checkmate engines can't find" when it's not a forced mate at all. The last puzzle is really cool because you can see how happy stockfish 16 is even for a second after 3. Bc3. You can see it briefly scan the position after black's second move, and dismiss it. And after the move, the evaluation for black is like +7 for a few cycles before dropping to -3.5. It's also interesting because leela more accurately discovers Nf7+ form the outset, whereas stockfish wants white to play something like Nxe3.
1st puzzle : lichess board analysis found mate in 8 with white 2nd puzzle : same but mate in 12 3rd puzzle : if you limit depth it says +1 or +2, but if you increase it, engine finds a draw if both player play perfectly. ( not a forced mate as you said ) 4th puzzle : engine find mate in 16. 5th puzzle : as you said, not a forced mate. +7.2 so it must be mate in 30 or more. Chess are just way better than us. If there is a problem we can solve and an engine can't, it has to be more than 20 moves, with a lot a playables moves each time, but easy enough for a human. good luck.
Engines rate moves based on some huerustics. Then search along the best rated moves first. Losing your queen is seen as bad, so that move has a low priority to be looked at. Depending on the engine, some may never look at certain moves, and thus there are moves that it can't find.
I really enjoyed this video. When I tested both Stockfish and Leela, I found that Leela often solved these puzzles faster than Stockfish. Although Stockfish would likely win in a direct competition, Leela seems to have a more intuitive grasp of these “unsolvable engine puzzles.” I’m using a stronger Leela neural network rather than the default, which I find fascinating. To elaborate, I believe Leela has a better understanding of the horizon effect because it was trained through self-play, similar to AlphaZero. While Leela doesn’t reach high depths quickly (it’s actually one of the slowest in depth progression), its self-play training has taught it many positions and their outcomes, even at lower depths. This makes these unsolvable puzzles easier for Leela compared to Stockfish, as Leela’s approach is less brute-force and more akin to human understanding. Edit: To add, here is how long it took Leela to solve each puzzle: 1. Instant 2. About 1 minute 3. Instant 4. 5 seconds to find the first move but doesn’t find h3 5. About 1-2 minutes (every engine takes a while on that last puzzle; Tal will always be remembered for solving it) In case anyone is wondering about my specs, they’re not particularly impressive, but it’s impressive for Leela to solve them so quickly considering the specs that one comment used for Stockfish: 16GB DDR4 RAM AMD Ryzen 5 4500 6-core processor at 4.2GHz RTX 3050 8GB VRAM Hope this helps anyone!
Because Qf2 leads to mate in 2 for white. White can go Kg5 or Ke7, then the bishop diagonal is opened up. If black captures the bishop, then f8=Q# will happen.
I only checked the first two. With Stockfish 16.1 I could find the first one however it failed with the 2nd one. I also have Komodo installed and it could find the 2nd check mate in a couple of seconds while Stockfish only could find it after giving it the first move. I really wonder why that is. It feels like a problem Stockfish should be able to solve, given that it is the strongest engine and has won so many titles ... when Komodo can find the solution in a couple of seconds (probably less, it was so fast, I had no time to really pay attention), then to me this looks like something in the Stockfish code could be improved. Any1 knows more about this issue and can explain a bit?
@@kerkertrandov459 Big Transformer 4. It's a network run. New network architecture which builds off of BT3 by adding two types of auxiliary heads, future heads and categorical value heads. The categorical value heads predict a distribution over values of q rather than a WDL outcome distribution, and the future heads predict the moves that will be played over the next two plies. RTX3080 here but even with a weaker GPU you're bound to get the solution with a very low node count.
Materialism seems to be a theme with these puzzles and Leela doesn't know the values of pieces so it's immune to such trickery. Many of her victories over Stockfish are long lines down material that Stockfish can't figure out.
Kf5 discovered check, queen has to block, then we promote the pawn to a queen or even a rook and it is checkmate(the queen can't take because it's pinned).
I am aware that it is sometimes difficult to introduce chess to people who think it has in some ways being solved. But I don't think the solution is being missleading about engines. Stockfish properly setup has no problem with any of these.
5:32 if we give stokfish plenty of time, and he analyse all the thing he decide to analyse ( maximum depth ) like on the second example, i'm sure he could find everything because of 50 move rule, and there a not realy a bunch a legal move per ply, so if he finish all he need to do and still have time, does he upgrade the number of ply before dissmising a possibility, or he just wait and do nothing ?🤔🤔
The second puzzle is a perfect example of engines not understanding a concept. This is because computers don't understand anything, they jjust ... compute.
The whole idea that humans can still play SOME chess better than computers is amazing. They should have used this idea in one of the Terminator movies (or Matrix movies) as the mechanism to get humanity to beat the robot overlords.
Unfortunately this video is false I used stockfish it found the solutions of 1-4 instantly and as for 5 is black plays perfectly the game can be drawn so it isn't a forced mate
So he says if you force the engine to analize a certain line, it will find the winning sequence, but just before he played the rook sacrifice the engine already considered it the best move, which means it has to have looked at that line in more depth. And after he played it, the engine still thought it was better for black. There were no more obscure sacrifices after that, so that's the only one that got me like, hmm... a bit sus.
atoms in observable universe are much more than legal and possible to reach positions in chess for example a king cant castle after moving already and ao on
Nice puzzles! That said, my quite old version of Leela Chess Zero running on a 2020 laptop was able to solve #1 and 3 in under 15 seconds, and #5 in about 2 minutes. Stockfish is not the best engine around imo.
TBH you are using stockish lite (in browser) which is about 40% of the capability of the full stock fish and slower for it to work in browsers. I tried all these puzzles in full stockfish and komodo (both using GPU acceleration), both of them solved them seconds. Only leela couldnt solve all of them, but since leela is an AI you have to teach it first and then it will solve it.
No "Horizon Effect" - the engine doesn't find the second "solution" because is not a forced solution. What if you keep attacking the bishop with the queen? Nice study idea, but It's just a draw.
Nuerul network engines wouldnt have hard time finding it but probably engines like stockfish 8 would since they dont consider piece sacrifice as most options. Its just analysis of 10 million positions at time, without any thoughts.
The explanation for why engines don't see the second position is quite naive. I think the real reason has to do with a heuristic called "null-move pruning", which is known to make it harder to find tactics that involve Zugswang. In this case it involves it many times in a row, and the heuristic makes the engine basically never see it. It turns out the null-move heuristic makes programs much stronger in general, so that's why most engines based on alpha-beta search use it.
In 3:17 ther queen f2 forking the king and the bishop which loses a bishop and white would lose after the queen takes the advanced pawn then the a file pa2n is unstoppable and black wins
For me in first puzzle, both Torch and SF16 get mate instant. Second was both broken. Third Torch get Qf6+ move, SF16 nope, but Torch moved Qxf6 not Kg8 that prevent mate. Fourth - Torch didn't get mate, SF16 get mate in 16 in an instant. Fifth - Torch didn't get your move Nf6+, SF16 get it but didn't go for fork Nf7+. I was using full version of SF16 not lite.
I don’t get how the Horizon Effect works in those particular cases. For instance, if we take the 2nd example, white have « only » 18 possible moves. How come the engine cannot calculate the continuation for these only 18 possible moves ?
stockfish : who is stronger n1QBq1k1/5p1p/5KP1/p7/8/8/8/8 w - - 0 1 or you komodo : well he might cause me a little trouble stockfish but would would you lose komodo : nah i'd win
Because the engine doesn't analyze each move equally deeply. Bad moves are often skipped, and since in zugzwang all moves are bad, zugzwangs are really confusing for engines. Especially long ones, like in puzzle #2.
For anyone wondering. Basically if an engine thought the move is just too dumb (sacrificing queen without any determined compesation), why bother calculating it.
@@deanwilliams433 a sacrifice is calculated if there is compensation (including positional compensation). Engine does not calculate dumb sacrifice if it yield nothing, in this case, either positional or material compensation. this is always happen if the possibility of moves is beyond engine limit (around 10 million board per second). Engine will throw away those dumb sacrifice
@@brukujinbrokujin7802 I think this is kind of misleading, the engine still looks at the move very deeply; otherwise it wouldn't have found the solution regardless how long it thought. However, for most of these puzzles it takes 7 to 8 moves for the advantage to reveal itself. This causes the engine to put off searching line and prioritizing an another before they can pass the 'horizon'. I don't think it is so dumb to actually discard without looking at any of the move shown in the video. It probably didn't allocate enough resources into searching the move.
The last puzzle has an amazing story IIRC.
A composer set it up at either the world championship or just some tournament, and the best players in the world at the time (Botvinik, Kasparov etc.) couldn't solve it.
Then the magician from Riga took a look at the board and after some time couldn't solve it. So he went for a walk to ponder the position,
When he came back he produced this solution. No wonder engines can't solve it, the best players in the world needed a magician!
Edit: Upon rewatching the video I sourced this knowledge from (It's an Agadmator video titled "Grandmasters and Engines couldn't solve it, then came the Magician from Riga") I realized Botvinik was _not_ present at the tournament, however it was still a strong tournament with players like Kasparov, Karpov, Larson and obviously Mikhail Tal. Check out Agadmator's video as it goes through the puzzle in much greater detail as he dedicated an entire video to it :)
i aint readin allat
@@Supercharge7868 durr atentin span to lo to red 4 one mint
Stockfish can solve them lol , but need depth +80
All positions analyzed with Stockfish 15.1 to satisfy my own curiosity about whether an engine could solve the puzzles on a better computer.
1. Mate in 8 found at depth 31, starting with Qh5+.
2. Engine still thought this was a draw at depth 72 and over 12 billion nodes searched, regardless of variations. Once Bc3 is played, it found the right continuation immediately before I could record the depth. FAIL
3. Qf6+ found at depth 27, but only by asking for 2+ variations. Not found by depth 60 with one variation. However, it found a defense you didn't mention in the video which is sacrificing the black queen back with Qxf6, exf6, Kxf6 where black barely survives at a +5 evaluation. It immediately notices the mate in 8 after Qf6+, as sacrificing back the queen is the only defense given.
4. Mate in 16 found at depth 126 starting with Kxe1. Not found before depth 126, but it only took about 10 seconds to get there. Much sooner than expected, as mentioned in the video.
5. Forcing line found at depth 25 starting with a +3.7 evaluation, creeping up to +10.13 evaluation by depth 42 and 15 billion nodes.
TL;DR most of these puzzles just need a stronger computer or multiple variations to solve. #2 is the only one that could not be solved by the computer without being given the first move.
what is yours pcs config?
Komodo can solve the 2
Rems engine can solve #2
Tips get the strongest possible analysis, use:
- Chess Engine: Stockfish 16 (45MB download)
The "lite" version doesn't have NNUE, so it is basically Stockfish 11.
- Number of Lines: 1
The more lines you have, the more resources Stockfish has to spend computing the 2nd, 3rd, 4th... best moves, instead of focusing on trying to find just the best one.
another tip: use lichess since you can allocate more of your CPU and memory
Yes in general. Although in this case high number of lines could help if it looks deeper into the correct line.
actually I found for the second I think it was if I put the lines to a very high number it would spot it instantly but at 1 or any low number it would never find it
no..@@George_Bland
this should be obvious
The second one is probably the most beautiful puzzle i have ever seen
I was trying to find it for about 5 minutes and I was waaaaay off the right path.
@@milke9254I don’t know how you’d ever discover that move. It’s incredible though!
3 pieces and if you move any of those 3 pieces where ever, then you lose. WTF.
I am not a strong player. But i have a doubt. In the second one why didn't black just move the pawn in front of the king which is keeping it blocked?
Bishop is on the same diagonal as black king, if he moves the pawn white can move king to g7 which is mate, it's check from bishop and king can't move because of white pawn and white king
Crazy how in the 4th puzzle the knight can’t return to A5 after an odd number of moves
And for a very intuitive reason: knight jumps go between opposite colored squares. So a knight would always need an even number of jumps to go from a dark square to a dark square.
And funnily the queen in this puzzle can also only switch between a light and dark square
And that's why bishops are better than knights
I disagree😂
@@thabomokhaneli6992There's nothing to disagree with. OP didn't even have a point.
Nuh uh
The horsey on the the fourth puzzle completely destroyed the black kingdom though.,,I think this is a tie😂
Bishop control more square
The 3rd puzzle is ridiculous. There may be 0.001% chance that Stockfish missed the initial Queen sac, but there is 0 chance that it will miss the mate after 6:46! That position is so simple even I, a merely 1200 elo player, can figure it out.
you're full of shit, no way you as a 1200 could find mate there without cheating
That's the trade off of the pruning. The stockfish is following move branches that look more promising. If you give it more time, it is also more likely to look deeper into good branches than the ones that at first seemed bad.
The other problem is that he's using browser version, it runs really slow, which exacerbates the above problem.
Stockfish essentially only doesn't find these because only one line is enabled. Enable more lines, and stockfish again can find the solutions.
Not really. From the video it was quite obvious they had mulitple lines enabled and still the critical lines weren’t found. These lines (branches) are simply removed (pruned) from the tree after a certain depth of being significantly worse.
10:33 It is a forced mating line, but mate in 32 moves or less instead of 15 moves.
I've heard that the reason chess engines struggle to find certain check mates is because the calculate one move at a time, places a value on each move and continues down the path of different move orders until they find which path has less risk and more value. When one move initially is very bad in terms of value, like sacrificing your queen, the engine will persue other paths first until they eventually calculates the queen sacrifice. So they will eventually find it, but humans will sometimes find a queen sacrifice faster because of our own interest in calculating a queen sac before other lines.
Stockfish 16.1 NNUE says that this position at 10:38 is +4.0 for white and if Nf7+ is played as in 11:28, Stockfish (at low depth) says that there is mate in 11 (+M11).
"Humans love to sacrifice queens " 😂🤣😂🤣😂
Stockfish dev finds the first and third positions near instantly. And the second after a couple minutes. Leela is also very fast on the 5th one. And some more puzzle focused engines like crystal can get all of them.
Also you should get off of the lite version. That uses the old HCE (handcrafted eval) that hasn't been updated since before SF12. So it will have less positional understanding than NNUE.
Stockfish 16 on my laptop can't solve Problem 2 after 20 minutes. That seems to be the toughest one of the bunch.
@@JohnDoe-ti2np #5 seems to be the toughest for dev. Ran dev for 15 minutes and 16 for 8 before giving up. #2 seems a lot faster on dev, probably from the removal of HCE.
@@许玄清 You mean #2? With 4GiB hash 1 thread dev-20240211-c115e517 takes 379M to switch to Bc7 and 387M to announce M12. Takes about 3G with 12 threads, though maybe that was unlucky. Previous dev happened to be better here IIRC.
you gotta tell the amount of nodes the engine searched through (or depth), these kind of "run for X minutes" comments are pointless, for all we know you could be running on a 10 year old laptop that costed $200 brand new 10 years ago (and thus would be nearly worthless in 2024)
@@JohnDoe-ti2np Komodo on my laptop finds the solution in less than 2 seconds
Latest version of Stockfish WAS able to find the solution to the last one with about 10 minutes of searching. The second longest puzzle was the second one, which it found in a minute or two. All the others were instantaneous.
Lichess engine found them immediately.
obviously engine will find mate in 8 if it reaches depth 16, if it doesnt then its poorly coded, idk what this vid is about
Right? This video seems absurd. How can an engine not calculate a forced mate in 8? Makes no sense.@@snekkers1609
@@snekkers1609 No, that's not how engines work, and that's what he says in the video. An engine cannot physically check every possible line and every possible move to see if it is good or bad, so it starts by first checking every move, and figuring out what black's likely best response is, and if the move lost a piece or is bad for white in some other way, and white doesn't gain (or doesn't to gain) anything from it, then it will move on and find a better move. So, moves that literally just blunder a queen will be glanced at, then ignored by then engine. A move that sacrifices a queen to expose the enemy king will not be ignored, though, since white does gain something: they now have a safer king than black does. But if the move loses a queen, but does not actually gain anything until like 8 moves later, the engine will probably never see it.
@@justinbombach9873 yes, its pruning some moves, but the moves it prunes are worse then the move your are ending up with, you have move ordering and a bunch of pruning algos for that, it might lose some tactical moves, but it for sure not missing a checkmate
@@justinbombach9873 "But if the move loses a queen, but does not actually gain anything until like 8 moves later, the engine will probably never see it." thats not true, engine studies each depth separatly so it will find the move, it just needs to get to that depth, and remember, its checkmate, so it can get there even in quiescence search. Stockfish get to that depth in less than a second.
The latest Stockfish dev version solves #1-#4 instantly and requires a depth of 49 ply for #5, which could be 10 minutes or more on slower hardware. Stockfish Crystal solves all puzzles instantly (e.g. depth 20 for #5).
I just tried latest stockfish dev, currently at depth 75 after 15 mins and still says gxh7, and i'm using 1 gb hashfile + 5 man syzygy tablebase.
The version "Date: Wed Feb 14 21:39:31 2024 +0100 Timestamp: 1707943171" finds Bc7 @ depth 71 on Apple Silicon with 4 parallel threads in about 3 seconds. Maybe the parallel search makes the result indeterministic. With one thread it takes depth 75 and about 52 seconds.
Every one of these "the engine can't find this" I have ever seen has been wrong, and Stockfish found it in a second or less.
I'm guessing that's because stockfish learned those puzzles
@@9nikola unless Stockfish trained on those puzzles, it hasn’t learned them
@@thebcwonder4850 If it didn't know them before, and it does know them now, then what does that tell you of whether or not it learned them? Naturally, one might deduce that it is quite likely it did train on those puzzles, wouldn't you say?
@@9nikola option #2: Stockfish 16 is a lot stronger when not stuck in a web browser "lite" version
@@9nikola Stockfish doesn't work that way
The puzzle that mikhail tal solved in the last one the engines cant figure out they can only see when we show them the 3 moves. Mikhail tal was truly an genius
So all those blunders which I had in game analysis were briliant sacrifices with winning idea far away for engine to see.
the fact that the chess engine can't even find every possible move pretty much shows that there might still be the most beautiful lines out there that have never been discovered in history, akin to whole continents that are yet to be explored
2nd one is insanely amazing
00:19 yeah bro thats stockfish 16 lite... like 200 elo lower than regular
I tried it with the latest stockfish on my own computer and it took 40 seconds to find Qh5
1) real stockfish solves in 10 seconds
2) real stockfish can't solve, looks like a bug, not a "horizon effect", stockfish does not evaluate this move even on depth 100. But with Syzygy endgame tablebases, it should be able.
3) real stockfish finds in 40s. 6:33 Kg8 is a human mistake, causing M+8. Instead, Qxf6 and game continues (not great for black, but not mate)
4) real stockfish solves in 1 second, even though stockfish is not designed to solve illegal positions
5) real stockfish can't solve
Note that "in-browser stockfish lite" is like a "book of good chess ideas", not a full-blown engine. Web-browsers are limited in terms of calculations (not enough memory, limited threading, limited instruction set running in vm).
For number 2, try and set MultiPV higher and higher, I'm on my phone rn so can't try it. My guess is that something like "setoption name MultiPV value 10" or "14" should get it.
5) real stockfish can't solve but.... Magician of Riga, the painter, the artist, the poet of chess Mikhail Tal did...
latest stockfish dev, nearly 20 mins gone, depth 76, 1 gb hashfile and 5 man syzygy tablebases, still hasn't found #2. No1 uses (or should use) more than MultiPV 1 in a real game, don't be ridiculous.
@@prgnify interesting, with MultiPV=10 Stockfish indeed can see the mate for number 2. But it is kind of cheating, it is a direct hint from human to estimate more first moves instead of relying to solutions tree.
@@kerkertrandov459 I mean, you are correct, but if we are talking about finding "creative" plays, breadth can be interesting, and #2 is a great example.
2 positions that Stockfish 16 can't find on a cheap laptop in a reasonable amount of time, one that takes over 3 minutes, one that takes 12 seconds, and one that takes milliseconds:
1) rnbq3r/pppp4/3b1pkp/5p2/2BP4/4P3/PPP2PPP/RN1QK2R w KQ - Mate in 8 (12 seconds)
2) n1QBq1k1/5p1p/5KP1/p7/8/8/8/8 w - - Still draw after 9 minutes @depth 74!
3) r2q1r2/1b2bpkp/n3p1p1/2ppP1P1/p6R/1PN1BQR1/NPP2P1P/4K3 w - - +3.84 after 42 seconds
4) 8/8/8/2p5/1pp5/brpp4/qpprp2P/1nkbK3 b - - Checkmate in 0.00 seconds!
5) 8/3P3k/n2K3p/2p3n1/1b4N1/2p1p1P1/8/3B4 w - - It takes 3:10 minutes @depth 43
What do you mean two positions? If you mean position 3 then rewatch 5:50. It isn't imminent forced mate because black can sac the queen back instead of Kg8, but you have to see that line to justify Qf6+. So +3.84 means SF saw it.
Also I've never seen 5 solved that fast by SF. 2 is solved in a couple minutes by SFdev, probably improved by the removal of handcrafted eval.
Thank you so much! This comment should be pinned
do you have 5 piece tablebase (around 1 gb i think)? those are crucial to find the wins. 6 and 7 tablebase are too big and don't help much anyway in terms of positions calculated per sec because they distract the engine, but 5 piece is mandatory.
Maybe someone said this but puzzle 3 isn't a forced mate at the starting position. At 6:40 - 2... Qxf6 leave white slightly ahead, but definitely not mating (following 3. Bh6+ Kg8 4. exf6. And stockfish picks Qxf6 over Kh8 or Kg8 because both are mate in 8.
Hardly a "checkmate engines can't find" when it's not a forced mate at all.
The last puzzle is really cool because you can see how happy stockfish 16 is even for a second after 3. Bc3. You can see it briefly scan the position after black's second move, and dismiss it. And after the move, the evaluation for black is like +7 for a few cycles before dropping to -3.5. It's also interesting because leela more accurately discovers Nf7+ form the outset, whereas stockfish wants white to play something like Nxe3.
The video already stated that it isn't a forced mate, just fyi
isnt that third example a leela game from cccc?
1st puzzle : lichess board analysis found mate in 8 with white
2nd puzzle : same but mate in 12
3rd puzzle : if you limit depth it says +1 or +2, but if you increase it, engine finds a draw if both player play perfectly. ( not a forced mate as you said )
4th puzzle : engine find mate in 16.
5th puzzle : as you said, not a forced mate. +7.2 so it must be mate in 30 or more.
Chess are just way better than us.
If there is a problem we can solve and an engine can't, it has to be more than 20 moves, with a lot a playables moves each time, but easy enough for a human. good luck.
Fortresses tend to be easier for humans to handle than engines.
Now we know that another variant of Mobius is just a chess presenter
How can an engine not calculate a forced mate in 8? Makes no sense
Engines rate moves based on some huerustics. Then search along the best rated moves first. Losing your queen is seen as bad, so that move has a low priority to be looked at.
Depending on the engine, some may never look at certain moves, and thus there are moves that it can't find.
Either LC0 or SF16 find solution in seconds in EVERY puzzle here
This goes to show that Tal on his day could crush even Stockfish ...
I really enjoyed this video. When I tested both Stockfish and Leela, I found that Leela often solved these puzzles faster than Stockfish. Although Stockfish would likely win in a direct competition, Leela seems to have a more intuitive grasp of these “unsolvable engine puzzles.” I’m using a stronger Leela neural network rather than the default, which I find fascinating.
To elaborate, I believe Leela has a better understanding of the horizon effect because it was trained through self-play, similar to AlphaZero. While Leela doesn’t reach high depths quickly (it’s actually one of the slowest in depth progression), its self-play training has taught it many positions and their outcomes, even at lower depths. This makes these unsolvable puzzles easier for Leela compared to Stockfish, as Leela’s approach is less brute-force and more akin to human understanding.
Edit: To add, here is how long it took Leela to solve each puzzle:
1. Instant
2. About 1 minute
3. Instant
4. 5 seconds to find the first move but doesn’t find h3
5. About 1-2 minutes (every engine takes a while on that last puzzle; Tal will always be remembered for solving it)
In case anyone is wondering about my specs, they’re not particularly impressive, but it’s impressive for Leela to solve them so quickly considering the specs that one comment used for Stockfish:
16GB DDR4 RAM
AMD Ryzen 5 4500 6-core processor at 4.2GHz
RTX 3050 8GB VRAM
Hope this helps anyone!
3:25 why not Qf2??
Because Qf2 leads to mate in 2 for white. White can go Kg5 or Ke7, then the bishop diagonal is opened up. If black captures the bishop, then f8=Q# will happen.
I only checked the first two. With Stockfish 16.1 I could find the first one however it failed with the 2nd one. I also have Komodo installed and it could find the 2nd check mate in a couple of seconds while Stockfish only could find it after giving it the first move. I really wonder why that is. It feels like a problem Stockfish should be able to solve, given that it is the strongest engine and has won so many titles ... when Komodo can find the solution in a couple of seconds (probably less, it was so fast, I had no time to really pay attention), then to me this looks like something in the Stockfish code could be improved. Any1 knows more about this issue and can explain a bit?
So the computer doesn't consider the first move if seems stupid?
Spenny Thompson is the best!
Just increasing depth allows to find the answer instantly...
Wow they were all nice, but that last one may be the nicest thing I've ever seen on a chessboard. Great video, more of these plz
Chessmaster 10th edition (2004) found the first checkmate in 11 min. and 2 giga nodes.
Leela BT4 solves these puzzles instantly.
@@许玄清 Leela can. Do you need a screenshot? not even 2 seconds of search
wtf is bt4, also whats ur gpu
@@kerkertrandov459 Big Transformer 4. It's a network run. New network architecture which builds off of BT3 by adding two types of auxiliary heads, future heads and categorical value heads. The categorical value heads predict a distribution over values of q rather than a WDL outcome distribution, and the future heads predict the moves that will be played over the next two plies.
RTX3080 here but even with a weaker GPU you're bound to get the solution with a very low node count.
Materialism seems to be a theme with these puzzles and Leela doesn't know the values of pieces so it's immune to such trickery. Many of her victories over Stockfish are long lines down material that Stockfish can't figure out.
3:47 Zugzwang with a bQ and a wB. Beautiful! (Zugzwangs with line-pieces which are not trapped are rare.)
The last puzzle 🧩 was my favorite.
And that’s why HCE engines are outdated
In the second puzzle, can't Black just play Qf8 after we play Ba1?🤔☝🏻
Kf5 discovered check, queen has to block, then we promote the pawn to a queen or even a rook and it is checkmate(the queen can't take because it's pinned).
I am aware that it is sometimes difficult to introduce chess to people who think it has in some ways being solved. But I don't think the solution is being missleading about engines. Stockfish properly setup has no problem with any of these.
Damn I'm going to show these to my friends at the bar and at the library
Stockfish: I can't solve this with all my intelligence.
Some former monkey: But what if you play stupid instead?
Stockfish: Ah yes, now I see it.
4:13
doesnt Qf2+ just win the bishop?
ahhh but it's a queen promotion mate that follows regardless of where the king moves. ahhh i see
5:32 if we give stokfish plenty of time, and he analyse all the thing he decide to analyse ( maximum depth ) like on the second example, i'm sure he could find everything because of 50 move rule, and there a not realy a bunch a legal move per ply, so if he finish all he need to do and still have time,
does he upgrade the number of ply before dissmising a possibility, or he just wait and do nothing ?🤔🤔
This is the most incredible thing I've seen in chess lately.
Thought so too but after reading the comments, engines will find all these solutions if the settings are correct
The second puzzle is a perfect example of engines not understanding a concept. This is because computers don't understand anything, they jjust ... compute.
Bro I used stockfish and it found the solution within 5 mins
@@varadsalunkhe9008
Ok?
I recall the 2nd puzzle is something Danya once showed
this is an interesting case of p vs np completeness
The whole idea that humans can still play SOME chess better than computers is amazing. They should have used this idea in one of the Terminator movies (or Matrix movies) as the mechanism to get humanity to beat the robot overlords.
Unfortunately this video is false I used stockfish it found the solutions of 1-4 instantly and as for 5 is black plays perfectly the game can be drawn so it isn't a forced mate
Pour le 2e mat, j'ai mis la position dans Lichess avec le réglage par défaut, il a trouvé mat en 12 en moins d'une seconde
So he says if you force the engine to analize a certain line, it will find the winning sequence, but just before he played the rook sacrifice the engine already considered it the best move, which means it has to have looked at that line in more depth. And after he played it, the engine still thought it was better for black. There were no more obscure sacrifices after that, so that's the only one that got me like, hmm... a bit sus.
In that first position, what if the black hing moves to g7 (and then f8)? Seems like there is no quick mate.
atoms in observable universe are much more than legal and possible to reach positions in chess for example a king cant castle after moving already and ao on
6:35 what if the queen takes the pawn instead of king moves? White is better but there's no forced mate anymore (or at least I can't find one)
first pazzle: What if instead of "bishop B5 check" black will play "bishop take h2"?
second puzzle: what if white move their bishop to B6?
It would be nice if you would include the FENs of any positions you reference.
Nice puzzles!
That said, my quite old version of Leela Chess Zero running on a 2020 laptop was able to solve #1 and 3 in under 15 seconds, and #5 in about 2 minutes. Stockfish is not the best engine around imo.
This doesn't seem to be the case with today's powerful engines
2:47 black can just move the pawnand get the king to escape (2nd puzzle)
Wow, I rarely like videos on UA-cam but this one deserves a big fat like
Very nice, concise video and great presentation as well
It’s just clickbait Stockfish finds it
0:54 and he sacrificed...
THE QUEEEEEEN
What about Qf2+ 3:20
The last puzzle was also analyzed by levi😂
Can't they make the engines to not prune their lines sooner to avoid horizon blindness?
3rd one is literally how i play chess (1100 elo)
TBH you are using stockish lite (in browser) which is about 40% of the capability of the full stock fish and slower for it to work in browsers.
I tried all these puzzles in full stockfish and komodo (both using GPU acceleration), both of them solved them seconds. Only leela couldnt solve all of them, but since leela is an AI you have to teach it first and then it will solve it.
No "Horizon Effect" - the engine doesn't find the second "solution" because is not a forced solution. What if you keep attacking the bishop with the queen?
Nice study idea, but It's just a draw.
Nuerul network engines wouldnt have hard time finding it but probably engines like stockfish 8 would since they dont consider piece sacrifice as most options. Its just analysis of 10 million positions at time, without any thoughts.
those are some beautiful checkmates
3rd one is actually not the forced mate - so as long as after like 2 minutes Lichess shows +4, it's correct. Black queen can sacrifice for the pawn!
He did say that. Did you miss the disclaimer?
@@kzkaa. yes, missed that
@@pavelcelba6976 Happen to all of us, so don't worry.
Is the first example forced mate or is it just hope chess and the engine wouldn't accept the sacrifice?
last one solved by only Tal
10:26 Kf2 Qa1 Nxb3#
The explanation for why engines don't see the second position is quite naive. I think the real reason has to do with a heuristic called "null-move pruning", which is known to make it harder to find tactics that involve Zugswang. In this case it involves it many times in a row, and the heuristic makes the engine basically never see it. It turns out the null-move heuristic makes programs much stronger in general, so that's why most engines based on alpha-beta search use it.
In 3:17 ther queen f2 forking the king and the bishop which loses a bishop and white would lose after the queen takes the advanced pawn then the a file pa2n is unstoppable and black wins
For me in first puzzle, both Torch and SF16 get mate instant. Second was both broken. Third Torch get Qf6+ move, SF16 nope, but Torch moved Qxf6 not Kg8 that prevent mate. Fourth - Torch didn't get mate, SF16 get mate in 16 in an instant. Fifth - Torch didn't get your move Nf6+, SF16 get it but didn't go for fork Nf7+. I was using full version of SF16 not lite.
Nice puzzles, but please stop hiding the board with analysis windows, or by removing it from the screen.
I love this
Just made my day
I don’t get how the Horizon Effect works in those particular cases. For instance, if we take the 2nd example, white have « only » 18 possible moves. How come the engine cannot calculate the continuation for these only 18 possible moves ?
12:34 still g4# but this man didnt find it.
stockfish : who is stronger n1QBq1k1/5p1p/5KP1/p7/8/8/8/8 w - - 0 1 or you
komodo : well he might cause me a little trouble
stockfish but would would you lose
komodo : nah i'd win
Um, well, technically, since there are only a finite number of positions, if you dont limit depth and don't alpha prune you can find all of these
I have some ones i invented that the engines also cant solve
way too dank chess puzzles
This was cool, thank you!
11:55 g4#
Why can't the computer find a mate under 18 moves when you have set the settings to 18 lines deep?
Because the engine doesn't analyze each move equally deeply. Bad moves are often skipped, and since in zugzwang all moves are bad, zugzwangs are really confusing for engines. Especially long ones, like in puzzle #2.
King moves... discovered checkmate😅
For anyone wondering. Basically if an engine thought the move is just too dumb (sacrificing queen without any determined compesation), why bother calculating it.
That's not how the engines work. They will still evaluate a sacrifice to a certain depth.
@@deanwilliams433 a sacrifice is calculated if there is compensation (including positional compensation). Engine does not calculate dumb sacrifice if it yield nothing, in this case, either positional or material compensation. this is always happen if the possibility of moves is beyond engine limit (around 10 million board per second). Engine will throw away those dumb sacrifice
@@brukujinbrokujin7802 I think this is kind of misleading, the engine still looks at the move very deeply; otherwise it wouldn't have found the solution regardless how long it thought. However, for most of these puzzles it takes 7 to 8 moves for the advantage to reveal itself. This causes the engine to put off searching line and prioritizing an another before they can pass the 'horizon'. I don't think it is so dumb to actually discard without looking at any of the move shown in the video. It probably didn't allocate enough resources into searching the move.
1:08 actually that’s king h4, not h8…..
3:15 "Why not a1? You'll see in a second." I'm afraid I don't see it. 😞
If Ba1, the trick doesn't work because it has to eventually move to b2 where Black's a-pawn can capture it.
Awesome video. Could’ve done without the last one considering it’s cooked.
On the 2nd puzzle why cant queen just keep checking king
what did u mean the third problem doesnt lead to a forced mate?