I Hacked Stockfish's Code To Learn How It Thinks (Part 1)

Поділитися
Вставка
  • Опубліковано 27 лис 2022
  • What is FEN?
    www.chess.com/terms/fen-chess...
    👉🏽This will take you from 1000 to 1500 📈
    🎓 chessvibescourses.thinkific.c... 🎓 Reviews: Overwhelmingly Positive (32 reviews)
    ✅ My Courses! ✅
    chessvibescourses.thinkific.com/
    The chess set behind me:
    The Pieces: bit.ly/3zBJzlO
    The Board: bit.ly/3b3B8pb
    👕 New Shirts! 👕
    Chessvibes.shop
    ☑️ Play Chess Here:
    www.chess.com/membership?ref_...
    ☑️ Support Nelson Here:
    paypal.me/ChessVibes
    Links are affiliate links and help support the Chess Vibes channel via a commission.
  • Навчання та стиль

КОМЕНТАРІ • 141

  • @sadfox9294
    @sadfox9294 Рік тому +252

    CS major here, the benchmark file is for testing, the listed position are likely test cases, where you expect the engine to behave in a certain way or to find a certain move, or you're just interested in how it behaves in that position.

    • @AbelShields
      @AbelShields Рік тому +20

      I mean, I'd imagine it's probably for benchmarking, I know stockfish devs are very concerned with performance regressions

    • @andreib1234
      @andreib1234 Рік тому +3

      on top of that it appears to also take in potentially other FEN starting position and which already have a predetermined outcome and evaluates the system versus those using the file CURRENT

    • @crispiervase2999
      @crispiervase2999 Рік тому +3

      This would make sense. With the name of Benchmarks, performance testing would be reasonable. It might even be a speed test for loading FENs.

    • @ZuppaD.Cipolle
      @ZuppaD.Cipolle Рік тому +1

      it's a way to get a nodecount (the number of nodes the engine searches) under certain circumstances (the default is a search at depth 13 i think), it can be used for
      1) having a unique identifier for a specific version of the engine
      2) test the engine speed on a specific cpu to scale fishtest workers

    • @greengoblin9567
      @greengoblin9567 Рік тому +2

      They basically reject a stockfish development build if it fails any of these tests. It is for curation.

  • @RikMaxSpeed
    @RikMaxSpeed Рік тому +65

    StockFish’s evaluation function has been replaced with the efficiently updatable neural network (NNUE backwards), that roughly halved the number of moves/sec it could consider, but the better evaluation counters that. All those carefully tuned “bitboard” tables about how many points to give pieces on different squares depending on phase of the game are now irrelevant. But if you look at Leela-0 you’ll see that its doing 100k nodes/sec vs StockFish’s 5M nodes/sec. That’s the critical difference: LC0 looks at way fewer moves, but has a much better understanding of the board. But I’d argue it’s still not a human level of understanding, as clearly no human evaluates 100k moves/sec.

    • @kugelblitzingularity304
      @kugelblitzingularity304 Рік тому +3

      Leela 1 node is about 2400-2500 elo tho. it is about GM level of understanding.

    • @scoutbane1651
      @scoutbane1651 Рік тому +6

      Yeah. It's not human level. It's above. That's why it has to consider that many positions. Duh. It could still probably be even more efficient, but also you just *need* to be able to crunch way more positions if you want to be >3600 elo lol. I think it's kind of ridiculous to think it "doesn't understand the game as well as humans" when it clearly understands it *better,* and not in just a tactical sense.

    • @DotaMobaUnionRu
      @DotaMobaUnionRu Рік тому +7

      The key difference is not merely an evaluation function, but also a search tree expansion algorithm. The original stockfish was based on so called minmax strategy, while alpha zero uses Monte Carlo tree search, which is somewhat different.

  • @kugelblitzingularity304
    @kugelblitzingularity304 Рік тому +46

    Benchmark is self explanatory. But the 'moves' part of FEN actually defines the position. For example, you can reach an opening position by inputting '*start pos FEN* moves *all moves*' and UCI would translate that as the opening position itself.

    • @rosiefay7283
      @rosiefay7283 Рік тому

      The point of 9:44 is that the moves to play mean that Rg6 would produce a draw by threefold repetition.

  • @AI_Jones
    @AI_Jones Рік тому +12

    Nelson, you always have very interesting content. I use a modified stockfish 13 I made to help me analyze my own games. I modified it to have increased valuation on mobility/space and opponent king safety. It doesn't play better than the original, but it plays a more sacrificial, aggressive game that I'd argue is more anti-human. I'd rather learn lines which gain initiative and aren't as deep. Moves which only make sense 10 moves later don't really teach me in any meaningful way.
    I made another stockfish 13 which only plays hardcoded gambit openings and accounts for permutations, as well as shifting to a different compatible gambit if possible. This helped me train my openings a bit.
    I'm working on another one which has a ridiculous amount of hardcoded trap positions it will always accept(made in a way somewhat similar to that Defaults list you were looking at). Stockfish won't otherwise play dubious anti-human traps which are actually effective against human opponents around my level.
    I have an insanely tough crowd at my chess club who have all been playing 20+ years, so the only way I'm able to win is through taking them out of their comfortable lines with gambits, keeping initiative, and the occasional traps they still fall for. Plus it makes for some really fun games. I've only been playing for a few years so winning makes feel like a swindler sometimes, but you gotta adapt and overcome.
    Starting with a very simple chess engine like Sunfish is a good idea if you want to understand some of the foundational ideas without going too deep.

    • @foreseer4552
      @foreseer4552 Рік тому +2

      Thanks for sharing, really interesting!

    • @luciangv3252
      @luciangv3252 Рік тому

      send me ur engine.

    • @est9949
      @est9949 Рік тому +1

      This is very interesting. I hope Nelson or someone else makes a follow up video on this, or maybe you could make one :)

  • @compiling
    @compiling Рік тому +5

    So a benchmark test is usually for testing how quickly the program can calculate something. I wouldn't expect those test cases to have much significance, but they could include edge cases that will exercise unusual code paths.

  • @Leyrann
    @Leyrann Рік тому +5

    I want to point out that "syzygy" is a word and should be pronounced as one, rather than the individual letters. It's meaning is an alignment between three celestial objects.

    • @swan4163
      @swan4163 Рік тому +1

      Syzygy (pronounced siz-a-gee) is also a term in poetry and rhetoric that means when the sound at the end of one word is the same sound as the next. For example, “Lizard dreams.”

    • @apopet
      @apopet 7 місяців тому

      Thanks, I was just about to write the same thing. By the way, it has a wider meaning than just the alignment of celestial objects.

  • @xitvono
    @xitvono Рік тому +5

    Those endgames are quite interesting and definitely deserve a video. The two knights one is not too complicated, and all the key themes can be explained in 1 video. I would probably start with the longest checkmate which is over 100 moves I believe. It transposes into the position that you gave.

  • @pauln7869
    @pauln7869 Рік тому +6

    As others have said, I imagine that the positions are there for some sort of testing. They're not positions for which Stockfish has a pre-programmed answer, instead they can be used to see how well the program is working.
    You seemed confused commentating on two of the positions. At 15:50 you seemed surprised that the black king was being forced towards its pawn. If white can't win with a king and two knights against a bare king, then it can't win by having one knight block a pawn and the other knight and king dance around the black king in an unrelated part of the board. You need the black king close to the black pawn to provide some sort of blocking that the white knights can't do alone.
    Also you seemed puzzled that the three bishops at 17:27 was a draw. The point is that the bishops are all on black squares. If black keeps his king on white squares then white can never put him in check. As long as he keeps his king towards the middle of the board (so that the white king can't block off all the white squares) he can keep going forever.

  •  Рік тому +15

    Looks like you could use some help from someone who knows C++ pretty well. 🙂
    You hit maybe the least interesting file when it comes to understanding what stockfish does. All it does is performance testing.
    Also GitHub search sucks since it sometimes misses things. In this case fens is used in a for loop at line 156. The loop does only a bit of processing to construct a new list of commands to be processed by the caller.
    Edit: material.cpp looks interesting, I suggest looking at it first.

  • @michaelbrantley6039
    @michaelbrantley6039 Рік тому +1

    Super interesting. More please!

  • @JohSmith
    @JohSmith Рік тому +5

    Interesting video. Stockfish code puzzles are fascinating.
    and FEN code makes sense now.

    • @victorfinberg8595
      @victorfinberg8595 Рік тому

      I first encountered Forsyth notation in one of my dad's old notebooks from 60 years ago.
      for you young whippersnappers, there were no public computers in those days

  • @user-cx9un9fs7q
    @user-cx9un9fs7q Рік тому

    This was really fascinating, quite interesting getting a better idea of how stockfish works.

  • @nblack2867
    @nblack2867 Рік тому +5

    Because I'm a C++ junkie, I decided to look up Stockfish's code for the fun of it.
    What happens with "fens" is that it's just a temporary variable used inside the function "setup_bench()". This function returns a list of fen positions that gets used in a file called "uci.cpp". What this particular function seems to be doing is getting a list of fen positions that can be sourced from an external file, or use the default positions (hence, "Defaults") from Stockfish's own source code.

    • @nblack2867
      @nblack2867 Рік тому +3

      And now that you got me looking at this, evaluate.cpp is an interesting read even though apparently Stockfish doesn't even use it all the time to evaluate positions. Instead, it uses NNUE Evaluate to evaluate positions most of the time. Stockfish's code claims that NNUE evaluate is more powerful, but when I glanced at it, it seemed less interesting for a human to read.

  • @shambhav9534
    @shambhav9534 Рік тому +1

    Feels so weird to have you go over Stockfish's Github page. Never expected to see this day.

  • @FellowChad69
    @FellowChad69 Рік тому

    Great video as always

  • @daboffey
    @daboffey Рік тому +5

    'push_back' is a method of std::vector (see vector<string> fens, list; on line 113) that appends the string to the end.

    • @robertakerman3570
      @robertakerman3570 Рік тому

      Douglas, beyond My paygrade-but I appreciate the input.

  • @ChristophFloat
    @ChristophFloat Рік тому +6

    These positions have absolutely no significance whatsoever. A benchmark is a piece of code that runs another piece of code, repeatedly and probably with different parameters, to test that code's performance. You just dug through their test-/benchmark cases and kept asking yourself what the significance of these positions is - there is none. You could maybe have learned something by looking at the actual benchmark code instead of the input data. But if you really want to learn how a piece of software works, you're usually better off looking at test code than benchmark code.

    • @user-sl6gn1ss8p
      @user-sl6gn1ss8p Рік тому +3

      to be fair, the benchmark positions might have some sort of significance in the sense of have been chosen for a reason, but yeah, that may not correlate much with "being significant to play chess well"

  • @elmiraguth
    @elmiraguth Рік тому +15

    The file with the "random positions" is called benchmark.cpp. Those positions are obviously used for benchmarking, i.e. testing if the engine can find the solution.

    • @user-mt2jm7vy6i
      @user-mt2jm7vy6i Рік тому +10

      They are not used for testing if engine can find a solution or not.
      They are used to calculate bench. Bench is a number that is basically "number of total nodes searched that is needed to calculate this positions to given bench with given hash". Default parameters are 64 MB hash and 13 depth.
      This has one single use - every attempt to improve stockfish uses this number as a "hash sum" - if it's not stable then it means smth is not initialised so patch is bugged, and if it doesn't change patch is (most likely) not doing anything.
      No one really cares what stockfish really outputs in this positions - it's just used for this and only this.

    • @elmiraguth
      @elmiraguth Рік тому

      @@user-mt2jm7vy6i Yeah OK, thanks. I don't know the details for Stockfish. Benchmarking is a commonly used term in a lot of disciplines and it's relatively close to "testing" in meaning.

    • @mirkosaor
      @mirkosaor Рік тому +1

      @@elmiraguth Not necessarily testing in general but testing performance is what it means

  • @n8style
    @n8style Рік тому +15

    That file looks like it's for checking the engine can correctly generate all legal moves in weird but known positions, ones that commonly trip up devs when creating an engine
    eg. the position at 6:00 is for testing the combination of checks, double pawn moves and en passant
    White can play e4, then normally black should be able to take it en passant with fxe3, but then that would put black into check so it's illegal

    • @fajdek7032
      @fajdek7032 8 місяців тому

      en passant is a forced move so e4 would be a stalemate

  • @shambhav9534
    @shambhav9534 Рік тому +2

    Those benchmark positions are just random positions for running benchmarks to test the program, lol. There's nothing special about them. The reason it stored that "bad" move is just to test the UCI command position fen moves . It's just for testing.

  • @elmiraguth
    @elmiraguth Рік тому +1

    Yay, that's the thumbnail I voted for!

  • @joecatalan
    @joecatalan Рік тому +1

    I found out trying a random fen in one of those that one fore some reason on board editor started on move 18 rather than move 1.

  • @dilshankavinda9911
    @dilshankavinda9911 Рік тому +5

    Can't wait for part 2.discovering stockfish's secrets

    • @robertakerman3570
      @robertakerman3570 Рік тому

      Nelson wasn't cheating; merely a "peek"(or poke) behind the curtains.

    • @dilshankavinda9911
      @dilshankavinda9911 Рік тому +1

      @@robertakerman3570 And I didn't say he's cheating

    • @etopowertwon
      @etopowertwon Рік тому +1

      Yeah, dissecting evaluate() would be actually fun

    • @robertakerman3570
      @robertakerman3570 Рік тому +1

      @@dilshankavinda9911 Correct Dilshan. Nels said that He felt that way. I apologize. The peek/poke were very old computer language terms, so "ha-ha". I made a poor joke. Cheers

  • @TigerChess2
    @TigerChess2 Рік тому

    I found that "Bench is a number that is basically "number of total nodes searched that is needed to calculate this positions to given bench with given hash". Default parameters are 64 MB hash and 13 depth."

  • @newkobra
    @newkobra Рік тому +5

    There is a bug in github search. fens variable used later in the same file to generate a list of commands to run benchmarks. I doubt that these positions are used inside of Stockfish for evaluating next steps, it's just random position that stockfish is using for measuring performance of the Stockfish.

    • @ralfhartmann8601
      @ralfhartmann8601 Рік тому

      Maybe it is not a bug, but a missleading feature. It shows the file count and the first occurences per file.

  • @DantheCat37212
    @DantheCat37212 Рік тому +5

    Hacking stockfish speaks for itself

  • @ncisfan111
    @ncisfan111 8 місяців тому +1

    Where’s part 2?

  • @X3m.Gaming
    @X3m.Gaming Рік тому

    i didnt knew tuna can ride horses like that these positions were for sure a thing

  • @user-cn8ey1os1n
    @user-cn8ey1os1n Місяць тому

    could you make a video explaining stockfish code base ?

  • @elmiraguth
    @elmiraguth Рік тому +5

    Also, isn't the basis of Stockfish neural networks? Those are notoriously difficult (if not impossible) to understand.

    • @schwingedeshaehers
      @schwingedeshaehers Рік тому +5

      not the basis, but some parts of it uses it (i think the NNUE is the part with it)

    • @bonaaq86
      @bonaaq86 Рік тому +2

      @@schwingedeshaehers The NN stands for neural network so yes

    • @QuotePilgrim
      @QuotePilgrim Рік тому +1

      Not really. Stockfish only started using neural networks very recently, and you can still use it without a neural network.

  • @cwevin
    @cwevin Рік тому +1

    16:00 these speed clones bruh

  • @extra...
    @extra... 4 місяці тому

    The fact that just a king and a knight can force the king to the other corner is just crazy.

  • @danielbillings1876
    @danielbillings1876 Рік тому

    @chessvibes these positions better be a puzzle video in the near future so I can pretend to know the solutions.

  • @shambhav9534
    @shambhav9534 Рік тому +3

    You literally spent the entire video discussing random positions meant for testing.

  • @godwincornelius201
    @godwincornelius201 9 місяців тому +1

    wheres part 2

  • @dannygjk
    @dannygjk 8 місяців тому

    That first idea you showed is book.

  • @jingjingzhang8179
    @jingjingzhang8179 3 місяці тому

    22:07 It says Rookh1 is a draw for some reason, which is quite fascinating, as Rh1 gives up a full rook.

  • @CapyHapy
    @CapyHapy Рік тому

    HAHAHAHAHA i love the duck chess board in the background

  • @shikaru1730
    @shikaru1730 Рік тому

    Is the FEN positions things that the direct code of stockfish cant compute and figure out fully by itself but the positions were known and figured out by people already?

    • @QuotePilgrim
      @QuotePilgrim Рік тому

      No.
      They're random positions for testing purposes.

  • @ChraO_o
    @ChraO_o Рік тому

    "I can't notice a thing"
    -non coder

  • @AbouTaim-Lille
    @AbouTaim-Lille Місяць тому

    3 bishops or even 10 bishops on the same tile colour cant make checkate. The besg thing u get is fotcing the king into a orner and then he has a stalemate.

  • @Coding-to4zj
    @Coding-to4zj Рік тому +1

    lol he wants to learn from stockfish but didnt open the evaluation file and opened the benchmark file and wonders what those testing positions are.

  • @spiriniar
    @spiriniar Рік тому

    new series - stockfish puzzles

  • @Ranoake
    @Ranoake Рік тому +1

    Not sure I would call any of this hacking. Pretty much its intended use...

    • @jaybingham3711
      @jaybingham3711 Рік тому +1

      Poked around source code files is just that. Hack would require changing a program's behavior. Driving clicks is a kind of hack though.

  • @BobChess
    @BobChess Рік тому

    Stockfish is super weird. It stalls the random positions. Why

  • @arcioko2142
    @arcioko2142 Рік тому

    19:34 this could make for a black to play and draw puzzle

  • @Neighbour1
    @Neighbour1 Рік тому

    Tbh the game in the intro was pretty obvious to sac your knight

  • @ajb129
    @ajb129 Рік тому +1

    I voted for the other thumbnail

  • @qad7098
    @qad7098 Рік тому

    Nice to meet you buddy we're on the same page
    It's just nice to be a programmer and a chess player it's just 8640 IQ

  • @mukileswara12
    @mukileswara12 Рік тому

    Try searching for the comments. I mean they put 5 man position mate nd4 thing as comments. If searching comments are possible then why not

  • @jeffreybensinger4668
    @jeffreybensinger4668 Рік тому +1

    I didn't think you'd actually go through the code

    • @amplitudemc
      @amplitudemc Рік тому +2

      He basically didn’t though. He just looked at some random benchmark positions. Honestly he picked the most boring file to look at. evaluate.cpp would be a lot more interesting and would actually have code that shows how stockfish ‘thinks’.

  • @arnoldvanhofwegen2255
    @arnoldvanhofwegen2255 Рік тому

    Haha! "Why is it storing the starting position?" Where else to start from in a standard game??

  • @zada4a
    @zada4a Місяць тому

    20:00 shouldve shown how to draw that..

  • @nunocabral6519
    @nunocabral6519 Рік тому

    Very interesting

  • @bobczech7774
    @bobczech7774 9 місяців тому

    Who else noticed he has a game of duck chess on the screen

  • @dilshankavinda9911
    @dilshankavinda9911 Рік тому

    I tried the same thing before.it's just weird😵‍💫

  • @Flareon406
    @Flareon406 Рік тому

    21:36

  • @mauer1
    @mauer1 Рік тому

    he opens up a random text file that isnt even code for stockfish

  • @moonyl5341
    @moonyl5341 Рік тому +2

    part 2?

  • @indiamalwa
    @indiamalwa Рік тому

    Stockfish is super human

  • @albertmagician8613
    @albertmagician8613 Рік тому

    Github is bad for browsing. Apart from the file name you have no clue what it is all about. The text besides a file name is the description of the change the file is lastly changed. This change could be unsubstantial. Browsing is only useful for people thoroughly familiar with the code and knowby heart the purpose of each file.

  • @onniruusunen9444
    @onniruusunen9444 Рік тому +1

    how can you be a Nm and not know the draw with a pawn and a rook

    • @danielyuan9862
      @danielyuan9862 Рік тому +1

      Because you don't have to know the draw with a pawn and a rook to be an NM.

  • @calculuslover2078
    @calculuslover2078 Рік тому

    These are just test cases...

  • @cyclingadventure273
    @cyclingadventure273 Рік тому

    Pro position

  • @zobayerzahid3368
    @zobayerzahid3368 Рік тому

    hi Qh4 is mate in 1 on 26:00

  • @shotx333
    @shotx333 Рік тому

    Hidden gem :d

  • @swarupgiri5241
    @swarupgiri5241 Рік тому

    Those fens are the puzzles that stockfish gives on lichess!!!

  • @scienceenthusiast9858
    @scienceenthusiast9858 8 місяців тому

    👍

  • @The.emerald39
    @The.emerald39 Рік тому

    Real stock fish beat him

  • @ChraO_o
    @ChraO_o Рік тому

    Wait, is stockfish stealing our data on chess? 😡💀

  • @stefandempf818
    @stefandempf818 Рік тому

    cap alpha zero is better

  • @reactionlessssbu-clips2723
    @reactionlessssbu-clips2723 Рік тому

    well there is a bot that beat stockfish15 ask GothamChess or Levy

  • @Nullicity
    @Nullicity Рік тому

    why dont we just learn the same stuff stockfish knows

  • @user-nb9ye4cv5g
    @user-nb9ye4cv5g Рік тому

    oh my

    Did you get in trouble???

  • @gkristof6504
    @gkristof6504 Рік тому +6

    If there are 1000 Nelson fans, I am one. If there are 100, I am one. If there are only 10, I am one. If there are no Nelson fans, I'm dead.

  • @victorfinberg8595
    @victorfinberg8595 Рік тому +1

    now we see it
    stockfish is NOT a chess engine
    actually, it is an alien intelligence, sent to conquer this planet

  • @my_pleasure_in_your_leisure
    @my_pleasure_in_your_leisure 3 місяці тому

    Misleading title and extremely disappointing material.
    Any half-decent follow-up on this?

  • @chessloverYT
    @chessloverYT Рік тому

    please pin me ❤️

  • @sevret313
    @sevret313 Рік тому

    Looking you be amazed at everything while having no idea about programming is fun to watch, but your title is widely misleading.

  • @robertaduncan8252
    @robertaduncan8252 Рік тому

    My name is Roberta and I'm with your sister right now driving to ashville!

  • @Pedoraruba304
    @Pedoraruba304 Рік тому

    Umm...but I thought Alpha Zero was the best engine in chess...

    • @danielyuan9862
      @danielyuan9862 Рік тому +1

      Stockfish is the best _open-source_ engine.

    • @ckq
      @ckq 11 місяців тому +1

      Alpha zero isn't a chess engine it's a research project and they cheated in the stockfish matches by giving alpha zero a stronger computer. It was just a proof of concept.
      Leela Chess 0 tries to implement Alpha Zero's technique but is currently slightly worse than stockfish

  • @Ahmed-cn6hj
    @Ahmed-cn6hj Рік тому +1

    Nelson do you love me

  • @priyadarshanhs1000
    @priyadarshanhs1000 Рік тому

    alpha zero is stronger than stockfish

    • @mysterioushackers5612
      @mysterioushackers5612 Рік тому +2

      Nope , stock fish 15 is much stronger than alpha zero

    • @Witek902
      @Witek902 Рік тому +1

      Stockfish, as opposed to alpha zero, exists and you can use it

  • @nebula2718
    @nebula2718 Рік тому

    Alpha zero is beter

  • @limiyan3627
    @limiyan3627 Рік тому

    Yikes, you are like that starwars guy