I Wrote HTTP "From Scratch" (It Was Easy)

Поділитися
Вставка
  • Опубліковано 5 жов 2024

КОМЕНТАРІ • 110

  • @lukeh990
    @lukeh990 2 місяці тому +337

    You know it’s going to be a good video when it starts out with: “We’re going to assume the presence of a TCP/IP implementation on your system”

    • @Ginto_O
      @Ginto_O 2 місяці тому +1

      well i mean it is preinstalled on windows but you know how to install it right? in network properties

    • @0w784g
      @0w784g 2 місяці тому

      Dag nabbit, here's me still using token ring networking.

  • @Q_20
    @Q_20 3 місяці тому +173

    I read that as C written in HTTP, and was hoping for next level black magic shit.

    • @seanbix5366
      @seanbix5366  3 місяці тому +47

      If anybody is determined to bless the world with this, I'll help where I can!

    • @vpd825
      @vpd825 2 місяці тому +1

      Same here 😂 This is still cool.

  • @shinej11
    @shinej11 2 місяці тому +35

    I spend most of my time with Python, with all the layers of abstraction. This video definitely rekindled my love for C. Thank you.

    • @seanbix5366
      @seanbix5366  2 місяці тому +15

      @@shinej11 Python's is imo the least evil scripting language, and can do some low level stuff too.
      I once implemented some serial networking protocols in Python on a microcontroller.
      Use the right tool for the job and have fun!

    • @gregh5061
      @gregh5061 Місяць тому

      @@seanbix5366 You probably won't see this, but what C projects would you recommend a Python dev work on to get a good grasp over C? I have nothing specific I want to do with C in mind, which is why I ask this.

    • @seanbix5366
      @seanbix5366  Місяць тому +5

      ​@@gregh5061Something that motivates or excites you personally.
      If I worked primarily with Python and didn't hate what I do, I would try rewrite some of that work in C.
      As an IT person by trade, I've rewritten high level scripts (including python) that could benefit from a performance boost in C just to learn and I think it's a fun, practical and quick way to pick up the nuances.
      If that's too daunting,
      then it's back to basics with data structures, algorithms (and books) imo.
      Maybe in a library you
      can reuse and improve!

  • @syedsaifuddin1413
    @syedsaifuddin1413 3 місяці тому +20

    Ahh very good explanation good sir.
    I was getting into c socket programming without understanding the underlying fundamentals to implement this protocol.
    Gained a lot of information from this one.

  • @shreeyashpandey3530
    @shreeyashpandey3530 3 місяці тому +38

    That's a very creative way to use gimp as a presentation tool!
    hella based video!

  • @felixkuhlmann9985
    @felixkuhlmann9985 Місяць тому +1

    Very concise and thorough introduction to the application of those theoretical ideas and protocols, most college classes lack this practical approach. Thank you!

  • @AngelsFishingRod
    @AngelsFishingRod 20 днів тому

    Taking Fundamentals of Data Comm rn and this video is so goated to actually understanding the shit im learning

  • @DailyFrankPeter
    @DailyFrankPeter 2 місяці тому +5

    In the worlds of Linus Torvalds - let's write it from scratch; how hard can it be!

  • @davidrieger3160
    @davidrieger3160 Місяць тому

    Dude this is such a good video. I hope you are a uni professor.
    Would love to see an implementation of TCP as well

  • @impaglg
    @impaglg 2 місяці тому

    Thank you for this video Sean! it really was eye opening.

  • @SamirPatnaik
    @SamirPatnaik 3 місяці тому +10

    S+ tier content

  • @darkshoxx
    @darkshoxx 2 місяці тому +7

    F*ck Powerpoint, we're using GIMP 😆! Excellent content

  • @whonehuljain
    @whonehuljain 3 місяці тому +3

    Great content, also amazing explaination!

  • @elgs1980
    @elgs1980 Місяць тому

    What can be more satisfying!

  • @JeremyAndersonBoise
    @JeremyAndersonBoise 2 місяці тому

    This is a great practice project for programmers. Very cool moves.

  • @DanielPopsuevich
    @DanielPopsuevich 3 місяці тому +2

    Execllent video! Just what I was looking for. I recommend thinking about a better title in future in order to attract more viewers to your content since this video is a fantastic deep dive, you explained protocols in less than a minute visually.

  • @kanha12297
    @kanha12297 Місяць тому

    Very informative video ❤ loved every bit of it

  • @SalirDeMatrix
    @SalirDeMatrix 2 місяці тому +2

    You are a good developer. 👍

  • @michelhertwig
    @michelhertwig Місяць тому

    very good video! please go on with your important and solid work. good job! very good!

  • @prashlovessamosa
    @prashlovessamosa 2 місяці тому

    Wow great learnt a lot keep uploading good stuff.

  • @s1ckret
    @s1ckret 3 місяці тому +2

    Great work, man!

  • @WomboBraker
    @WomboBraker 19 днів тому

    ok ok at 2:30 coup de grace, i subscribed gj

    • @seanbix5366
      @seanbix5366  18 днів тому

      Thank you, more stuff on the way

  • @AndrewNijmeh
    @AndrewNijmeh 2 місяці тому +1

    this is how real men program 🙏🏼

  • @androu-f3b
    @androu-f3b Місяць тому

    Nice video and very clear explanation! For some reason I thought http is much more complicated.
    One question I have, is there a specific reason you search for "GET /" in the request and not just "GET"?

    • @seanbix5366
      @seanbix5366  Місяць тому +1

      @@androu-f3b Because here I'm checking if the client is specifically getting "/", the root directory.
      Its the GET request the server receives when they (the client) just type in " IP_ADDRESS/".
      Its rudimentary but eh it works.
      Otherwise it's "GET /SPECIFIC_RESOURCE".
      Thank you!

  • @s14vc
    @s14vc Місяць тому +1

    How did you were able to actually implement the theory of networks? I just saw a course in college about OSI model but it was almost entirely theorical, what did you have to learn?

    • @seanbix5366
      @seanbix5366  Місяць тому +4

      TCP/IP is typically implemented as a part of the OS kernel.
      If you're not using an OS, e.g. in embedded (very very cheap) systems, software libraries will implement the lower level protocols for you (up to level 4 is common).
      Here, I've used the TCP/IP implementation in Linux ("Berkeley Sockets API") to implement HTTP (although I could have used a library for that too).
      TL;DR: I didn't!

  • @artaxerxes360
    @artaxerxes360 2 місяці тому

    For a need like me this is treasure.

  • @utuberrocks-q5j
    @utuberrocks-q5j 2 місяці тому +4

    How different is this from the binary HTTP protocol? And can you leverage your code into that easily?

    • @seanbix5366
      @seanbix5366  2 місяці тому +7

      I literally googled HTTP RFC and immediately started writing the code. I was roughly today years old when I saw there was http 2 and 3 so I don't know yet 👉👈
      I've heard they're the same protocol, but with a binary format and extra features, so it should be doable!

    • @petertriller8343
      @petertriller8343 2 місяці тому

      @@seanbix5366 http2 and 3 are a lot more complex, you can have several transfers simultaneously over the same TCP connection and weird stuff like that and http3 uses UDP / Quick as underlying network protocol. So don't count on being able to reuse too much

  • @ahmadgazzz
    @ahmadgazzz Місяць тому

    This is awesome, but i kept getting thrown off because of the audio

  • @andregeraldo6556
    @andregeraldo6556 2 місяці тому

    just got smarter watching this lol, good video

  • @TodorTsankov-vz3sd
    @TodorTsankov-vz3sd 2 місяці тому +1

    writing HTTP 😇 vs TCP 👹

  • @iSam36O
    @iSam36O 2 місяці тому

    moonfly gang 🔥 great video btw, keep them coming!

  • @Mikee512
    @Mikee512 2 місяці тому +2

    Easy in C*
    *Except for dynamically sized strings. :P

  • @MadMatty72
    @MadMatty72 2 місяці тому

    La-mans talk, live it

  • @kamertonaudiophileplayer847
    @kamertonaudiophileplayer847 Місяць тому

    Cool, thank you.

  • @Gersberms
    @Gersberms 21 день тому

    Thanks for saying TCP/IP doesn't follow the OSI model. So many people get this wrong!

    • @seanbix5366
      @seanbix5366  21 день тому

      @@Gersberms I was taught in school that TCP+IP+HTTP = OSI but saw in books that its distinct!

  • @fabienWendpanga
    @fabienWendpanga 3 місяці тому +1

    Very interesting! Now, i am wondering as a c# developer is it something i cam replicate? Also do you have it on a repository ? Thank you very much

    • @seanbix5366
      @seanbix5366  3 місяці тому

      Yes, and yes.
      This is "socket" programming, the operating system's abstraction of TCP/IP which you can do in C#.
      That's how you would send the HTTP header string and data.
      This is purely for learning purposes though, it's not massively practical or idiomatic.
      C# has easy HTTP interfaces.
      The project is linked in the description!

  • @xcelestialdemon8245
    @xcelestialdemon8245 Місяць тому

    Every time you said "haych" and angel lost its wings

    • @seanbix5366
      @seanbix5366  Місяць тому +1

      You couldn't fathom the width of the brush I sweep them up with.

  • @singhyuvraj122
    @singhyuvraj122 3 місяці тому +3

    Great Video, Please more tutorials on Live coding of Email Server, HTTP and TCP /IP in C from scratch

    • @seanbix5366
      @seanbix5366  3 місяці тому +1

      @@singhyuvraj122 I won't be doing any TCP/IP implementation until I end up working more in embedded and need to make one. An email server sounds rad though!

    • @MarkHall-cf6ji
      @MarkHall-cf6ji 2 місяці тому

      ​@@seanbix5366i think sockets allow you to send raw link-layer packets (ethernet frames) so you can probably implement TCP/IP from userspace

  • @Alfred-Neuman
    @Alfred-Neuman 2 місяці тому +1

    Next step is to write HTML from scratch, and then Javascript...
    Good luck! 😀👍

  • @minma02262
    @minma02262 3 місяці тому +2

    Try http2 and http3. Very nice content 👍👍👍

  • @justcurious1940
    @justcurious1940 3 місяці тому +1

    Is the operating system involved in the TLS protocol or the OpenSSL library will take care of encryption and the decryption of the data and then just use the send and recv functions provided by the OS ?

    • @seanbix5366
      @seanbix5366  3 місяці тому +1

      That's abstracted away by the library such that it's simple.
      The operating system is involved in the protocol, such that the library that implements the protocol, interfaces the operating system for various utilities to assist it in doing so.
      Encryption algorithm: library code
      Reading a certificate file: OS code
      So yeah its more than likely that it (openssl code) encrypts the data before just sending it over send().

    • @justcurious1940
      @justcurious1940 3 місяці тому +1

      ​@@seanbix5366 Thank u for clarifying this. Good luck with what ever u are doing today or u are planing to do tomorrow 🙃.

  • @bennguyen1313
    @bennguyen1313 3 місяці тому +3

    Any thoughts what would be involved in writing an ESP32 application, that takes data from the uart, and passes it wirelessly either bluetooth (SPP) or via Wifi. For Wifi, I imagine the ESP32 could act as a web server (TCP) and display the RS232 data, but maybe there's a UDP way it could send data to a PC on the same network?

    • @seanbix5366
      @seanbix5366  3 місяці тому +2

      (Edit: I got a job writing ESP32 firmware)
      Yeah totally, you can do a webserver.
      Arduino libraries for ESP32 have you mostly covered either way; UDP, TCP or HTTP.

  • @grenadier4702
    @grenadier4702 3 місяці тому

    Maybe I didn't notice but have you tried using non-blocking event-driven i/o? As far as I remember it's the fastest way to handle requests

    • @seanbix5366
      @seanbix5366  3 місяці тому

      @@grenadier4702 All of my sockets are "non blocking", but they block anyways (lmao) because each connection is accepted in its own thread. TCP multicasts make full use of non blocking IO on one thread through.
      All file reads are lazy loaded and cached into memory with mmap so its eh, good enough on that front I guess.
      Not a hash map or event queue in sight, just a simpleton, liberally mutex locking.

    • @grenadier4702
      @grenadier4702 2 місяці тому

      ​@@seanbix5366 Yeah, I see now. Here, right?
      while (er == RECV_TRYAGAIN) {
      er =
      tryAcceptConnection (localhost, remotehost);
      }
      You're basically looping until an error or a connection arrival?
      And here in each thread you just check for blocking and repeat the same loop again?
      else {
      if (errno == EAGAIN || errno == EWOULDBLOCK) {
      continue;
      }
      fprintf(stderr, "
      Socket error, closing connection...
      ");
      closeConnections(args->remotehost);
      }
      But why do you need non-blocking sockets here at all? When you accept a connection, if it would block, you repeat the same loop again so it's the same as a blocking socket. In second code block, it;'s the same thing: even if it's non-blocking, you do the same loop, i.e. waiting for data arrival, blocking other tasks in your thread pool
      Maybe I misunderstood your code?
      UPD: fuck, i realised you wrote "but they block anyways", sorry :) But the question is why did you use non-blocking sockets at all, then?

    • @grenadier4702
      @grenadier4702 2 місяці тому

      @@seanbix5366 oh gosh, freaking youtube deleting comments
      I checked out your code.Why did you use non-blocking sockets in the first place? Because I see even if errno is EWOULDBLOCK, you still iterate over and over waiting for data to arrive or send

  • @KangJangkrik
    @KangJangkrik 2 місяці тому

    Now try SSL implementation and let's see if u can handle it

    • @seanbix5366
      @seanbix5366  2 місяці тому +2

      Hah!
      Nice try, Sense Of Child-like Wonder!
      Unfortunately, I want to finish the project and get it working.
      SSL/TLS is a whole new can if worms for another time...

  • @bladekiller2766
    @bladekiller2766 3 місяці тому

    Isn't this simple HTTP server, that parses the requests and returns some response?
    Most of the backend frameworks already have this functionality implemented.

    • @seanbix5366
      @seanbix5366  3 місяці тому +17

      @@bladekiller2766 Oh for sure.
      Dead simple, its in the title.
      If you want to actually be productive, you can always grab a library/framework for whatever.
      But now I don't have the third party dependencies, and I learned the protocol inside and out!
      From my experience- there's always an underestimated leap from thinking you understand a technology, to actually being able to implement it which can be fun to discover.
      Plus this is targeted more towards IT people, who often overestimate a lot of complexity in the tech they're using.

    • @bladekiller2766
      @bladekiller2766 3 місяці тому

      @@seanbix5366 Couldn't agree more.
      All the tech stacks and networking complexity looks extremely complex, but when you try to implement it from scratch by yourself you will suddenly realize that it's doable and it will give you insights on why some things are the way they are.
      Props for doing it in C :)

    • @zokalyx
      @zokalyx 2 місяці тому

      @seanbix5366 now implement ipv6 😛

  • @ArtieOddity
    @ArtieOddity 2 місяці тому

    Please Do Not Throw Sausage Pizza Away

  • @energy-tunes
    @energy-tunes 2 місяці тому

    how long did this take you? is the original spec long?

    • @seanbix5366
      @seanbix5366  2 місяці тому

      @@energy-tunes original spec is linked.
      I implemented a working subset I needed in like 2-3 coding sessions.
      The hardest part was probably MIME types and understanding + writing CORS.
      What took longer was just making a useful interface to the Linux socket API...

  • @truthybyte
    @truthybyte Місяць тому

    HTTP: ☺
    HTTPS: 💀

  • @shiyunze7157
    @shiyunze7157 Місяць тому

    gold

  • @xorxpert
    @xorxpert 2 місяці тому

    TCP/HTTP is easy, then I went too hell with WebSocket but I pulled through lol, but I hate those frames.

    • @seanbix5366
      @seanbix5366  2 місяці тому

      @@xorxpert Nice!
      I have a websocket deepdive on this channel too- the variable length length encoding and payload masking were wicked, but worth it in the end!

    • @xorxpert
      @xorxpert 2 місяці тому

      @@seanbix5366 It was mostly challenging for me at the time. I usually like to learn how things work for experience but mostly avoiding third party or standard libraries. Whilst working on a multiplayer game, I built my own websocket server & client in the process along with a protocol library, for handling sending packets efficiently properly writing/reading and parsing data (binary), while converting between respectable objects. latency was very important, all unmanaged code. Performance, speed, and efficiency was the focus. Per protocol standard for large payloads, you have to deal with splitting the data into frames, and you know TCP, you got to handle waiting and acknowledging packets - multi (safe) threaded 😅
      Took me about a mouth until i was entirely finished it, though was fun and worth learning!

    • @seanbix5366
      @seanbix5366  2 місяці тому +1

      @@xorxpert So far none of my websocket payloads are larger than a few dozen bytes... But I was under the impression that I could simply cram an arbitrary amount of data after a websocket header and TCP would eat it?
      I don't remember, need to look at the code! I fetch all the BIG data with http from JS anyways.... Good work!
      I couldn't even find a simple websocket library anyways.

  • @michaelrenper796
    @michaelrenper796 2 місяці тому

    I wrote HTTP from scratch in 1997 in Java. I was even easier.

  • @gatogordo4131
    @gatogordo4131 Місяць тому

    I'm a fraud as a programmer, that's a real pro

    • @seanbix5366
      @seanbix5366  Місяць тому +2

      Oh no, I'm very much just a guy.
      You'll ask me what an abstract factory is and I'll say "sorry I don't listen to stoner rock".
      Go spend a few days writing some C or ASM, read some books, convene with the machine spirits, and you'll be way ahead of me for sure if you weren't already!

  • @TomZeta-ik8tb
    @TomZeta-ik8tb Місяць тому

    same bro

  • @bonbonpony
    @bonbonpony 2 місяці тому

    Dude! Ever heard of scope? As for someone who claims he wrote this code himself, you talk surprisingly few about the code, and surprisingly lot about unrelated deeper layers of protocols that you don't even attempt to implement yourself, and which are therefore totally outside of the scope.

    • @seanbix5366
      @seanbix5366  2 місяці тому +2

      Fair.
      I suspected the overlap between between people who don't know HTTP and how the TCP/IP stack works was near 1:1, and targeted that audience.
      The implementation itself is not impressive (it's in the title) but it seems to have served it's purpose in giving lots of people a deeper intuition for the entire system (and powering my webserver)!
      I'm sure we can agree that conventional explanations (like wrapping a parcel) fall short.

  • @magibai
    @magibai 2 місяці тому

    You Sir, deserve a subscribe

  • @Awwe12675
    @Awwe12675 2 місяці тому

    Zoom

  • @kaushikkundu
    @kaushikkundu 2 місяці тому

    Cfbr

  • @atomictraveller
    @atomictraveller 2 місяці тому

    holmes, i'm asking you now, to stop doing anything from scratch. don't you know?

  • @petertriller8343
    @petertriller8343 2 місяці тому +1

    http is so easy, and that is why I hate http2 and http3. they ruin the elegance of http/1.1

  • @ReligionAndMaterialismDebunked
    @ReligionAndMaterialismDebunked 3 місяці тому +2

    First. Hehe. Shalom. Yeah, my nose is congested, too. I barely get sick as a vegan that exercises often, meditates, hydrates enough (unlike 75% of Americans and so forth), sleeps enough, good amounts of nutrients, breath work, fasting, etc.

    • @seanbix5366
      @seanbix5366  3 місяці тому +2

      I'm sorry to hear that

    • @MercerBray
      @MercerBray 3 місяці тому

      I have a soup recipe that'll fix you right up if you wanna hear it!

    • @stefanalecu9532
      @stefanalecu9532 2 місяці тому

      You forgot this isn't Reddit

    • @FRANKONATOR123
      @FRANKONATOR123 2 місяці тому

      @@stefanalecu9532yeah lmao like who asked 😂

  • @swojnowski453
    @swojnowski453 2 місяці тому

    It does not matter what you build, but what gets adopted ... Today you can build almost everything but who cares if nobody touches or builds on top of it.

    • @seanbix5366
      @seanbix5366  2 місяці тому +6

      @@swojnowski453 I just want a cool video game for my friends and I, and to learn these technologies along the way.
      I'm building on top of it, I care, and its pretty chill!

  • @MercerBray
    @MercerBray 3 місяці тому +1

    You cover nothing I wanted to know. Your grasp on the subject matter is below that of a hobbyist. This is harmful to the community, basically misinformation. Could you maybe sound less enthusiastic??

    • @seanbix5366
      @seanbix5366  3 місяці тому +1

      Well, what would you like to know my guy?

    • @MercerBray
      @MercerBray 3 місяці тому +1

      @@seanbix5366
      When I made chicken pot pie, the filling was fine but the puff pastry on top didn't puff up though it did cook through. How can I remedy this? The pastry was the store-bought frozen variety. The temperature was 375 °C and I used and egg wash for the glaze.

    • @seanbix5366
      @seanbix5366  3 місяці тому +4

      @@MercerBray 1. Thawing - Avoid an IP Collision
      Proper Thawing: Make sure your puff pastry isn't experiencing an IP (Instantly Pulled) collision by giving it time to thaw in the refrigerator, ensuring no network (temperature) congestion.
      2. Temperature - Keep Your Layer 4 Connections Strong
      Correct Heat: Your oven needs to be set to 375 °F (190 °C). Think of it as keeping your TCP (Thermally Controlled Puff) connections strong. Too hot or too cold and you'll have trouble establishing that reliable connection for puffing!
      3. Baking Setup - Packet Delivery
      Preheated Dish: Preheat your baking dish to prevent any packet loss, ensuring a more reliable delivery of puffiness. And remember, the filling should be hot, so the top layer gets a good handshake with the crust.
      4. Egg Wash - Smooth Communication
      Application: Apply your egg wash gently, just like a smooth handshake protocol, to establish a golden connection without network (pastry) congestion.
      5. Pastry Handling - Avoid Fragmentation
      Thickness: Roll out the puff pastry to about 1/4 inch, keeping it uniform to avoid data fragmentation in the layers, ensuring all packets (layers) arrive intact and puffed.
      6. Pastry Resting - Time for ARP Resolution
      Cooling: Let your pastry rest for a bit in the fridge after rolling it out, allowing the Address Resolution Protocol (puff pastry layers) to map the correct layer addresses (butter) before baking.
      7. Avoid Overhandling - Prevent DDoS (Dough Denial of Service)
      Minimal Handling: Don’t overhandle the dough or you’ll experience a DDoS attack on your puff pastry’s ability to rise!
      8. Ventilation - Manage Network Traffic
      Steam Release: Make small slits in the pastry to manage network traffic (steam), ensuring no data packets (steam) get lost, causing network (pastry) puff loss.
      9. Cover the Edges - Implement Firewall Rules
      Edge Protection: If the edges are browning too fast, cover them with foil as a firewall rule, letting the rest of the network (pastry) puff up securely without packet loss (burning).
      Troubleshooting Tips - Ping Your Pastry
      Temperature Checks: Make sure the oven temperature is accurate, much like pinging to check network latency.
      Proper Layering: Ensure the pastry's layers aren’t merging like a VLAN (Virtual Layers Absent Network).
      Steam Release: Remember, too much trapped steam can cause a denial of service on your puff pastry’s puffing ability!
      Summary
      Thaw properly: In the refrigerator, avoiding an IP collision.
      Correct temperature: 375-400 °F (190-200 °C) for solid TCP connections.
      Preheated dish: Ensures no packet loss.
      Gentle egg wash: For smooth handshake protocol.
      Minimal handling: Prevents DDoS on dough.
      Steam vents: Manage network traffic effectively.
      Edge protection: Implement firewall rules with foil.
      Let these tips serve as your protocol stack for perfect puff pastry puffing!

    • @MercerBray
      @MercerBray 3 місяці тому

      @@seanbix5366 Thanks but could you give the instructions for a 300w microwave oven? Its all I got.

    • @seanbix5366
      @seanbix5366  3 місяці тому +1

      @@MercerBray Ah, you got me.
      The only thing I'll never open source.

  • @timemanager3239
    @timemanager3239 2 місяці тому

    Goat 🦾🦾

  • @ROBOTRIX_eu
    @ROBOTRIX_eu Місяць тому