Massively Speed Up Requests with HTTPX in Python

Поділитися
Вставка
  • Опубліковано 7 вер 2024
  • In this video, we learn how to massively speed up web scraping with asynchronous requests in Python, using the httpx module.
    ◾◾◾◾◾◾◾◾◾◾◾◾◾◾◾◾◾
    📚 Programming Books & Merch 📚
    🐍 The Python Bible Book: www.neuralnine...
    💻 The Algorithm Bible Book: www.neuralnine...
    👕 Programming Merch: www.neuralnine...
    🌐 Social Media & Contact 🌐
    📱 Website: www.neuralnine...
    📷 Instagram: / neuralnine
    🐦 Twitter: / neuralnine
    🤵 LinkedIn: / neuralnine
    📁 GitHub: github.com/Neu...
    🎙 Discord: / discord
    🎵 Outro Music From: www.bensound.com/

КОМЕНТАРІ • 32

  • @frostiiify9505
    @frostiiify9505 Рік тому +13

    Nice! I have never worked with HTTPX, so this is really interesting to me. Would love to see more!

  • @justas3000
    @justas3000 Рік тому +4

    Big ups for all the stuff you do man! All videos are very clear, concise and on point. Keep it up

    • @oguzhantopaloglu9442
      @oguzhantopaloglu9442 Рік тому +1

      I agree, he makes videos about libraries that nobody fucking knows a thing or two about and those libraries are actually awesome

  • @suomynona7261
    @suomynona7261 Рік тому +2

    I’m happy you still posting! Thanks for what you do.

  • @tamgaming9861
    @tamgaming9861 Рік тому

    Ich habe mit einem 2 Jahre alten Video von dir gestartet - und hatte gehofft dass du immer noch postest ... klasse das du es tust!

  • @sumitsingh-CANDY
    @sumitsingh-CANDY Рік тому +7

    ❤ just learned new thing !!

  • @dot_dot_pwn2650
    @dot_dot_pwn2650 Рік тому +3

    You're awesome dude, loving your new pace. Keep up the great work.

  • @ahmadowaihan9349
    @ahmadowaihan9349 Рік тому +3

    this video was very helpful! exactly what I needed :) keep it up! (this vid was suggested to me at random lol)

  • @noertri618
    @noertri618 Рік тому +3

    Python 3.11 + httpx = 🔥🔥🔥🔥

  • @tony_jin
    @tony_jin Рік тому +2

    Thanks a lot for the video! Does "httpx" work with webdriver from selenium?

  • @swarnodipnag
    @swarnodipnag Рік тому +1

    Love from India. 4th comment.. daily viewer .. python lover ❤️🫂

  • @praveenkumawat2662
    @praveenkumawat2662 Рік тому

    wow, it works amazingly. Thanks for sharing this

  • @aryankothari4634
    @aryankothari4634 Рік тому +1

    Have you covered python queue.Queue() with threading?

  • @benlong1062
    @benlong1062 Рік тому

    This was a good one, really interesting.

  • @oguzhantopaloglu9442
    @oguzhantopaloglu9442 Рік тому

    This is INSANE!

  • @Jelvix
    @Jelvix Рік тому

    Awesome video! We talked about a similar topic on our channel a few videos ago!

  • @makankananian3838
    @makankananian3838 Рік тому +1

    are the async requests that we gather in the same order as the URLS list??

  • @InvestLarge
    @InvestLarge Рік тому

    Comment for the algorithm :) Perfect video lesson

  • @davehawkins5265
    @davehawkins5265 Рік тому

    Great video!

  • @AhmdsYT
    @AhmdsYT 5 місяців тому

    4:29 using httpx

  • @ken6349
    @ken6349 4 дні тому

    does this cause 429 error? (too many requests error). if done in bigger scale

  • @clxymox
    @clxymox Рік тому

    very interesting, it is possible to also have the url that we are testing with the return 200

  • @2ru2pacFan
    @2ru2pacFan Рік тому

    Thanks!

  • @throwyourmindat
    @throwyourmindat Рік тому +1

    Hi ,
    I tried executing HTTPX POST method, but it is returning 419 instead of 200. the calls are similar to what I used to do with request.Session.. can you please create a view on POST call. what is the trick to make it work ?

    • @RavanaMad
      @RavanaMad 8 місяців тому

      The difference between accepting GET or POST requests is defined on the server side. GET carries request parameter appended in URL string while POST carries request parameter in message body which makes it more secure way of transferring data.

    • @throwyourmindat
      @throwyourmindat 8 місяців тому

      @@RavanaMad thanks it's been 7 months now. It was https post specifically. I have figured out already

  • @pamdemonia
    @pamdemonia Рік тому

    Very cool.

  • @connectrRomania
    @connectrRomania Рік тому

    Is this efficient on getting access to different ip camera addresses

  • @jaspreetsingh9688
    @jaspreetsingh9688 Рік тому

    Can you solve rubik cube with python if yes then please make video

  • @miguelhyto54
    @miguelhyto54 9 місяців тому

    4:30

  • @agnichatian
    @agnichatian 6 місяців тому +1

    books to scrape is huge malware site now