Comprehensive Python Beautiful Soup Web Scraping Tutorial! (find/find_all, css select, scrape table)

Поділитися
Вставка
  • Опубліковано 14 гру 2024

КОМЕНТАРІ • 454

  • @KeithGalli
    @KeithGalli  5 місяців тому +4

    I made a new tutorial building off of the knowledge learned in this video! Check it out!
    ua-cam.com/video/DcI_AZqfZVc/v-deo.html

  • @BennyHarassi
    @BennyHarassi 4 роки тому +435

    Shouts to Keith for giving us all an MIT education without the MIT debt

    • @KeithGalli
      @KeithGalli  4 роки тому +131

      Haha I took one for the team xD

    • @Viralvlogvideos
      @Viralvlogvideos 4 роки тому

      haha

    • @hkemal2743
      @hkemal2743 4 роки тому

      Haha. That was a good one.

    • @krishnahare3638
      @krishnahare3638 3 роки тому

      @@KeithGalli how to start never been good in math 50 years old sitting at home? thnx;-)

    • @tanmaytiwari2450
      @tanmaytiwari2450 3 роки тому

      @@KeithGalli since breaking bad minivans are you know swag 😉

  • @apsilal
    @apsilal 3 роки тому +39

    I paid a bootcamp for learning. But Keith you are way above all that. I understood the concepts from your video only. I owe you man!! Keep going and please don't stop putting up such videos.

    • @KeithGalli
      @KeithGalli  3 роки тому +5

      I appreciate the support! Happy that the videos are helpful

  • @rogueknight2414
    @rogueknight2414 5 місяців тому +1

    I wanted to attempt your recent Advanced web scraping tutorial where I then stumbled upon this amazing tutorial and I'm so glad I did! Thank you very much

  • @adrianapetrova196
    @adrianapetrova196 4 роки тому +8

    The last time I tried to understand BeatifulSoup I gave up. You explain it so easy to understand. Thanks for the hard work and the time you spend on teaching us :)

    • @KeithGalli
      @KeithGalli  4 роки тому +2

      Love to hear it! You are very welcome :)

    • @rahuldavid4831
      @rahuldavid4831 4 роки тому +1

      Me too! It's almost like Keith is a godsend

    • @andyn6053
      @andyn6053 4 роки тому

      @@rahuldavid4831 he sure is :)

  • @ajaykushwaha-je6mw
    @ajaykushwaha-je6mw 3 роки тому +1

    The Best thing about your tutorial are that you start from scratch and teach basic and explain each fragment of code with concept. Love from India.

  • @symnshah
    @symnshah 4 роки тому +5

    I have watched a couple of other videos on BeautifulSoup but believe me this one from Keith is the best one. Keith will take you from scratch to a decent level. Thank you so much.

  • @doomimic315
    @doomimic315 3 роки тому +16

    This tutorial was incredibly helpful! Web scraping is something I've always found interesting but just hadn't been bothered to start learning, yet this video made it easy to understand and covered a huge range of ways to deal with potential problems. Seriously can't thank you enough for this video and will certainly be sticking around for any new tutorials you upload.

  • @santoshvaidya3752
    @santoshvaidya3752 3 роки тому +3

    This is one of the finest videos i have ever seen on training. You are an amazing trainer and most importantly you are explaining things in very simple english, also with examples or exercises that would give an hands on experience for viewers......thanks.

  • @kinwong6383
    @kinwong6383 4 роки тому +4

    CV Update: Web Scraping expert.
    Joke aside what an awesome tutorial. Felt so satisfying to get the secret message with what you taught!!
    Brilliant work!

    • @KeithGalli
      @KeithGalli  4 роки тому +3

      Haha love to hear it! I had a lot of fun putting this one together, so I'm happy to hear that you enjoyed it :)

    • @andyn6053
      @andyn6053 4 роки тому

      @@KeithGalli your tutorials are really appreciated! thanks man :)

  • @bobharford5643
    @bobharford5643 2 місяці тому

    The exercises at the end were great -- a big difference between typing along with the screen and actually having to think through on your own how to solve a problem. I will be off to the new video on web scraping in a few days. Many thanks.

  • @ClaireCodesStuff
    @ClaireCodesStuff 4 роки тому +6

    This is a fantastic tutorial. When I last tried to learn beautiful soup, we were in the awkward transition phase between python 2 and 3 and every tuturial was in python 2 because they hadn't released code for 3 yet. I learned 3 because it was "the future". Of course, I then wanted to use BS so I had try and figure out what I wanted to do in python 2. I gave up in total frustration. This is a crystal clear guide and now I actually understand how it works and how to use it. Thanks Keith!

    • @KeithGalli
      @KeithGalli  4 роки тому +1

      Happy that this tutorial could clarify the details and remove some of that frustration! :)

  • @ranveersharma1666
    @ranveersharma1666 4 роки тому +9

    i am from india . we really dont get this quality stuff here.. so thanks to youtue and you.. for spreading wonderful knowledge.. keep rocking !

  • @lefu7812
    @lefu7812 4 роки тому +63

    Your tutorials are the best, honestly. Thank you so much for doing this.

    • @KeithGalli
      @KeithGalli  4 роки тому +7

      Glad you enjoy them!! You're very welcome :)

  • @LoganNinefingers
    @LoganNinefingers 4 роки тому +3

    Keith you'll be the first one I cite when I write my nobel prize winning book or whatever it is nobel prize winners write. Golden content. Gracias!

  • @rahuldavid4831
    @rahuldavid4831 4 роки тому +8

    Thank you so much for this wonderful tutorial Keith! Words cannot describe how much I am grateful to you for making this gem of a video that covers everything you need to successfully scrape a webpage! Trust me when I tell you that NOBODY HAS MADE A BETTER VIDEO ON BEAUTIFULSOUP than you!!! If I could have the liberty of suggesting future videos, I would love if you made a video about "Regular Expressions". Keep up the good work and God bless!!!

    • @KeithGalli
      @KeithGalli  4 роки тому

      Very happy to hear you enjoyed!! A regex video is a great idea :)

  • @dhruvrathore2022
    @dhruvrathore2022 4 роки тому +33

    Please do a Seaborn Tutorial ! like you did with Pandas, Matplotlib etc. I watched all of them, really glad i found your channel. Simple, informative & on point.

    • @andyn6053
      @andyn6053 3 роки тому

      @Lucas agree, Derek Banas has a great Seaborn tutorial at his channel!

    • @fardinahsan2069
      @fardinahsan2069 3 роки тому

      If you know matplotlib you know most of seaborn, its a matplotlib wrapper. all matplotlib methods work in seaborn too

  • @manu93ize
    @manu93ize 4 роки тому +1

    by far the best tutorial on youtube for web scraping. you are very good at dumming it down, even total beginner can even understand.
    waiting for NLTK tutorial.
    thank you

  • @wahaha108
    @wahaha108 2 роки тому

    The best python video i have ever seen. No wasted words, dive into the important topic. Lol, great!

  • @modernmistyk4341
    @modernmistyk4341 3 роки тому

    You saved my life, I hope you're getting all the beautiful things in life you deserve

  • @dusty6193
    @dusty6193 8 місяців тому

    Only a third of the way through this video and I already feel like I understand this better. Thank you, brand new at this

  • @TheFearlessGoat
    @TheFearlessGoat 4 роки тому +6

    I love that you have exercises for us to do in the videos! Learned so much from this.

  • @mohitupadhayay1439
    @mohitupadhayay1439 2 роки тому

    As someone earlier said, Big SHOUT OUT to Keith for getting the community such amazing content!

  • @fabianrestrepo82
    @fabianrestrepo82 3 роки тому +2

    Man watching that ending was almost like watching Jack sink, beautiful ending!! keep it up man, great content

  • @soesevenonesix
    @soesevenonesix Рік тому

    Keith, your videos are excellent. You are totally getting me through grad school just watching your tutorials. Keep it up!

  • @investandcyclecheap4890
    @investandcyclecheap4890 4 роки тому +10

    This is such a great tutorial ! I loved being able to pause and figure out the problems on my own. I really learned a lot! Thanks Keith, you rock!

  • @ranveersharma1666
    @ranveersharma1666 4 роки тому +2

    we just love your content.. u taught me pandas very well....!!!

  • @jamesdavies5386
    @jamesdavies5386 2 роки тому

    Hey this tutorial is great! I've been looking for a decent one like it for some time now and I can't believe it took the algorithm this long to show this on my recommended page

  • @kaustubhgupta12
    @kaustubhgupta12 4 роки тому +18

    When keith do it, its perfect 🤩

    • @KeithGalli
      @KeithGalli  4 роки тому +1

      Aww I appreciate that 😊

  • @ikki411
    @ikki411 3 роки тому +1

    This tutorial was incredible. I've done 2 Python courses that touched the 'Web Scraping' subject, but I wasn't able to fully understand it. This video was one of the two videos that made me fully understand it, and I couldn't be more happy about it. And finding out the secret message was amazing too :D

    • @h4zmeister
      @h4zmeister Рік тому

      wanna share the other video you found helpful ? :)

  • @chiranthchangappa6231
    @chiranthchangappa6231 4 роки тому +1

    One of the best web scraping contents I've seen to date. But the ending was hilarious!

  • @nasser_omar
    @nasser_omar 2 роки тому +1

    Hi Keith,
    I'm really excited to watch this video. Actually, I used to watch your all Python-related videos, especially the Pandas one.
    Keep going, and I hope to meet you one day.
    THANKS, A LOT

  • @esspi9
    @esspi9 4 роки тому +12

    Amazing.
    Thanks Keith!
    Looking forward to the Selenium and scrapy series.

    • @KeithGalli
      @KeithGalli  4 роки тому +3

      You're welcome!

    • @esspi9
      @esspi9 3 роки тому

      @@theduck3126 Try John watson Rooney channel.
      He's got everything covered.

  • @panakitikos
    @panakitikos 4 роки тому

    man when i accomplished that secret word challenge on my own i swear to god i just clapped for u and wished i hadn't already liked the video and being a subscriber so i can like and subscribe. thank you very much keith. you're like my geeky genius best friend that im learning from

    • @KeithGalli
      @KeithGalli  4 роки тому +1

      Haha this comment put a big smile on my face :). I appreciate the support!

  • @andrewp319
    @andrewp319 4 роки тому +1

    This is by far the best tutorial I have found after searching through the internet for hours. I subscribed just because of this one great video. Please keep doing videos of practical applications of Python. Project tutorials are the best.

  • @hemanthkumaar3681
    @hemanthkumaar3681 4 роки тому +1

    i learned numpy ,pandas and other things from ur play list. i was strucked for the past 3 days in webscraping i watched a lot of yt videos bt i coudnt understand as ur content...Thank you so much brother :D . Now i hit(smashed) the bell icon too...

    • @KeithGalli
      @KeithGalli  4 роки тому +1

      Awesome glad this video could help clarify some of the confusion you had. Thanks for smashing the bell icon! xD

  • @andyn6053
    @andyn6053 4 роки тому

    At 27:45 "div.text " works just as good as "div.get_text()" to print out the strings of the element. You can even use "header.text" instead of "header.string". Seems like ".text" is universal for getting all kind of strings.

  • @sreeragmsudheesh
    @sreeragmsudheesh 2 роки тому

    52:30 Not sure if this was posted before but this works for the duplicates. Thanks for all the help Keith!!!
    import pandas as pd
    table = soup.select("table.hockey-stats")
    df = pd.read_html(str(table))[0]
    df

  • @MarceloSantos-nc9wq
    @MarceloSantos-nc9wq 4 роки тому +3

    Keith, many thanks for giving us too many excellent information about hard topics. You do the things seem totally simple to do. Sincerely, your tutorials are the best. Again, thank you so much for sharing all of this with us.

  • @nallym82
    @nallym82 4 роки тому +1

    I am very glad that I found your videos. I learnt more from you than all other tutorials combined. Please do a tutorial on xlwings. Thank you

  • @rogerwprice
    @rogerwprice 4 роки тому +1

    Another fabulous real-wold tutorial. Thanks for the google and stack overflow searches and the errors with recovery!

  • @Some_random_guy_16
    @Some_random_guy_16 4 роки тому

    Oh man.. your tasks are excellent. It helped me to get a better confidence in working with soup..

  • @muhammadkazimraza3456
    @muhammadkazimraza3456 2 роки тому +1

    Very very good video and great exercises specially last one.
    Thanks for such videos

  • @XuX-l8q
    @XuX-l8q Рік тому

    one of the best beautiful soup videos, and really want to say thanks! Keith

  • @sagebaram5951
    @sagebaram5951 3 роки тому

    How do you know you’ve learned something ?
    Completing the challenge within 1 minute no hints. Thank you so much for all your efforts :)!

  • @kallenmulilonalyanya4181
    @kallenmulilonalyanya4181 2 роки тому

    I like how you make simple stuffs that were really scary. Bravo man.

  • @shin-mg7hn
    @shin-mg7hn 2 роки тому

    Your video really help a lot to understanding the Beautiful Soup, thank you, Keith!

  • @bhupindersingh4347
    @bhupindersingh4347 2 роки тому +1

    This is a very will organized web scrapping tutorial. Thanks for sharing.

  • @adrianobavaresco76
    @adrianobavaresco76 Рік тому

    Thank you Keith! This is the best video that i watch about bs4. 👏👏

  • @PrielCohen1
    @PrielCohen1 Рік тому

    Thank you for the video!
    You explain things so clearly

  • @tralfazy
    @tralfazy Рік тому

    Great video and well done. I learned a lot from it. Thanks Keith!

  • @alic
    @alic 2 роки тому

    Brilliant, amazing channel. Major kudos to you Keith!!

  • @iklintsov
    @iklintsov 4 роки тому

    best most concise and detailed tutorial on bs

  • @futuregootecks
    @futuregootecks 2 роки тому

    Wow path navigation is so powerful! Thanks for this!

  • @lalitsharma-gl4kr
    @lalitsharma-gl4kr 4 роки тому

    Value for time invested in watching your videos. Along with the subject knowledge, we understand how to practically approach a problem. Thanks a ton for sharing your knowledge.

  • @victordias8899
    @victordias8899 4 роки тому

    Bro you're great at these videos. Keep it up. I'm very glad I found your channel and I'm learning a lot from you.
    Regarding the task of getting the "is" from fun-facts, you can get them by this simple one liner:
    [li.get_text() for li in webpage.select('ul.fun-facts li') if 'is' in li.get_text()]
    no regex, no extra loops... just plain string methods with list comprehension!

  • @zainabkhan5859
    @zainabkhan5859 3 роки тому

    This is exactly what I was looking for. Thumbs up Keith for this awesom video :-)

  • @irfanshaikh262
    @irfanshaikh262 3 роки тому +1

    Subscribing, coz I loved it.
    Glad I found you @keith.
    Exploring your channel now.
    Appreciate the way you did it so perfectly making it simpler to understand for me.

  • @amranazad4540
    @amranazad4540 3 роки тому

    This guy deserves the world

  • @zakriajanjua9170
    @zakriajanjua9170 3 роки тому

    I literally hit like because of that secret message lol. Cheers man really grateful for all your content

  • @fahad203
    @fahad203 4 роки тому

    Are you god? I have a simple approach to your videos. I like them first, then I watch the video. Thanks a lot man, you and few others youtubers are going to put universities out of business

  • @gavreleric3493
    @gavreleric3493 3 роки тому

    Wow, really impressive. One of the best channel ! Keith you are very clear with your explanations.
    Thank you for sharing your knowledge :)

  • @OgoidRei
    @OgoidRei 4 роки тому +1

    Thank you Keith, amazing content, easy to follow, clear explanition, great exercices (with walkthrough) and love the funny breaks/comments during the video. Followed and like

  • @AndyRhye
    @AndyRhye 4 роки тому

    The idea with the secret message was super cool!) You've got that like! Well deserved.

    • @KeithGalli
      @KeithGalli  4 роки тому

      Glad you enjoyed it! I had fun setting that up :)

  • @pablomora7880
    @pablomora7880 4 роки тому

    Well done! First class of Web Scrapping! Awesome

  • @Dee-bk3gk
    @Dee-bk3gk 4 роки тому

    You have a lifetime sub from me. Been looking for videos like this for a long time. Keep up with the great content!

  • @khinekhinezaw27
    @khinekhinezaw27 4 роки тому +4

    This is the best web scraping tutorial. Thank you so much!

  • @sunnywen9483
    @sunnywen9483 4 роки тому +1

    so surprised to find treasure youtuber here, will go through all your perfect tut in my summer holiday, hope that u will gain more and more subscribers~

  • @rodrigomonteiro8780
    @rodrigomonteiro8780 4 роки тому

    Man you save my life. your tutorials are amazing.

  • @Beyond..Horizon
    @Beyond..Horizon 2 роки тому

    That's the data scientist's way to tell "Like and Subscribe ". Thanks for sharing knowledge!!

  • @schoolstudentarea4199
    @schoolstudentarea4199 4 роки тому

    i wish i had a cool teacher like you

  • @Amulya7
    @Amulya7 Рік тому

    Beautiful video Kieth. Loved it.

  • @sarahburkhardt2037
    @sarahburkhardt2037 3 роки тому

    Thanks for sharing this! I am mostly just popping in to learn, but this is helping me know how to think about data & see that there are a lot of options.

  • @pratiksarani4947
    @pratiksarani4947 Рік тому

    wow a fun exercise !! Have a great fun , Next one is the Pandas One

  • @Inksaver
    @Inksaver 4 роки тому

    Thanks for this excellent video. I usually teach C# and am familiar with the HTMLAgilityPack, but I was asked by a GCSE pupil to teach him webscraping using Python. We have been following this together and found it very easy to use. Subscribed.

    • @KeithGalli
      @KeithGalli  4 роки тому

      Awesome! Glad to hear that and thanks for subscribing :)

  • @lokotock
    @lokotock 4 роки тому

    Thanks a lot! Your video are clear and pretty useful! And it’s a joy watching them! I’m glad that I found your channel ✨

  • @muthonigathage263
    @muthonigathage263 2 роки тому

    This was a fun video! Thank you Keith Galli.

  • @vatsdimri3675
    @vatsdimri3675 2 роки тому +1

    Really learned a lot. Loved the exercises.

  • @armandaske
    @armandaske 4 роки тому

    Yes! Awesome tutorial dude. Looking forward to your next web scraping video. Cheers!

  • @abdoooooo8583
    @abdoooooo8583 4 роки тому

    Great video .. and I watched A LOT videos about beautiful soup. Keep going with the series

  • @shrutipancholi3544
    @shrutipancholi3544 3 роки тому

    One spot for all my Python needs. Thanks Keith! ; )

  • @benlucke7763
    @benlucke7763 3 роки тому

    Thanks for the tutorial Keith! Keep up the great work

  • @danniliu2544
    @danniliu2544 3 роки тому +1

    Hi Keith, i second many of the viewers comments. Your tutorial on web scraping is by far one of the best ones out there. Thank you so much for producing this. I do have a question though. Hope you can help clarify, I've not had much success googling. Can you clarify what the difference between select function vs. find_all function? when would you use one over another?

  • @unsignedperson476
    @unsignedperson476 4 роки тому

    You are perfect ! You know how to teach. Thank you so much man. Liked your style, and got the subject i have been struggling. Liked and subbed.

  • @chineduezeofor2481
    @chineduezeofor2481 3 роки тому

    Wow! This is just too good. Thanks for the video Keith

  • @bernardobritto8352
    @bernardobritto8352 3 роки тому

    LOL, loved the secret message. Great work, thanks for the video

  • @nasser_omar
    @nasser_omar 2 роки тому +1

    LOL. Nice secret message.
    I appreciate your effort :)

  • @soumyaranjandash3597
    @soumyaranjandash3597 2 роки тому +1

    Amazing Lecture. Here we understood Everything. Thanks a lot Broo 🔥👍🙂

  • @andvad6475
    @andvad6475 4 роки тому

    Thanks Keith. A really great video. Keep them coming, really useful videos I am learning a great deal from you. Many thanks.

  • @TheAwesomejay
    @TheAwesomejay 3 роки тому +1

    I did the secret mystery challenge by myself! i feel incredible, thank you for this tutorial, keith!

    • @KeithGalli
      @KeithGalli  3 роки тому

      Awesome, love to hear that :). Nice work and glad you enjoyed the tutorial!

  • @carlosroquesuarezgurruchag8681
    @carlosroquesuarezgurruchag8681 3 роки тому

    of course i will smash that button!! Sos un crack amigo, gracias por la buena onda y dedicacion!

  • @shakilahmed31
    @shakilahmed31 3 роки тому

    7 star out of seven .Thanks a lot keith , i found your tutorial after searching a lot about web scrapping , can not compare your lactures with any . plaese make one video with road map of web scrapping , next expected tutorial from you on Machine learning , thanka again .

  • @yinshengkou9637
    @yinshengkou9637 4 роки тому

    I thought the message was 'Make America Great Again!!' when I saw the first word 'Make' on my screen, lol. I practiced along with the video and this kind of interaction is really interesting.

  • @pchebbi
    @pchebbi 4 роки тому

    *SMASH*ing that like button with all 10 fingers! Wish I had more fingers! Thank you so much! Amazing tutorial. You are so meticulous and your teaching is very methodical. Thank you!

  • @ajayrana4296
    @ajayrana4296 3 роки тому

    we need more analyzing videos from you

  • @ukaszkopyszko9818
    @ukaszkopyszko9818 4 роки тому

    Thanks Keith. I'm moving on to next web scraping tutorial rn

  • @yogeshuttekar8542
    @yogeshuttekar8542 4 роки тому

    Bro loving your videos and the cool stuff you teach us. It really helpful for me. hope you and your channel grow well. plus you keep posting these crystal clear skills well. hoping too see new video soon.

    • @KeithGalli
      @KeithGalli  4 роки тому +1

      Very happy to hear that you've been liking the videos!

  • @r.w.5642
    @r.w.5642 3 роки тому +1

    Great tutorial. I've watched quite a few of your videos recently; they're fantastic.
    I will say, while I watched the secret words be revealed, I felt a little like Ralphie in "A Christmas Story," during the Ovaltine secret-decoder reveal. lol

    • @KeithGalli
      @KeithGalli  3 роки тому +1

      Lmao great comparison 😂

    • @KeithGalli
      @KeithGalli  3 роки тому +1

      Glad you enjoyed the tutorial!

  • @atoamuah8868
    @atoamuah8868 4 роки тому

    Thumbs up bro. You make this thing look so basic and fun. Thanks a lot.

  • @luchoargentina1
    @luchoargentina1 9 місяців тому

    Son increibles tus videos!! Gracias Keith