Python, NoSQL & FastAPI Tutorial: Web Scraping on a Schedule

Поділитися
Вставка
  • Опубліковано 29 лис 2024

КОМЕНТАРІ •

  • @CodingEntrepreneurs
    @CodingEntrepreneurs  3 роки тому +5

    0:00:00 Welcome
    00:00:58 Demo
    00:12:58 Overview & Requirements
    00:15:22 Project Setup
    00:19:26 Start the Python & Cassandra Integration
    00:25:11 Configure Python cassandra-driver
    00:30:13 Your First Cassandra Model
    00:36:08 Create Data using our Cassandra Model
    00:43:20 Adding a New Column to an Existing Model
    00:46:26 Using UUID1 as Primary Key
    00:55:19 Using Jupyter with Cassandra Models
    01:06:35 Using Pydantic for Data Validation and Cleaning
    01:14:22 FastAPI & Environment Variables
    01:21:26 FastAPI + Cassandra & Pydantic
    01:36:01 Convert Cassandra UUID Field to Pydantic Datetime Strv
    01:44:50 Endpoint to Ingest Data for FastAPI & AstraDB
    01:56:41 Celery, Redis & Basic Task Offload
    02:13:18 Integrate Cassandra Driver with Celery
    02:23:47 Running Periodic Tasks
    02:35:24 Basic Scraping with Selenium
    02:45:35 Selenium & JavaScript Endless Scrolling
    02:52:52 requests-html & Parsing Data
    03:11:34 Implement the Scrape Client Parser
    03:27:20 Putting it all together
    03:36:58 Thank you

    • @sukumarsanu7169
      @sukumarsanu7169 3 роки тому

      @1:01:36 I believe it's parenthesis error after objects. It should be objects(). worked for me.

    • @sukumarsanu7169
      @sukumarsanu7169 3 роки тому

      or not, it working for me either way.

  • @dark4krad
    @dark4krad 3 роки тому +15

    Justin you are one of the most amazing persons on the internet. My respect for you is immense. The way you explain in such a lucid language and with so much command over the topic... Incredible. Wish we had teachers like you in school and college. I'm a big fan. I learnt Django from your course. I'm so grateful for your videos. Thank you!!

  • @konstantink3869
    @konstantink3869 2 роки тому

    I really like how you split such a long tutorial for short parts. Especially, this music transition really draws my attention back. Love this beat, amazing job!

  • @perudevlabs
    @perudevlabs 2 роки тому

    This is pure gold dude, also breaking it up on phases.

  • @ericbroun4657
    @ericbroun4657 3 роки тому +1

    سلام بسیار عالی ممنون از زحمت شما امیدوارم که همواره موفق باشید

  • @codetrap3036
    @codetrap3036 2 роки тому

    Hey Justin I'm from India and I start recently watching your videos.your videos are amazing love from India.🙂

  • @asalvats
    @asalvats 3 роки тому +1

    About the UUID1 datetime : that day 'started' the gregorian (current) calendar. BTW, thanks for this great tutorial !!!

  • @tz2014
    @tz2014 2 роки тому +1

    Its amazing, I learned a lot

  • @hayat_soft_skills
    @hayat_soft_skills 2 роки тому

    Awesome! it's my wish to learn more from you. Thanks!

  • @edinhofilho22
    @edinhofilho22 2 роки тому

    I had some issues with Apple's new architecture, like some dependencies and packages on different architecture.
    But the tutorial is insane, thanks

  • @thecodingchronicles
    @thecodingchronicles 11 місяців тому

    @CodingEntrepreneurs is there a easy way fixing issue on mac M1 with how creds are extracded from zip into temporary folder?

  • @nateriver8261
    @nateriver8261 3 роки тому

    Amazing tutotial, except the music between chapters ))

  • @bumblebee2012able
    @bumblebee2012able 2 роки тому

    Hi, Could you please advise me some online/pdf book to learn FastAPI along with Cassandra? Thanks so much.

  • @skyblaze6687
    @skyblaze6687 3 роки тому +1

    Why can't django use fastapi compare to flask ?

  • @MrMardok
    @MrMardok 2 роки тому

    Justin for real a web scraping project (price comparison site) what you prefer to use : this configuration or a Django Rest Api + React Frontend ?

    • @CodingEntrepreneurs
      @CodingEntrepreneurs  2 роки тому

      Depends on the project honestly and how the data will be used.

    • @MrMardok
      @MrMardok 2 роки тому

      @@CodingEntrepreneurs I would get the data from various site by scraping with scrapy, populate a database (maybe nosql) and create an API to consume the data via React page. My scope is to display a price comparison page with various elements and use the affiliate to monetize them.

  • @official.letsfeellove
    @official.letsfeellove 3 роки тому +1

    Hello sir i really love your Videos. Sir i can't solve django rest framework email verification and reset password problem. Can you please help us about it?

  • @rajat971
    @rajat971 3 роки тому +1

    just started fastapi ... :P

  • @vincentle8395
    @vincentle8395 2 роки тому

    It would be supberb if you could provide an alternative for Celery because Celery seems to be not working on Windows. I had to stop following the video because I couldn't make it work. But thanks a lot, the other things helped me big time

  • @nateriver8261
    @nateriver8261 3 роки тому

    Hey! Thank you so much! Can you show an example how to scrape AJAX pagination with selenium?

  • @kamar202
    @kamar202 2 роки тому

    Please how do i drop a table if i made an error

  • @mhmdjavadsbrn4585
    @mhmdjavadsbrn4585 2 роки тому

    very very goood.
    tnx

  • @sahil_singhai
    @sahil_singhai Рік тому

    Hey how do I delete all that Old data from the DB?

  • @anthonydattolo6297
    @anthonydattolo6297 3 роки тому

    Really cool. Would be interesting to see how FastAPI holds up vs DRF for stuff like jwt authentication flows. In DRF it’s simple you just import some libraries that handle the heavy lifting

  • @steelwolf180
    @steelwolf180 3 роки тому +1

    Wow that's cool :)

  • @pravarsharma
    @pravarsharma 3 роки тому

    How can we store large file on NoSQL dbs as in mongodb we can only have a Document size limit to 16Mb, is cassandra have some kind of limit

    • @CodingEntrepreneurs
      @CodingEntrepreneurs  3 роки тому

      A blob has a limit of 2gb docs.datastax.com/en/cql-oss/3.x/cql/cql_reference/refLimits.html

  • @jorgezapata8540
    @jorgezapata8540 2 роки тому

    awesome

  • @Rob-cq1nf
    @Rob-cq1nf 3 роки тому +1

    👍👍👍

  • @phunguyentrong722
    @phunguyentrong722 3 роки тому

    thanks

  • @sukumarsanu7169
    @sukumarsanu7169 3 роки тому

    is it really necessary to clone git repo

  • @mypegionworld7612
    @mypegionworld7612 3 роки тому

    First.