How To Build a Web Scraping API for Large-Scale Data Collection Using FastAPI

Поділитися
Вставка
  • Опубліковано 30 січ 2025

КОМЕНТАРІ • 3

  • @CTOSellsideSpA
    @CTOSellsideSpA 3 місяці тому

    Gracias por compartir , saludos desde Chile

  • @joshuatemitayo4544
    @joshuatemitayo4544 2 місяці тому

    Please how does using multiprocessing scale? imo it limits you to number of cpus available and may lead to long wait time on the job queues if all the core are running a task. I think running in async should be better, please educate me

    • @AkamaiDeveloper
      @AkamaiDeveloper  2 місяці тому +1

      Hey there - What Tim may be getting at here is by using FastAPI, an asynchronous framework, along with multiprocessing you're getting the best of both worlds to overcome some of the limitations of python. This may not be the best configuration for everyone though.