I only used Puppeteer in the past, but I found it very similar to how Playwright works. I used it to scrape the data about power interruptions from our local power plant website and send email using Node since they don't provide any notification method. I then run the script using Windows task runner which runs a simple batch file on my local machine once a day :)
You're welcome. Yes, that's what's so fun about these automation features, tying this together with Express or your own sessions / tokens, you could bring screenshot / scraping / testing features to end-users as well.
Hi Brad, I like all of your tutorials and I've purchased 3 of your course on udemy. I want to build a comparison site but need to web scrap the data from the websites I'm comparing so a user can search and filter the data. Is this possible from using this method of web scraping?
Yes absolutely, but it does depend on the specific websites you need to scrape. Some websites that anticipate being scraped will have things set up that detect scraping scripts (if your script is running pretty frequently) and you might need to use a scraping service; but I don't want to advertise or directly endorse any 1 company.
Thanks! Do you mean like unit testing? Or do you mean why didn't I include the screenshot and scraping features within the test / assertion / report area?
As always, a great project based tutorial! By the end of it you will have autogenerated screenshots and scraped data very simple to understand!
I only used Puppeteer in the past, but I found it very similar to how Playwright works. I used it to scrape the data about power interruptions from our local power plant website and send email using Node since they don't provide any notification method. I then run the script using Windows task runner which runs a simple batch file on my local machine once a day :)
Awesome tutorial, ty
You have great presentation style. 👍
Thank you! 🙏
hi master! do you know how to deploy this in vercel or netlify ?
Thanks for the quick tips on Playwright! I guess this can run behind an express server so that users can send certain queries...
You're welcome. Yes, that's what's so fun about these automation features, tying this together with Express or your own sessions / tokens, you could bring screenshot / scraping / testing features to end-users as well.
Hi Brad, I like all of your tutorials and I've purchased 3 of your course on udemy. I want to build a comparison site but need to web scrap the data from the websites I'm comparing so a user can search and filter the data. Is this possible from using this method of web scraping?
Yes absolutely, but it does depend on the specific websites you need to scrape. Some websites that anticipate being scraped will have things set up that detect scraping scripts (if your script is running pretty frequently) and you might need to use a scraping service; but I don't want to advertise or directly endorse any 1 company.
@LearnWebCode Thank you Brad!
Hreat!
But why not to use a test function?
Thanks! Do you mean like unit testing? Or do you mean why didn't I include the screenshot and scraping features within the test / assertion / report area?
@@LearnWebCode
No. Basically I meant why you didn’t use the test() fixture function instead of just a simple function?
You are missing a favicon for your website