I've been using this together with your discordjs playlist to make a bot that reports tickets status from a customer helpdesk site directly to chat, on demand. However i didnt use cheerio, since puppeteer has the ´evaluate´ method wich allows you to select HTML elements and extract inner text or html. keep going dude
Thanks I've been trying to make an app in JavaScript to help my grandma use her phone and she needs something like this to get a list of information from external sites
How do you crawl and scrape the entire page or a whole category automatic and not just one specific page of amazon for an example? great and easy tutorial to follow tho!
You have many options, one example is Vultr. I have a video that explains how to get $100 in free hosting credit here: ua-cam.com/video/C3nC2w-pIdo/v-deo.html
Hey, you could scrape a web page every X hours or days (however frequently you want) and store the price in a database. Once you scrape again you can compare the new price with the previous one in the database, if there's a change then send a message
@@WornOffKeys how can I automate it though? Do I use setInterval or setTimeout? Also how do I compare I appreciate you taking your time to reply in the first place :)
Either of those are fine to use when it comes to repeating the functionality. I wouldn't suggest doing it too often though, as they might block your IP for automated data scraping. For comparing I would look into how to store and read data from MongoDB (or another database technology of your choice) and then storing the price every time you scrape the website. After scraping you can then pull the previous data (if it exists, it won't the first time you scrape for example) from your database and simply use an if statement to compare the two values. I have a playlist on the channel about MongoDB basics. If you want to really fancy with it you could automate the web scraping via AWS Lambda scheduling. That would be a very low cost solution where you won't need a server running 24/7. I have videos on AWS Lambda on my channel as well
🙋♂ Need help? Ask in our Discord community:
wornoffkeys.com/discord
I've been using this together with your discordjs playlist to make a bot that reports tickets status from a customer helpdesk site directly to chat, on demand. However i didnt use cheerio, since puppeteer has the ´evaluate´ method wich allows you to select HTML elements and extract inner text or html. keep going dude
yes its the best video.. solved all my queries.. don't worry
ive been trying to work on this project of mine and i cant thank you enough for the video
Glad I could help!
That's gold! Subscribed! Thank you!
Thanks
I've been trying to make an app in JavaScript to help my grandma use her phone and she needs something like this to get a list of information from external sites
Do you have more on this topic? for more complex websites with subcategories and the like?
How do you crawl and scrape the entire page or a whole category automatic and not just one specific page of amazon for an example?
great and easy tutorial to follow tho!
Please where do you deploy the app with packages like cheerio & Puppeteer included.
You have many options, one example is Vultr. I have a video that explains how to get $100 in free hosting credit here: ua-cam.com/video/C3nC2w-pIdo/v-deo.html
How can I make it that when the price changes, my discord bot sends a message ?
Hey, you could scrape a web page every X hours or days (however frequently you want) and store the price in a database. Once you scrape again you can compare the new price with the previous one in the database, if there's a change then send a message
@@WornOffKeys how can I automate it though?
Do I use setInterval or setTimeout?
Also how do I compare
I appreciate you taking your time to reply in the first place :)
Either of those are fine to use when it comes to repeating the functionality. I wouldn't suggest doing it too often though, as they might block your IP for automated data scraping. For comparing I would look into how to store and read data from MongoDB (or another database technology of your choice) and then storing the price every time you scrape the website. After scraping you can then pull the previous data (if it exists, it won't the first time you scrape for example) from your database and simply use an if statement to compare the two values.
I have a playlist on the channel about MongoDB basics. If you want to really fancy with it you could automate the web scraping via AWS Lambda scheduling. That would be a very low cost solution where you won't need a server running 24/7. I have videos on AWS Lambda on my channel as well
@@WornOffKeys youre very appreciated 🥰
I'm not a big specialist but I think in this particular case using cheerio was unnecessary
08:05
Bro WAAAYYY to many ads!! It's impossible to watch this video