How to web scrape data using no code with Octoparse
Вставка
- Опубліковано 10 лип 2024
- Often times we find ourselves in a situation where we need to collect data from websites but perhaps a big hurdle is limitation on the technical know-how that may traditional require extensive coding. In this video, I will provide a practical tutorial showing how you can web scrape data without coding knowledge by showing use cases on extracting data from UA-cam, Twitter and Amazon in the form of structured spreadsheets (Excel and CSV file formats). Particularly, we will be using Octoparse (which is a no-code software and also the sponsor of this video) that you can use to perform web scraping in just a few clicks.
👉 Download Octoparse agent.octoparse.com/ws/430
👉 Black Friday (November 17 to December 3) agent.octoparse.com/ws/431
0:00 Introduction
0:55 Getting started with Octoparse
2:25 Web scraping UA-cam data
5:49 Web scraping Twitter data
7:49 Web scraping Amazon data
13:44 Export web scraped data
19:32 Conclusion
Icons used in the video:
www.flaticon.com/free-icon/we...
www.flaticon.com/premium-icon...
www.flaticon.com/premium-icon...
www.flaticon.com/premium-icon...
Support my work:
👪 Join as Channel Member:
/ @dataprofessor
✉️ Newsletter newsletter.dataprofessor.org
📖 Join Medium to Read my Blogs / membership
☕ Buy me a coffee www.buymeacoffee.com/dataprof...
Recommended Resources
📚 Books kit.co/dataprofessor
😎 Taro (Tech Career Mentorship) www.jointaro.com/r/dataprofes...
📜 Google Data Analytics Professional Certificate imp.i384100.net/google-data-a...
🤔 Interview Query www.interviewquery.com/?ref=d...
🖥️ Stock photos, graphics and videos used on this channel 1.envato.market/c/2346717/628...
Subscribe:
🌟 Coding Professor / @codingprofessor
🌟 Data Professor ua-cam.com/users/dataprofessor...
Disclaimer:
Recommended books and tools are affiliate links that gives me a portion of sales at no cost to you, which will contribute to the improvement of this channel's contents.
#datascience #machinelearning #dataprofessor - Наука та технологія
Hey man, I'm transitioning from another platform and now using OctoParse to attempt pulling some data. Do you know if 2captcha, deathbycaptcha, or anything else out there would be sufficient for getting those captcha pops for every 30-90 rows of data?? I got the project done and it works but this is my only holding pattern at this point. Adding in custom user agents, rotating them, etc has been complete. Next is also residential IP's etc
Option to schedule tasks for future scraping: Leads Sniper allows you to schedule tasks for future scraping, which means that you can automate your data extraction process. Octoparse may not offer this feature.
Thank you for the video sir 🙌🙌
Option to extract data on a recurring basis: Leads Sniper allows you to extract data on a recurring basis, making it a useful tool for businesses that need up-to-date data on a regular basis. Octoparse may not have the same level of recurring data extraction capabilities.
Option to extract data from various sources: Leads Sniper allows you to extract data from various sources, including websites, databases, and APIs. Octoparse may not have the same level of versatility.
great stuff. thanks
Can you search a site that requires clicking elements to navigate from the homepage to a form, then enter search parameters in the form and click to return the search? Or other screen activities like selecting options from a drop-down box.
Yes you can highlight an item and select "click item" this will make the crawler select the item and move to the next page. You can keep doing this until you reach the form you need.
Thanks a lot for sharing useful information.
Glad it was helpful!
Hi,
Can we use free tier of Octoparse to scrape comments from Amazon product page.
Can you please help to extract data in selected items have pdf links
How do I scrape graphs and turn them into data list?
Option to extract data from multiple websites: Leads Sniper allows you to extract data from multiple websites, making it a versatile tool for businesses. Octoparse may not have the same level of multi-website extraction capabilities.
cool, thank u
This is awesome Sir... ❤
Thanks Lakshan
Can one web scrape auction sites like eBay, Heritage auctions etc? I'm interested in pulling historical sales prices for trend analysis.
Is there a tutorial on how to scrape or crawl top reviews of businesses in an industry?
What If I need to enter data first, then scrap the results? For example airways ticket prices. I need to enter set of routes and flightdates
Hi, I think you can try the Skyscanner template. Or you can copy the URL of your flight query (after you have made the query) and enter it into the Octoparse software to perform the webscraping
@@DataProfessor but generating query is the hardest part in this case. It would be grate If we could uplouad a bunch of rules and parameters that are necessary to generate query (set of routes and dates in this case) and launch the scraping :)
@@natsarev have you found any other software that does what you were looking for?
@@superdragonz1 no
octoparse is awesome!
Thanks for tuning in Benedict, if I recall correctly, you’ve written a great tutorial on Octoparse as well.
Hi Bene, nice to find you here in Data Professor's comment section!
how to scrap pages that is redirecting to another page?
Can you scrap multiple pages ?
Lets say i want to scrape a page results from a footbLl leagueq, and all results are gathered in 5 different pages? Can i do that in 1 job?
Yes it auto-detects the pages or you can specify it as well. The Amazon web scraping shown in this video was done on multiple pages.
@@DataProfessor didn't notice that, thanks!
I have a question, Can we use this software to extract data from different Chemistry databases like Reaxys etc?
Theoretically it can be done but we will have to be extra careful on the legal limitations. Public databases like ChEMBL would probably be okay.
hey man, can you help set some ecommerce website to scrape ? (payed)
Ability to handle large amounts of data: Leads Sniper has the ability to handle large amounts of data, making it a suitable tool for businesses that need to extract large amounts of data. Octoparse may struggle with large amounts of data.
Nice content sir
Thank you. 😊
Option to extract data from different formats: Leads Sniper allows you to extract data from different formats, including HTML, XML, and JSON. Octoparse may not have the same level of format support.
Hello Sir, this guide was awesome. I am finding it hard to scrape data for different companies stating their Name | Street Address | contact Info | Email Address | data in that format.
Kindly advice.
Subbed.
Welcome to the channel :)
Affordable pricing for businesses of all sizes: Leads Sniper offers affordable pricing options for businesses of all sizes, making it accessible to small and large businesses. Octoparse may have more expensive pricing options.
Can you scrape Facebook or Instagram?
Are u cho zen from karate kid 2?
Flexible extraction options: Leads Sniper offers flexible extraction options, allowing you to extract data in a way that suits your business. Octoparse may not have the same level of flexibility.
Minimal technical skills required: Leads Sniper requires minimal technical skills to use, making it accessible to a wider range of users. Octoparse can be more difficult to use, especially for those with limited technical skills.
Why Octopharse really hard for login?
And if there's no template? (the search for a useful video on how to use octoparse continues. . . .)
Hi. Is this totally legal and legit? I need a tool to collect comments from Facebook, Instagram to conduct behavioural analysis of consumers. It will be for my business so it has to be compliant to regulations
1st comment
2nd comment
Internet and World Wide Web ARE NOT synonyms, remove that section ASAP.
3rd comment
Option to extract data in bulk: Leads Sniper allows you to extract data in bulk, making it a suitable tool for businesses that need to extract large amounts of data. Octoparse may struggle with bulk data extraction.