This is awesome content. Thanks for sharing/making video on this. Gonna try this using Selenium with Java, I know I may have to write some extra lines of code. Specially with respect to merging the 2 arrays, writing the data into excel and at last sending mail with attachment.
thank you very much for these useful videos on Python and Selenium. Could you please provide vidoes on developing a simple framework including few scenarios to automate using Python & Selenium on any application.
HiMukesh, I have qucik Question - how to get skip with the advertisement/ Deals section to get the correct data. Ex - in the above phones example - if the page has the advertised/Sponsored page. Which in turn giving the wrong details in combination of price and names. Can you plz let us know - how to skip from that with original data
Sir i have a doubt if a website has add to cart button like zomato if you click that than popup window shows up now if i want to extract price from it from different restaurant the xpath changes for every restaurant then how to automate it with all restaurant? Pls reply sir.
I have one doubt when I run FOR loop second time to append data in Excel sheet . No data is apended in Excel sheet then I tried to print the data first then append all in sheet. But no data is printed in output. I am not sure why it is happening that first time when I run FOR loop all data prints from list(finallist) but after that how list(finallist) Don't print data. Please help on this
Hi Mukesh, How would you do if the price is not available for one of the phones? So basically if the lists that result do not have the same length. Cheers
Hi Diego you can just write xpath which will only match with nodes which has phone number or you can use findElements then remove elements which does not have phone. Once you have final list then you can export in excel.
awesome content .I am facing some issue. 'Workbook' object has no attribute 'active' how can I remove this error. I unable to find your video on workbook
hello sir, i tried to do the same but to aliexpress website. i always get just the first 8 results. i have used the scroll function (driver.execute_script('window.scrollTo(0, document.body.scrollHeight );')) but it get me the same result. some times I get the all data when I scroll manually. but not all the time. any help please thank you.
thanks for this fantastic tutorial. i am facing problem when i get name and review data because some product does not have review. so when the review list has a gap it does not match with name. i have tried to find the gap but i failed. suppose site has 12 product but 3 of of them with no review. sel. is providing list of 9 review. which 3 product does not have review. if i could fetch single review at a time then i can find the gap can u help me on this, please. thank you
Hi Rafiuzzaman, you can change xpath. Might be locator must have changed. Please use new locator. Here is the link for xpath ua-cam.com/video/As72m9qrAQ8/v-deo.html
request to provide solution of this error : "RecursionError: maximum recursion depth exceeded while calling a Python object" at the line no 57 - "server.send_meassage(msg)".
@@Mukeshotwani Thanks in any video that there is in this platform nobody makes a video using IIS with python web scrapping and I think it would be quite good for you and your growth
Awesome, I liked this one.
Thanks Naveen
Strongly recommended this video bro.... great
Thank you so much mate.
Excellent Tutorial....... Sending Love
Thank you! Cheers! Subroto
This is awesome content. Thanks for sharing/making video on this.
Gonna try this using Selenium with Java, I know I may have to write some extra lines of code. Specially with respect to merging the 2 arrays, writing the data into excel and at last sending mail with attachment.
Hi Arvind, yes please try with java too because concept is same. If you need an y help then let me know.
thank you very much for these useful videos on Python and Selenium. Could you please provide vidoes on developing a simple framework including few scenarios to automate using Python & Selenium on any application.
Yes, soon its coming Rama stay tunned
Awesome, I like the way you explained web scraping. Can you please explain data comparison between two excel sheets or files?
Awesome … very clear explanation loved it
learned a lot how to use xpaths, thanks!
Great video , simply explained!
Hi Mukesh,
Could you please tell us how to do scrape multiple pages with this script
Thanks
Sir, I get an error in append ..
It shows that :- AttributeError: 'WebElement' object has no attribute 'append'
Please share screenshot over email mukeshotwani@learn-automation.com
Thanks a lot . This video is very helpful .
Thanks Buddy
Amazing 👏👏👏👏👏
my browser closes automatically when code lines ends even if i havent put driver.quit() function too
can u help me with that
Hi Mukesh, thanks for sharing! I realised that the web scrapping stops at page 1. How do you code the loop such that it scrap all pages? thank you
HiMukesh,
I have qucik Question - how to get skip with the advertisement/ Deals section to get the correct data.
Ex - in the above phones example - if the page has the advertised/Sponsored page.
Which in turn giving the wrong details in combination of price and names.
Can you plz let us know - how to skip from that with original data
Hi Mukesh, can you please tell me how to do web scraping with java and store it in excel as you showed with python??
Sir i have a doubt if a website has add to cart button like zomato if you click that than popup window shows up now if i want to extract price from it from different restaurant the xpath changes for every restaurant then how to automate it with all restaurant? Pls reply sir.
I have one doubt when I run FOR loop second time to append data in Excel sheet . No data is apended in Excel sheet then I tried to print the data first then append all in sheet. But no data is printed in output. I am not sure why it is happening that first time when I run FOR loop all data prints from list(finallist) but after that how list(finallist) Don't print data. Please help on this
Hi Rishabh, please debug your script ua-cam.com/video/v_j6iG13va0/v-deo.html
Hi Mukesh,
How would you do if the price is not available for one of the phones?
So basically if the lists that result do not have the same length.
Cheers
Hi Diego you can just write xpath which will only match with nodes which has phone number or you can use findElements then remove elements which does not have phone. Once you have final list then you can export in excel.
Hi, thank you for your demos great content. Do you have the project files in your GitHub repo please?
How if you wanted to add current date and time to the result file in excel as a new column say like DateTime for each line ?
awesome content .I am facing some issue. 'Workbook' object has no attribute 'active' how can I remove this error. I unable to find your video on workbook
Hi Shreya, you need to refer this video first for excel with Python ua-cam.com/video/nsKNPHJ9iPc/v-deo.html
hello sir,
i tried to do the same but to aliexpress website.
i always get just the first 8 results.
i have used the scroll function (driver.execute_script('window.scrollTo(0, document.body.scrollHeight );')) but it get me the same result.
some times I get the all data when I scroll manually. but not all the time. any help please
thank you.
Hi Mate, you can scroll till the last step or next button. Once you get list which might have uplicate data then you can convert into set.
you are awesome, thank you so much for your videos!!
Can selenium extract images as well from e-commerce along with other details in excel cell
You can extract image src in excel.
Really nyc thank you sir
Most welcome Rama Sudha
Thanks a lot. Can you share the code to save some time for us? Regards
Sure Yasser will do soon.
Sir please tell how to do if my data is present in excel file....how to perform in multiple loops
if huge number of data's is there to extract within pages and inside tabs ,how can we extract standardly without missing datas
Hi Pankaj you can do in the same manner as I showed or you can use python lib to scrap complete data.
sir please explain how to put value of send_keys by your keyboard on different positions in code
Hi Zeeshan what do you mean by different positions in code?
Awesome 👍
Thanks 🤗
Thank you very much ...
Welcome kamlesh 🙏
Sorry but I dont see link in description to on How to Send Email in Pyhton with SMTP and with attachments.
Mukesh, please provide link to that.
Hi Bhargav, here is the link ua-cam.com/video/yzfLVL34_oA/v-deo.html
Sir plz shire some more informatic video about webscraping
Awesome
Thank you 😍
Great! Thank you!
Welcome Sir
thanks for this fantastic tutorial. i am facing problem when i get name and review data because some product does not have review. so when the review list has a gap it does not match with name. i have tried to find the gap but i failed. suppose site has 12 product but 3 of of them with no review. sel. is providing list of 9 review. which 3 product does not have review. if i could fetch single review at a time then i can find the gap can u help me on this, please. thank you
You could try implementing Error handling mainly Try Except block and it will surely help you.
ElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//span[text()='Samsung']"} why this?
Hi Rafiuzzaman, you can change xpath. Might be locator must have changed. Please use new locator. Here is the link for xpath ua-cam.com/video/As72m9qrAQ8/v-deo.html
Super
Thanks Sundar
Must watch playlist to learn selenium ❤️
thank you.
Awesome video
no such element: Unable to locate element: {"method":"xpath","selector":"//span[text()='Pedigree']"}
please i am getting like this help me
Sir how can extract data from json website using python?
Hi Rahul, yes you can extract the data in JSON file as well.
But sir how can we extract it. Kindly help me to extract target.com
Why do you use pycharm and not vs code?
why do ppl like burger king instead of wendy's?
request to provide solution of this error : "RecursionError: maximum recursion depth exceeded while calling a Python object" at the line no 57 - "server.send_meassage(msg)".
Do you post your code on Github as well.
Can you run this aplication in IIS
Hi Moises, you can execute python script on any web server or CI sever.
@@Mukeshotwani You can do it in a video
sure Moises will try
@@Mukeshotwani Thanks in any video that there is in this platform nobody makes a video using IIS with python web scrapping and I think it would be quite good for you and your growth
Can someone help me I am getting Stale element reference exception
Hi Janice, You can find element again.
@@Mukeshotwani Thank you
in line 15 how phonename can be iterate?
Hey Paras, you can iterate them as String or Number.
I have a image in browser and image has some data can i extract the data from image and print it in terminal
not working
20:28
?
Bro I need ur support I want speak personally can u please help me bro I will pay if u teach means please let me know