Great video as always, John. I really enjoy your content, I think it helps me in order to grow as a data engineer. Btw, I have a doubt and I hope you can help me. Do you know any way to generate the _view_state parameter in jscript websites? I usually extract, in that kind of cases, data with selenium but my personal objective is using just requests for everything haha.
hey thanks! I'm glad you liked it. I'm not entirely sure what you mean with your question but as far as i'm aware there's no way of running any javascript without an actual browser. sorry if thats not what you meant!
Hi John! And when does a website need, in addition to cookies, a token that is generated when the user interacts with the page? Thanks for the great content!
That is a good question. I once figured out how to pass that kind of situations. Unfortunately, my method was not useful in all the pages I tried. First of all, you need to read the HTML file and the Jscript is behind it. Normally the token keys are generated by a jscript function. Then, as soon as you find out what function generates the token, you can use Selenium in order to open the page and execute that piece of code passing the jscript command through driver.execute (if I am not mistaken). That execution will return you a token you can use in your requests. As I said in the beginning, this method does not work always but hope this helps. It would be interesting if John talks a little about that kind of situations in a video.
Well.. Thanks for telling what AJAX is, wasn't sure that I can name such stuff this way 😅 I think it's a great video for people who didn't learn this technique yet.
The first 1,000 people to use the link will get a 1 month free trial of Skillshare:
skl.sh/johnwatsonrooney06221
Bro you have no idea how long I have been waiting for this
Thanks I hope you enjoy it
i was waiting for this video 3 montths ago , keep it brother we gonna need to the future
Best video on the topic! Thanks John!
Very welcome!
I hope we're getting a whole playlist on this.
Thank you.
WE hope WE*
@@mrmileitv lol true!
Ayo you helped me so much, after 4 days of figuring it out 🔥
Glad I could help
Great video as always, John. I really enjoy your content, I think it helps me in order to grow as a data engineer. Btw, I have a doubt and I hope you can help me. Do you know any way to generate the _view_state parameter in jscript websites? I usually extract, in that kind of cases, data with selenium but my personal objective is using just requests for everything haha.
hey thanks! I'm glad you liked it. I'm not entirely sure what you mean with your question but as far as i'm aware there's no way of running any javascript without an actual browser. sorry if thats not what you meant!
Hi John!
And when does a website need, in addition to cookies, a token that is generated when the user interacts with the page?
Thanks for the great content!
That is a good question. I once figured out how to pass that kind of situations.
Unfortunately, my method was not useful in all the pages I tried.
First of all, you need to read the HTML file and the Jscript is behind it. Normally the token keys are generated by a jscript function. Then, as soon as you find out what function generates the token, you can use Selenium in order to open the page and execute that piece of code passing the jscript command through driver.execute (if I am not mistaken). That execution will return you a token you can use in your requests.
As I said in the beginning, this method does not work always but hope this helps.
It would be interesting if John talks a little about that kind of situations in a video.
Hello John, could you make a video explaining how to get json data from ajax? Keep with the amazing videos!
Great content, as always. 👌
Appreciate it!
Hi John your setup is amazing! may I learn your arms and monitors
thanks! BenQ PD3200Q 32" main monitor, BenQ PD2700Q 27" second monitor on an Invision gas assisted dual monitor arm: (affiliate link) amzn.to/3O5PhAI
How do handle POST requests that have a specific payload?
Can you do this with python directly, without Insomnia?
Yes you can. I find it easier to work with it while I’m testing without having to write code yet
Well.. Thanks for telling what AJAX is, wasn't sure that I can name such stuff this way 😅
I think it's a great video for people who didn't learn this technique yet.
XMLHttpRequest is such a dumb name, they could have called it HttpRequest lol
Is this really a video on "How to get data from AJAX Requests in Python?"
Hi John, I like your mic please can I use it on Kali Linux??
I do all my recording on Ubuntu so yes I assume so!
@@JohnWatsonRooney thanks
Can I make a request for a video? I would love to see a Twitter webscraper which takes an url and json outputs the entire thread.