Website login using requests library in Python
Вставка
- Опубліковано 30 лип 2024
- IN this video, learn the step by step procedure to login to a website using requests library in Python.
Example code here: gist.github.com/nikhilkumarsi...
Intro to web scraping:
indianpythonista.wordpress.co...
Intro to requests library:
indianpythonista.wordpress.co...
------------------------------------------------------------
Explore my tutorials: nikhilkumarsingh.github.io/tu...
More awesome topics covered here:
WhatsApp Bot using Twilio and Python : tinyurl.com/y9n9ott5
Serverless Rest API using AWS and Python : tinyurl.com/y8yg4vem
Creating Chat Application using Flask, Socket.IO & mongoDB : tinyurl.com/y8kbn4xu
Curses in Python : tinyurl.com/y8ws3x4l
Discovering Hidden APIs : tinyurl.com/y7sdegoq
RegEx in Python : tinyurl.com/y84cnuns
Python for Data Science : tinyurl.com/yd7x6j5e
Introduction to Pandas : tinyurl.com/ycr4qytt
Introduction to Matplotlib : tinyurl.com/y8caem3r
Introduction to Numpy : tinyurl.com/yc8uhtn9
Functional Programming in Python : tinyurl.com/ybubzum6
Python Package Publishing : tinyurl.com/ya2fn3wb
Multithreading in Python : tinyurl.com/ycnummb8
Multiprocessing in Python : tinyurl.com/y8qkjqtx
Parallel Programming in Python : tinyurl.com/yakl5udt
Concurrent Programming in Python : tinyurl.com/y9ppkkhc
Dataclasses in Python : tinyurl.com/y8zfqleq
Exploring UA-cam Data API : tinyurl.com/y7jhs8rm
Just For Fun : tinyurl.com/y9on8gb5
Exploring AWS : tinyurl.com/ydxmhhz9
Jupyter Notebook (Tips, Tricks and Hacks) : tinyurl.com/y8qdhcrp
Decorators in Python : tinyurl.com/y7ghraa9
Inside Python : tinyurl.com/yd8xqbjj
Exploring datetime : tinyurl.com/y96jzylm
Collections in Python : tinyurl.com/ybxltgvk
Networking : tinyurl.com/y97eroql
Computer Vision for noobs : tinyurl.com/y8pxz436
Python for web : tinyurl.com/y829py3g
Awesome Linux Terminal : tinyurl.com/y8no3zso
Intermediate Python : tinyurl.com/yasmecaw
Tips, tricks, hacks and APIs : tinyurl.com/y9xqzzl4
Optical Character Recognition : tinyurl.com/ybt3to6q
Facebook Messenger Bot Tutorial : tinyurl.com/y9wvjc9g
Facebook: / indianpythonista
Github: www.github.com/nikhilkumarsingh
Twitter: / nikhilksingh97
------------------------------------------------------------
#python #requests #logging
By this time my code consists of: 25% Stack overflow scripts, 10% my own work, 40% people sharing ideas on UA-cam and 25% pure luck (no one know how or why the code works but it does!). Thank you so much for this video!
LOL
Relatable 😃
Dude, This is like the most underrated Python Programming channel I have ever seen.
subscribers = 0
while True:
subscribers+=100
print("Nice Content"*99999)
LOL
it will never print "Nice Content"*99999 since you have an infinite loop
@@simthegreat5690 loool
Very nice!
Just keep in mind that different sites use different tokens to login with. You may need to play around in the inspector for a bit until you find it.
Super cool, I was struggling to login into an ASP login, with your neat steps and some tricks I am able to get in now :)
After hours of searching for a way to edit online forms through a python script. (I'm a noob at requests) This video gave me all the tools I needed in ~12 mins. Wish I had found this earlier...
very clear instructions and explanation. great job and thanks for the tutorial
Really great video. Thank you pretty much.
Thank you! Useful stuff :)
Thank you ! Now I get it with the options like headers and login_data ... and also where I get the login_data dictionary values ! Thanks man ! :)
Before start watching, I want to say thank you !
Finally found what i was looking for ...thank you for high quality content
Thank you. This helped a lot!
Awesome video! Keep up the great work guy!
BestT 12 mins of Educational video ever!!!
I followed the instructions and it really worked
Great video, I learned alot! Keep up the good work!
Super useful and well explained - thanks!
Great tutorial, thanks a mil man!
Thank you very much indeed, this video has helped me doing exactlly what I wanted, and with 0 errors. Now that I understand it thanks to you, I can keep doing my work. You're great :D
You're the best!!!
If I wanted to make a post request, do I need to main website or the request url link in the dev tools?
THANK YOU !! U saved my life!
hey and is there anyway to parse javascript values ? cuz there is a needed header that i need to send on the post method , how can i get this header value ( knowing that its calculated in a js FILE ) from what i have seen in inspector
Thanks for the video ! :)
I have an additional question. I wish to get the redirected url, that is in Web Inspector under the name "location". Yet I cannot see it anywhere in the soup. From where can I get it ?
Thanks for this tutorial, good work.
Thanks for tutorial, helped my project!
Exactly what I needed, thanks for the video!
Bro I have m3u8 which is encrypted
I have key
But I don't know how to decrypt the file ?
Awesome, thank you so much!
amazing video, it looks very simple, thanks man
You're the best!!!
Thank you very much
Great content, sir. Thank you!
THANK YOU FOR THIS VIDEO!
Thank you! It really helped me with my poject
Thank you! Great Video!
Thank you so much! This helped me a lot, with this explanation I was able to submit in multi page forms. Greetings from Brasil! ⭐⭐
i cannot find the form_build... how can we solve that? or what can we do? help please!
The video that I was looking for. Thanks.😊
Thank you for nice explanation.
Hey, i have a python script that fetches attendance for which i use selenium for logging in and everything works pretty fine, but my question is that after ruuning the script it automatically opens up the browser and does the login and then clicks on the specified link to fetch the attendence !...............What to do if i want to do all of this behind the scene without the browser actually been showing up?
good video and explanation! thanks!
And what if in my html there are more then one form_key (because there are more actions (login, register, newsletter)? The key are in the same
Very helpful. Thank you!
OMG! Thank you soooo much!!!!
Great, thanks a lot from Morocco
Hey man I tried this but the website I'm trying has 2 POST requests. One to login, and one to validate with a unique token. The problem is when I try to parse the URL it's using to 'validate' it says error parsing body. So I can get the unique token value. Any tips?
great video, very helpful to scrape website protected with login
Great video. I was trying to do something similar in java without using selenium. This helps.
Still using Java at this modern age for web scraping?
wow bro i learned good thing today
When I run the code, it just prints out thelogin page, why is this happpening
hi, the website i am trying to login does not show the csrf. only when i inspect i can see it. but then it is omitted.
any idea how to get it?
Thank You Nikhil for the video.
I am facing an issue while using this code to login into a microsoft page, specifically Sharepoint page which is a kind of company's internal site.
Also, when I inspect on the sharepoint homepage to look for user-agent and form_build_id, I can find the user-agent, but cannot find the form_build_id information. There is no section of "Form Data" under the Network when I inspect. Is it because the sharepoint login is "Connected to Windows"? Can you please help me out here?
I just ask if i send request and the web site have 504 bad getaway time out , the request success or not please
Dude, thank you!
Hi I'm trying to do this and I'm looking at a website that I'm trying to login, but when I submit my login information and click the "login" button I don't get a "post" notification, just a "get" request which is called app.js, anyone know what to do in this situation?
I can not find form build id in my login form data headers or anything seems like it with a different header name. What can I do about it?
I don't see anything about login data from indexPage.action, only encrypted cookies, what should I do?
I want to login into facebook as a sample. And when I click the "login" button I always find the "www.facebook.com" network process but Its an "get" request with the code "200". I cant seem to find a post request with the categorie "Form Data" with password and username in it (etc.)
Hey dude, do you have something for SAML v2 based Single Sign On?
There is csfr token also now a day how to get that for login_data pls tell
Dude you're awesome! Ty
For those who cannot login access to the url, there's a new item under 'Form Data', which is 'csrfToken'. You'll need to include it to the 'login_data' using the same method searching for 'form_build_id'.
Code for 'csrfToken':
login_data['csrfToken'] = soup.find('input', attrs={'name': 'csrfToken'})['value']
Hello Michelle, were you able to login to the same site after card token addition? I am getting 200 response but the username and password fields are empty
@@ashwinis.chavan4598 Hi, I suppose you need to have an account of the website beforehand. Then, you shall use your password and username in the code.
The website that i want to try to scrap, don't have form data area. So i can't get formbuild id name etc. Can u help about it ?
can someone help me trying to do this for slamjamsocialism but can't find the login request?
um can someone help me out here?
I copied the video word for word but the error 'TypeError: 'NoneType' object is not callable' is showing up
for this line:
login_data['form_build_id']=soup.find('input',attrs={'name':'form_build_id'})['value']
I have no fricken idea as to why this is happening.
thank you very much
you earned a sub , thank bhai
Thank you so much it is very helpful ...nikhil
Very useful. Thanks
Very good. Thanks!
what should i do if there is nothing called 'form_build_id' and there is no value?
skip and watch another video
Check post request
@K H check if your site have something like a token, in the request headerss.
I tried this and got SSL exception
requests.exceptions.SSLError: HTTPSConnectionPool(host='mwhlvcmut2.corp.amdocs.com', port=28501): Max retries exceeded with url: /B2B (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1056)')))
can you please help me here?
Pure Gold !!!
Very helpful video thanks
Awesome man!!!
Very useful video; thank you (y)
Thank you so much!!!
hi sir im not able to log in to my website.. actually it is a website for our college academics, which is acccesebile through a vpn, the get request is working fine but post request fails ..
please help asap
hey dude,im getting the 405 Not Allowed error even after passing the headers..plz help
Very helpful thank you
Can u help me..
How to inspect a website network requests in android?
To get form data
Can you help me with this ?
login_data['csrfToken'] = soup.find('input', attrs={'name' : 'csrfToken'})['value']
TypeError: 'NoneType' object has no attribute '__getitem__
We have something called requests.auth, in which we have a class named HTTPBasicAuth which serves the same purpose. Is it possible for you to explain that?
bro can you help me. my response is 405 error: Method is not allowed.
Thank you so much
Hey cool tutorial. One question, the website i am trying to do this on has an encrypted password in the form data. How can we go about doing it?
did u try passing the encrypted password as the value for password key?
Thanks for the efforts
your example work , I tried with another website in our intranet and I am unable to login,. I just replace the url (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')])")))
How can I do if the developer tool does not show me form data? Please help me with this, thanks in advance
why it is post request?? I have tried one website and login there it was showing me get request with status code 200???????????
How we can scrap the website where SAML authentication is there
I have used the same thing exactly but my problem is different.
After Authentication, there is a long loading page then using content.text i get tags of that loading page html instead of one after the loading is done
i want to login to a website using requests but the login form is an can you explain me how?
Great tutorial
Help please. In your line number 19 in git code its showing "TypeError: 'NoneType' object is not subscriptable."
did you find the resolution ? i am getting same error
so if you don't enter the header is it ok?
Great video. Very helpful for getting started. However what do I do if I get redirected from my login page. For example say I log in at www.example.com, and press sign in, and I get redirected to www.example.com/account
How do I scrape the page from there
Thank you so much bro
Thanks you so much sir
Love it!