Automated Form Submissions with Chrome Puppeteer & NodeJS
Вставка
- Опубліковано 31 лип 2024
- In this Automated Testing Tutorial, we will be learning how to use Chrome's Puppeteer to automate a form submission process.
Starting from the very beginning, I'll set up Puppeteer and write an automated script for logging in, then submitting a form taking in command line data.
If you enjoy these videos and would like to support more videos, take a look at my WebdriverIO course to see if it may help you:
learn.webdriver.io
Or your can show your support with a one-time donation:
paypal.me/klamping/5
Code:
github.com/klamping/rain-puppet
Links:
Community Collaborative Rain, Hail and Snow Network
www.cocorahs.org/
Puppeteer Homepage
github.com/GoogleChrome/puppe...
Accepting Command Line Arguments in NodeJS
stackoverflow.com/questions/2... - Наука та технологія
Short, neat and explanatory. Nice one Kevin!
It is very important for the development of complex and complicated programs!
Awesome video man.What I needed right now. Thanks
Great video dude, evenly paced and your voice is like caramel!!!!!
Thanks for the tutorial man! it really helps
Subscribed so damn fast, this is what I'm currently required to learn
Excellent demo. Thanks.
Great tutorial! Thank you!
Thanks Kevin great tutorial
This is good man, thank you
Great job, exactly what I was looking for, thanks.
Awesome to hear! Let me know if you have any other ideas for things to try out with Puppeteer automation
Great tutorial. Appreciate your effort!
super very neat!
Awesome video, topic is well explained. Subscribed already...
Great Video !
And to think, that I've lived all these years not knowing that NaN stands for Not-A-Number.
This is the best and practical demo I have seen so far. Is there a way to import form inputs from a JSON file to run through the form and then web scrapping the results?
Thanks. Well narrated!!
Thanks!
Great tips thanks haha
Great stuff! What about if the classes you're trying to grab are dynamic and long? I cannot seem to get it to work on certain websites. Any insight?
Wow! Is it a time for new instrument?)
Thanks for your videos.
I was looking for exactly something like this..
Nice!
You my friend just made me fire entire office workers and save me $$$. Great job
the best tutorial you nailed it , thank you so much, i had one clarification, i have seen that you are seeing constants in the password area but i didn't see that you are importing the constants, can you please guide me how to use constants when in case hardcoded values or user-id logins
I have a problem on deploying it on server how to run it without cmd ? please enlighten me thank you!
Can this tool only work with web form submission or can it also automate logon process of a desktop chat client or a bot like neatbot.
how can i click a td element without id but its a chidren of a table with id ?
Hello Kevin, could you helpme find out how could I substitute: document.querySelector('[title="Send Message to Group"]').click();
I plan to automate some click/paste between two websites, i hope this tool will work 24/7
did you make it work? copy/pasting doesn't seem very straightforward at least on mac
yes, i use at work without problem.
Very useful when you can add some specific logic. Compared to simple Macro application, this tool have advantage.
And you can try another similar tool: Playwright
Hi sir
Suppose we have timer for before login then how will reduce that time bro
hi, is it possible to make this for google form? but i have text type input and file type input.
I also need help making a script to press the checkbox on page
I have a page i'm scraping that will resubmit the page after i click the button. The button stays on the page which causes an infinite loop of page submit. How do i prevent the loop? I just want to click the button once. Instead, it looks for the button, submits then sees the button again and keeps hitting it which causes infinite page reload. Please help if you can.
Hi Kevin great work
if there is image captcha on login how to handle it
Sometimes on login sometimes on submission page if we have image captcha then how to deal it
Please provide solution
Is there a way we can show the captach in a small window and write it and the whole thing automate when I submit it
Or page by page it will show captcha and once I fill captcha rest thing it automates
Nice video!
I have experience in webscraping using Python just tasting node js waters as regards webscraping
Node.js is faster amiright
Hi, great video !! But how does this process.env works ? Do you have a video on it ? Can you pass a brief text about so I can search for it ?
really good video. Would you build a robot on pancake swap using puppeteer?Is it possible to automate all the trading process like buying and selling coins
You da maaaaaaaaaaaaaaan!
Thanks for the video Kevin‼️ As of now which is your go to, WebdriverIO or -Puppeteer?-
Depends on your need. For basic automation (not testing), Puppeteer is a good option. I'd definitely go with WebdriverIO if testing is the goal though.
I am interested about SPA apps, for example a React App, will puppeteer work just fine? especially wait for navigation? will components work fine. As you know with React there is no page loads. I am wondering if you already have a video about testing a fully SPA app. Thanks. This video was very helpful especially the three clicks part.
Puppeteer should still work there, but you may need to add some "wait" conditions to check on elements being loaded. If you're going to be writing tests, I'd recommend using a tool like WebdriverIO that has some of this built-in
I dont understand why you used function waitForNavigation before clicking,? We need to click and than wait for next page, am i right?
github.com/GoogleChrome/puppeteer/blob/v1.19.0/docs/api.md#pageclickselector-options
Can i ask you some questions about this tutorial ?
19:28 cool
what if the element doesn't have an id?
Thanks Kevin great video.. but before i get too far into the Puppetter.. I need to ask a question. Can the headless version of puppeter handle form submission with headless version. As i think of form submisson the normal way needing a display... and i can;t convert it to a post as the form has so much security it you tamper with the form in any way it quits.. and grabbing all the data and posting it direct would be so simple for them to detect.. so which ever headless package I settle on it must be headless and must be able to submit a full blown form without the usual display... I need to know this early on so while waiting for Kevin to respond it would be great if someone who knows could jump in.. Thanks in advance.
Have you reached a solution??
what if we dont have "id" tag for html element?
I just found this video right now and saw your comment. Probably way too late for you (who knows? ^^), but maybe it could help others:
If you don't have an id to work with you can go with any other CSS selector like you would with jQuery $('xxx') or element.getElementsBy...('xxx').
For instance if you have an input with a name attribute, let's say , you can do it with await page.type('input[name="first_name"]', "your value");
Great video, Kevin. Ngl, I got distracted multiple times seeing some of the lines ending in a semicolon and other not.
Heh, that's just my poor habits at work. I prefer always using semicolons, but I often forget. I should get a linter set up for that :)
@@frontendtesting prettier's got you ;D
Does it handle CSRFTOKEN if it was there as a hidden input maybe ?
I think so... it should treat it as a normal website. I'm not really sure what CSRFTOKEN is though.
hi I want to pass parameters like username, password at the time of script running through command ho can i do this?
You can use either environment variables, or command-line parameters to do this. Check out npm packages like Yargs or Dashdash:
www.npmjs.com/package/yargs
www.npmjs.com/package/dashdash
Have you used Playwright? Would love to see a video on that
I have not used it yet, but agreed that it would make for a great video. Would love to compare their features with what WebdriverIO or Puppeteer has
how may i copy and paste text on my browser(on mac system) please help !
You can execute custom JS code that uses the Clipboard API to run copy/paste commands:
developer.mozilla.org/en-US/docs/Web/API/Clipboard_API
Not sure how to do this in Puppeteer off the top of my head.
and for #Shadow-root??
How can we use puppeteer if there is an OTP login ?
Great question! Unfortunately I don't have a good answer. You could 'pause' you script and manually enter the OTP code, but I don't know of a way to fully automate it.
and a captcha comes.
:(
I'm just starting out with Puppeteer but I'm guessing one could use Cookies to login faster 🤔🤔
Yep, definitely!
Can someone please tell me is puppeteer testing is worth to start work on?
It depends on what you want to use it for. It you want to do Automated Testing, I highly suggest using WebdriverIO, which does have puppeteer support built-in so you get the best of both worlds. If you want to use it just to automate some process, then yeah, learning puppeteer is definitely worth it IMO
Doesnt work for me... Can't even make the screenshot. It says... Error: Parsing error: The keyword 'const' is reserved
Pls help...
You gotta learn the basics of javascript first
Hello , Can You make a video for captcha cracking using puppeteer.
Hey... unfortunately, that's not going to happen :) CAPTCHAs are purposefully made to avoid that sort of thing, so it's just not a simple task at all, and I don't have the CS degree to figure it out.
How to deal with recaptcha while using puppeteer
That's a really tough one. The whole point of recaptcha is to prevent automation, so you're going to have an extremely tough time trying to write automated code to get around it. I'd recommend two approaches:
1. Create a specific test server that has a workaround to captcha, either disabling it or providing an alternative
2. Pausing your test when the captcha step occurs and manually entering it, then unpausing to continue
@@frontendtesting Can you please help me out with this issue. stackoverflow.com/questions/57017108/puppeteer-isnt-working-with-lazy-loaded-images-even-after-scrolling-from-npm-p
There is also a Puppeteer Recording extension on Chrome that can auto-export your steps - but it works like shit.
Do these recording tool ever work? The one for Testcafe is hell as well.
Time to make a sub bot lmfaoo
Well of course. How do you think I got to 3k subs :p
@@frontendtesting LMAO
please help me
Sublime editor?? *Closes video immediately* jk haha (or not).
Hah. Used to be you'd get criticized for *not* using Sublime Text
this is for the complete newbs because it took me a min... if you're using VS code you would use the word "code" instead of "subl" 🤦🏽♀
Yeah, Puppeteer is a bit noisy for a full-scale web testing, too much typing and the scripts tend to be too long, which makes is difficult to navigate if you come to it after some time.
Yep. For full-scale testing, I definitely recommend a tool like WebdriverIO or Cypress.io
@@frontendtesting I actually had a try over the weekend, just for fun and learning purposes. And it wasn't as bad as expected. I mean it probably takes a bit longer to type it all, but the tests are really stable and fast, so it will probably pay off later if they are used on a regular basis.
Cypress is fine, but I can't stop thinking whether or not it's so pupular just because their good marketing... when I see the number of basic questions on Stackoverflow or elsewhere, it makes me wonder why people pick a tool they can't handle or work with effectively.
@@silentexploration Yeah, Cypress definitely has some hype to it. But I also think the founders are really smart folks and definitely looking at how to move the industry forward. We can certainly be skeptical, but also appreciate what Cypress introduces to a bunch of folks who wouldn't be interested otherwise.
Great Video !