only-changed and repeat can be used for test burn-in anywhere easily, very cool. I had a question about Checkly, perhaps we could consider it at a point where we build the test architecture and are ready to consider prod testing. At the previous job, prod authentication was a big no, so we did not test on prod. Instead we used vite to build and bundle the app with prod config at PR level, and test localhost:3000 like that. We also capitalized on feature flags and their an approach by myself to shift their testing to PRs. We were doing 1000 releases a year with 2-3 prod issues at most. So things worked. In that kind of a situation, what can Checkly bring to the table, and how do you get around prod auth restrictions?
Checkly is all about enabling developers to test and monitor their preview and prod environments with Playwright. We allow you to run your existing Playwright tests from anywhere and on a schedule. The default and "works out of the box" features require an accessible URL, though. (There might ways to work around this with our private locations feature but I haven't seen someone spinning up a dev-server on localhost with them yet) So for what you describe, unless you spin up a public tunnel to your described dev server, Checkly checks can't test or access your "private app" The Playwright tests run in arbitrary AWS locations at the end of the day. 😅 That said, we do believe in production testing and monitoring because it's the only way to reliably know that production is really up'n'running. Spinning up your app locally with prod config is good test for the new code or feature regressions, but there could still be plenty of things going wrong in prod. With Checkly you can take your existing tests run them on preview deploys and/or prod, but then once you release, run the same (or a subset) of your PWT tests in an interval to really be sure that everything's working at all times. If something's off, you'll receive a proper alert to be able to react quickly. I hope this answer makes sense and happy to explain more. :)
I have an annoying error. Tests work fine when running from vscode or terminal but not from GitHub. The point to same stage environment. What causes that.
Great content as always, we love you Stefan
Thank you for following along! 💙🦝
Thank you!
only-changed and repeat can be used for test burn-in anywhere easily, very cool.
I had a question about Checkly, perhaps we could consider it at a point where we build the test architecture and are ready to consider prod testing. At the previous job, prod authentication was a big no, so we did not test on prod. Instead we used vite to build and bundle the app with prod config at PR level, and test localhost:3000 like that. We also capitalized on feature flags and their an approach by myself to shift their testing to PRs. We were doing 1000 releases a year with 2-3 prod issues at most. So things worked.
In that kind of a situation, what can Checkly bring to the table, and how do you get around prod auth restrictions?
Checkly is all about enabling developers to test and monitor their preview and prod environments with Playwright. We allow you to run your existing Playwright tests from anywhere and on a schedule. The default and "works out of the box" features require an accessible URL, though.
(There might ways to work around this with our private locations feature but I haven't seen someone spinning up a dev-server on localhost with them yet)
So for what you describe, unless you spin up a public tunnel to your described dev server, Checkly checks can't test or access your "private app" The Playwright tests run in arbitrary AWS locations at the end of the day. 😅
That said, we do believe in production testing and monitoring because it's the only way to reliably know that production is really up'n'running. Spinning up your app locally with prod config is good test for the new code or feature regressions, but there could still be plenty of things going wrong in prod.
With Checkly you can take your existing tests run them on preview deploys and/or prod, but then once you release, run the same (or a subset) of your PWT tests in an interval to really be sure that everything's working at all times. If something's off, you'll receive a proper alert to be able to react quickly.
I hope this answer makes sense and happy to explain more. :)
so useful!!! you might have seen what I am struggling with!!!!
Glad it was helpful! 💙🦝
Awesome content!
great video thanks for sharing
Thanks for watching! 🦝
Thank you awesome content
Thanks for watching and following along! 🦝 💙
Thanks!
@Checkly which font are you using
The font should be "Fira Code". :)
@Checkly Which theme are you using ?
The theme is "Yi Dark". :)
I have an annoying error. Tests work fine when running from vscode or terminal but not from GitHub. The point to same stage environment. What causes that.
It's very hard to say without looking at the details. I recommend to set up and agregate trace fails for CI failures and debugging. :) Good luck!
--only-changes=main