Brilliant video! I watched this video at home, put it on TV via Chromebox. Even my girlfriend liked the video - she's not into IT yet the video itself, the presentation, was totally fun to watch. Good job!
I watched almost all videos of this channel and this is my favorite one. You are doing a great job, your videos helped me a lot of times. Greetings from Colombia.
26:10 "Young developers, ......... Jesus that sounded arrogant" 😂😂😂😂 Man your videos are just great! This is what schools and universities need, very joyful people like you who has enough practical experience to be able to teach software engineering in a very entertaining way, Jeez you're so good. Love your videos man, hope you read this comment. You made me love JS and testing which are things hard to make people love easily LOL 😂
Thank you for this video, as a junior dev, test automation is perhaps one of the most important things I haven't explored much yet and there aren't many accessible resources on it. I like that you started with a purpose of tests. Keep it up.
Jeez! Seems like I blinked and you flew from 10k to 50k subs! Congrats! :D I'm glad to see you're popularity start reflecting your competence, character, and awesome teaching style. You're the best Instructor I've found on UA-cam. The attention to clarity and detail is much appreciated. Thanks for spending your Sundays helping myself and tens of thousands of others grasp JS concepts! Keep "wasting" that paper! ;P Here's to 100k subs, you'll get a silver play button soon enough!
As a software tester moving more and more into a development role, this is the first video of yours where I felt I already understood the subject matter. You did a great job communicating it! You're definitely right in that the biggest downside of unit tests is that they don't test the contract between two "units" whereas integration tests do. However, I don't think it's necessary to go to full-blown integration tests to actually test the contract in most cases. Given that the way to unit test is to select a "unit" and mock everything around it, I've proposed the idea of "unit integration" tests, where in order to test a contract between units, you select two "units" and mock everything around THOSE (in this example, one of the "units" from the view, and one from the controller). This, combined with the individual unit tests for those units can really help protect you from making breaking changes whenever you need to refactor things in the area. Do you think that's something worth pursuing, or do you think the idea of wanting three different kinds of automated tests would be too much to ask of a development team?
Hey Joe! Tester -> developer is an interesting move, and it's interesting that you've decided to follow the channel. I wonder if this is a normal career move and if it's a common pattern in my audience.
Well if it helps, among the main reasons I subscribed is because when you're trying to learn to be a good developer, super-language specific tutorials that just go through how to use a particular feature are of limited use. You tend to delve into when to use particular things, what they're good for, and why. That really helps when you're trying to learn how to develop and engineer, not just how to write code. In my experience, tester -> developer is a fairly standard career move, I've seen it a couple of times. What I've never seen is developer -> tester :)
I'm interested in this question too! It makes since to me to just extend the boundaries of what a unit test is to test the contracts; your definition of a unit integration test would do that. This makes it easier to write test code IMO. If the one downside to a unit test is not being able to test the contracts between units, then why not just invent a type of test that just tests the contracts only (ala unit integration test)?
There are already. Software engineering formally defines the integration testing that is end to end, system tests. And keep the integration testing name for tests that cover more than a unit. In the video, MPJ cites that you can stub out components inside integration tests. I should add that "contract" testing is not that rare. "Growing oo software guided by tests" book uses it with spy/mocking frameworks (these containing expect, record and rewind), and last few chapters are examples and discussions about how this couples your test and implementation. Your test may become a copy of the protocol between the two objects that is already better expressed in code. Again, as MPJ talk the costs of such tests are higher than unit testing. Some would argue that end to end testing and unit testing should have cost effective coverage enough for most cases (where your software is not going through formal validation or is safety critical). While being in Java, "Growing oo software guided by tests" is a nice book about design smells that appear when you try to test your code.
Very clear video. I was taught about unit testing on my onboarding for junior dev job, but integration testing was very much a black box for me. This helped a lot!
Awesome video! I'm a senior developer and couldn't think in a way more didatic to talk about both kind of tests. Congrats and keep making those awesome videos! You won my sub.
I was able to make automated testing one of our teams written goals (it's now part of our quarterly performance review) by leading by example. Show them why they need to write tests by demonstrating what they do!
Very excellent breakdown of these two testing methods. If anything I've taken away from it so far is that the first step is just to get it in there and get dirty and write trial and error tests until it starts fo make sense. Looking forward to the more in depth examples to come. Thanks for the food for thought, as always.
thanks again for another hilarious and instructive video! maybe you could consider doing the summary at the beginning of the video. 30 mins is kinda long and your 1 min wrapup gets your point over pretty well too.
I agree both are needed. I usually describe the reason for both as the unit tests help prove the functionality works as the developer envisioned it and the integration tests, provided they were written from a user perspective prove that the whole behaves as desired. I also just like the code structure you end up with as a byproduct of making it testable. Thanks for a great video as always!
I watched this 20 minutes before a technical interview in which I was asked "Can you tell me the difference between a unit test and an integration test?" Needless to say, I had no problem with that particular question, even if I bombed the rest of the interview :-D
11:28 downside of unit tests 11:36 1. they do not test the contract 14:14 integration test 14:34 2. 17:56 downside of integration tests 18:01 expensive 18:18 meaning of expensive: 18:24 1. slower to run 20:12 2. *brittle* 21:13 3. harder to write 18:07 cannot simulate errors 18:15 they don't tell you where the problem is
They say in order to grow you need to find a excellent teacher and you sir are more than excellent. Thank you for the insightful video. Could you please create a video series on integration testing just like you did for unit testing
Aside from execution time, writing integration tests for applications or services in data-intensive or complex environments takes a lot of development time. Spinning up a dummy database is one thing, populating it with all of the data needed by your application is quite another. And keeping that dummy data in sync with future changes to the production database schemas is a major commitment.
Another great talk thank you, with a well balanced presentation of the issues. Many of the projects I've worked on have had raging arguments over testing - the die-hard unit testorists on one corner who want to burn the integration tests and the integrationists on the other side (not many in favour of no testing though). My own experience has been that, generally, it is mostly integration tests that catch regressions more than unit tests. I think that is because most unit tests I've seen are too low-level and brittle: they just test every individual internal function with all context removed or faked (which is then a huge maintenance cost when you have to change the internal details for some reason). (I even saw one unit test for a function that calculated the distance between two points; it mocked the call to Math.sqrt to make sure it was called with the right number, but didn't bother checking the actual final answer was correct.)
You know a great deal more about javascript than I ever will and I frequent a lot of your videos. However, I am proud to say that I know a little bit more about automated blackbox testing I believe (you labeled this as integration testing throughout the video). This is because I work as quality assurance, rather than as a developer for corporations, so it's usually not my responsibility to write the app, but test the app. I'll highlight some other things for other people interested: Selenium grid is one method in which you can run integration tests in parallel. It turns a 8 hour nightly test run into sub 15 minute runs at best for individual tests. The only limitation is the amount of browsers you can physically open on the machine. Page object model is the approach I go about to writing solid integration tests. It is a hefty amount of boilerplate, but it is tried and true, and it gives an approach as if you're writing -an app to test an app-, rather than tests of an app. The philosophical approach of using page object model requires knowledge of the concept of scraping, which is what the "brittle" parts the op was describing and he is without a doubt correct. Lastly - screenshotting, or grabbing stack traces isn't too hard from the client, however getting serverside (or even db) stack traces is difficult or even impossible if you stick to a blackbox model for your integration tests.
I've always felt that most people get scoping on integration tests wrong. While it is important to test unit interactions (contracts), it is not necessary to test them in the context of a holistic "day in the life" of some sort of data transformation. If you're doing proper unit testing, you've already got tons of facade/mock implementations of your system's components lying about, so you should be putting them to good use and limiting the scope of any given integration test to specific contract behavior. Done correctly it effectively eliminates two of the downsides you cited as it gives you the granularity to both simulate errors and detect where problems exist. It increases how expensive integration testing is as you will no doubt have to generate/spin up excessive amounts of boilerplate data/or components, but I view this as the price of sanity in testing. That said it does potentially afford you significant savings by limiting the combinatorics of mutli-component/platform testing. I've never been in an organization with the (mis)fortune of trying it both ways, but it isn't too hard to imagine the small scope approach's expense being similar to that of a test suite with too large a scope defined when all is said and done.
I felt you could have simplified it better, but definitely appreciate it. It's clear to me what it is though, it took a few times to watch this video though.
On the subject of integration tests... I would recommend CasperJS, which is basically an assertion library on top of PhantomJS. It's very nice, and also compatible with SlimerJS if that's your thing. Even better is nightmare (github.com/segmentio/nightmare). Nightmare uses electron under the hood. The plus side of this is that you can actually integrate with your existing node code fairly easily. This would be much harder to do in Casper / Phantom, because they run in their own execution environment.
E.g., you can combine mocha (or your testing library of choice) with Nightmare to write your integation tests. Which is really neat. One of my beefs with Casper is that you have to learn a bit extra casper-specific stuff to get rolling.
Has anyone used nightwatchjs.org/ it's using Selenium's JSON API you can write tests in Node. Compared to Phantomjs and Electron shell it's quite battle tested solution. I have experience with Casper.js(Phantomjs engine) but it's quite difficult to debug if something breaks and it usually does.
Great video! Just one thing about the downsides of integration tests. One of the most downside is they don't design your components. This is very often to be forgotten that main purpose of the unit tests is to design your components. Just my 2 cents.
Hi MPJ, a big fan here.. What are your thoughts on ORMs? (such as elloquent etc.) 1. Use almost-pure ORM? 2. Use database 3. Use a combination of both? What do u use? (maybe an inspiration for a video here)
Great video, and I love your way of explaining trade-offs. But I have one critique. It would've been a lot more helpful to answer the "which kind of testing is more useful/important?" question (EDIT: after discussion, I realized I'm actually re-stating the question as "where should I start learning testing?") by saying something like, "unit tests improve code quality and integration tests test functional correctness - so pick which characteristic is more important to you/your project and start with that". You did touch upon these earlier in the discussion of downsides, and I believe it would've helped a lot to make those explicit here rather than making a funny but ultimately superficial analogy to the human body :) Btw, if anybody is interested in understanding this further, there's a section in the book "Growing Object-Oriented Software" that explains this *really* well. That book contains lots of fantastic advice for testing although I wouldn't recommend it to a beginner.
+Vicky Chijwani maybe I'm misunderstanding you, but I just don't subscribe to the notion that you can pick or prioritize one or the other, and I don't want to encourage that line of thinking.
+funfunfunction fair enough. But to clarify my point: from the perspective of a total beginner, they have to start *somewhere*, and in that phase all this testing business is super overwhelming. They can't just start writing unit + integration tests right off the bat, especially because they also have to sell it to their teams/managers (which you explained how to do eloquently, for unit tests, in another video). So I guess my point is, it helps to know which aspects of a project are impacted by which kind of testing. I don't want to pretend I'm an expert though, having just embraced testing recently. Not long ago I was in the phase where I would write tests willy-nilly with no coherent strategy and no idea of what to expect out of it, and it was hard to appreciate the value of tests. Only after consuming many more talks and articles did I get that crucial knowledge.
+Vicky Chijwani oh, then I understand a bit better. If we redefine the question we're talking about as "what tests should I start learning" or "how do I get my team started", then that is a much more interesting and valuable question to me. I agree the learning curve for getting started with automated testing is very problematic, and that also goes for organizations learning it as well (selling to inexperienced software managers etc). I don't have a good answer for that, but I think it merits exploration.
Testing Javascript is hard at the beginning. You need to understand a lot of concepts. Testing Frameworks, asserting libraries, BDD vs TDD, headless browsers, test runners. Also there are a LOT of those in the wild. Its very confusing to know if use Mocha or Jasmine, or maybe Karma? Also if you need to test the DOM, this is another complexity layer, would you use Phantom JS, or maybe JSDOM? @MPJ could you please make some videos regarding these topics? Your videos are AWESOME!! You encourage us to do testing in our applications, and we REALLY want to, but setting up the right environment to make them work is not easy.
Good video as always but I don't quite get the "doesn't test contract" part, in unit test you have to test the public interface, that's what unit test is for, and even if you're doing integration test (or end to end test) it won't point out where you got wrong, it's just test the public contract of the entire app (or the usecase).
F# has contract checkers for the database and others through Type Providers (apparently Idris has recently added them too). This makes for one less test that needs to be done! The downside, as your app grows and connects to more and more services your app takes longer to open/compile than before. So, you have to put that part of your data into other projects, AKA modularize your app. Or, even just create a smaller app to begin with.
Containers to mitigate >some< integration test expense. Docker might have that thing you don't want to spend time on. Can also spawn a cloud of bee's to stress you out.
I tend to write my unit tests defensively, e.g. I tend to write them in a way that expects someone to change the code I'm testing in a way that messes things up. Also, what the deal with QAs obsession with coverage? I've seen getters and setters removed because they "weren't being used or tested, and so were removed to increase coverage" -but their placement in the object were there to signal what fields could be changed without f-ing up internal state. *sigh*
This was great! Thanks, MPJ! Now a question. Because integration tests are so expensive, how should I go about prioritizing which integration tests to perform against a given application? Should I just be testing core integrations - like payments, to touch your example - and work "outward"? I'm assuming units become the bulk of the suite, but by how much?
I have a similar question. Because integration tests are so brittle, it seems like you have to draw some limits. Is there a certain range of number of integration tests you've found that are good? 300-400? I know there is no hard and fast answer, but would appreciate hearing what your thinking would be about making these kinds of decisions. Love your videos!
Amazing vid MPJ, helped me a lot! I liked the fast forwarded summary at the end. Would you be able to clarify where E2E (end to end) testing would fall in this whole story? I've read quite a couple of articles talking about the pyramid-like hierarchy that tests should (loosely) follow, which, from top to bottom is: E2E, integration and unit tests. What would an E2E-test entail in a JS context? Is it just a nomenclature thing and does your definition of integration tests already cover this?
Do not test contract? If you have done it deeply then how you ensure some foreign point was called or something logged without accessing the database without forgetting to test whether a webservice or a log to the db was called? You might let it run inside but then you threaten yourself to stumble upon many additional mocks needed. Test testing a unit from 2010 failing when adding a change to 2020 code. It sounds easy like dont do this cause its difficult, but you dont support it by facts.You can rely on a specific curl handler to be called, but it might be conditioned by ten settings which you must mock for a simple call then, one method breaks tests of 10 different classes
Love your videos MPJ. One thing I've noticed, I believe it was since the last episode - the audio quality has gotten worse. It sounds a lot more muffled. Maybe you could fix this?
The mic he got is, I believe, a Lavelier mic. It's actually a really really good microphone. It just takes some time to get used to - every microphone is different. I'm sure MPJ will figure out the best way to utilize this new mic. Give him time.
Hi MPJ Could you please make a video about reactive programming (RxJS specifically)? Every video I see is always like "ah yeah, u can use events like arrays and do filter and map and all that great functional stuff". Can you come up with some real-world examples to clarify what RxJS is good for? Many thanks :)
Thanks mate! I had seen that video already and some few more from Ben Lesh. Probably I have to clarify my question a bit further. It's not that I didn't understand the technical part of RxJS, but I don't see that much value add in using such a library. I was wondering if there are some real-world scenario examples (besides of the cancellable requests), which make me feel like: "yeah, I would absolutely need RxJS for that". But thanks anyway for your help, I appreciate it very much :) Kind Regards, Steve :)
What about a test that clicks on a button, but mocks payment system to simulate error? Will it be an integration or unit test? or a hybrid? how would you call it?
+Juan Pastás it's just words, really. It's like developer or programmer or software engineer. I use integration tests as a catch-all term for non-unit tests. At Spotify we called the tests you refer to as integration tests.
Hey MPJ, I can't tell you how long I have been waited for an episode like this. Thanx a lot to cover this topic. My Question to you is: I do not really have the possability to setup fake databases to run integration tests with fixed expectations. Since I use angular, I can easily unit test directives and check something like this: expect(myDirective.find('h1').text()).toEqual('1.75€) But without being able to set the value to 1,75€ in the database, I can only check something like this in an integration test: expect(myDirective.find('h1').text()).toMatch(/\d.\d\d€/) Am I right with this assumption or how should I integration test it without a mock database?
CaRDiaK You are so right, but please tell that my organization. As front-end developer I am several layers far away from those guys who are programming the access layer that not even have the permission to check if they program against an abstraction or not. of course I can mock the layer next to me, but then this is no full Integration test anymore.
Do you consider integration tests and end to end tests to be the same thing? Because it seams like you are more talking about end to end tests in this video.
We use the term functional test when a browser is involved. Our integration tests run the business logic without wrestling with the UI. In our case server side. But in any case, this was a spot on summary of my experience. Although I am a bit sad that there is no easy way to create fully covered tests without building the boilerplate, this was really a great video. It will help our devs understand the motivation to have both layers. So thank you very much!!
I couldn't disagree more about integration tests having all these issues. I agree integrations are substantially slower and harder to setup,. I also think unit tests are much more important than integration tests. HOWEVER, that doesn't mean that integration tests commonly suffer from something like inability to simulate errors. Integration tests are from the highest level you can test. You can absolutely mock up systems to generate errors. Yes mock up. You can mock up fake processes, services, databases at a high level. It doesn't need to be an in code contract for you to mock up a fake external script, processes, web service, etc. The only thing about integration tests that is substantially more costly is the initial setup of code and a dependencies. After that it might be easier to write than unit tests. Integration tests can also replace QA in some fields. Writing firmware for an MRI machine? That needs QA. Somebody can die from a bug and therefore the failure acceptance rate asymptotes at none. Writing a flight booking system website? The tests here can be fully automated and don't need QA. What's the worst that can happen? Somebody books a flight and it doesn't work? OK well fix that unlikely bug and then take $5000 from the $300,000 that you saved on a QA team and pay the customer for the inconvenience. Unit Tests are FOR the developer Integration Tests are FROM the customer Do both, but if you're stressed for time do unit tests and leave the rest to QA.
Great video as always! I don't feel you made a great case for integration tests though. If you have to write more complicated tests, in greater quantities, and they don't tell you where errors are (which is kind of the point of tests) how come we need them just as much as unit tests? Excuse the noob question, have a great week.
Part of the argument he made is that Integration tests do a better job of testing the contract between the db, controller, and view layer. When you write a unit test, a lot of assumptions are made about what input the system will receive. The controller could have been refactored and as a unit works just fine still but may have broken the view layer. So you need something that takes a look at the application as a whole.
The whole "not testing the contract" thing is a very big deal. Using the example in the video, if in reality your view doesn't actually make the same calls to the controller that the controller's unit tests are mocking, then your controller's unit tests have zero value, but they will keep on passing and will never inform you that there's a problem.
I'm not sure he said we need both to equal degree, just that we need them both. Once your application works fine on the system you typically use, you deploy integration tests to see if it also works on other hardware, OS, browser, etc. When doing unit tests, you dig into the dirt to find specific bugs. Hope this helps.
gregg4 It does! thanks for replying. I got a bit confused by the heart and brain simile. Hadn't thought about the runtime environment. Still, integration tests sound like they require lots of planning an maintenance and I'm extremely lazy.
I don't think it's reasonable to run integration tests on the fly. I prefer having them on the build server and let them run when you submit a PR. Sure - it's annoying if you submit a PR that breaks a test, but it's quite rare, so I prefer not having to run them while developing.
Brilliant video! I watched this video at home, put it on TV via Chromebox. Even my girlfriend liked the video - she's not into IT yet the video itself, the presentation, was totally fun to watch. Good job!
+Māris Zemgalis thanks a ton!!!
I watched almost all videos of this channel and this is my favorite one. You are doing a great job, your videos helped me a lot of times.
Greetings from Colombia.
An integration test is something that almost made me fail Calculus 2 😂
Lmaooo... i think it was like a sequence is convergent if and only if the integral from 0 -> infinite is convergent r smth?
26:10 "Young developers, ......... Jesus that sounded arrogant" 😂😂😂😂 Man your videos are just great! This is what schools and universities need, very joyful people like you who has enough practical experience to be able to teach software engineering in a very entertaining way, Jeez you're so good. Love your videos man, hope you read this comment. You made me love JS and testing which are things hard to make people love easily LOL 😂
The best start of the week! Every week! LONG LIVE FUNFUNFUNCTION!
Thank you for this video, as a junior dev, test automation is perhaps one of the most important things I haven't explored much yet and there aren't many accessible resources on it. I like that you started with a purpose of tests. Keep it up.
I like the speedy summary at the end of the video
Jeez! Seems like I blinked and you flew from 10k to 50k subs! Congrats! :D
I'm glad to see you're popularity start reflecting your competence, character, and awesome teaching style.
You're the best Instructor I've found on UA-cam. The attention to clarity and detail is much appreciated.
Thanks for spending your Sundays helping myself and tens of thousands of others grasp JS concepts!
Keep "wasting" that paper! ;P Here's to 100k subs, you'll get a silver play button soon enough!
Thank you so much for your kind words, Ryan. Thanks for staying with me for so long, I'll toast to you when I get the silver button.
Your TL;DR summary at the end was fantastic! Every development UA-cam tutorial should have that :D
As a software tester moving more and more into a development role, this is the first video of yours where I felt I already understood the subject matter. You did a great job communicating it!
You're definitely right in that the biggest downside of unit tests is that they don't test the contract between two "units" whereas integration tests do. However, I don't think it's necessary to go to full-blown integration tests to actually test the contract in most cases. Given that the way to unit test is to select a "unit" and mock everything around it, I've proposed the idea of "unit integration" tests, where in order to test a contract between units, you select two "units" and mock everything around THOSE (in this example, one of the "units" from the view, and one from the controller). This, combined with the individual unit tests for those units can really help protect you from making breaking changes whenever you need to refactor things in the area. Do you think that's something worth pursuing, or do you think the idea of wanting three different kinds of automated tests would be too much to ask of a development team?
Hey Joe! Tester -> developer is an interesting move, and it's interesting that you've decided to follow the channel. I wonder if this is a normal career move and if it's a common pattern in my audience.
Well if it helps, among the main reasons I subscribed is because when you're trying to learn to be a good developer, super-language specific tutorials that just go through how to use a particular feature are of limited use. You tend to delve into when to use particular things, what they're good for, and why. That really helps when you're trying to learn how to develop and engineer, not just how to write code.
In my experience, tester -> developer is a fairly standard career move, I've seen it a couple of times. What I've never seen is developer -> tester :)
I'm interested in this question too! It makes since to me to just extend the boundaries of what a unit test is to test the contracts; your definition of a unit integration test would do that. This makes it easier to write test code IMO. If the one downside to a unit test is not being able to test the contracts between units, then why not just invent a type of test that just tests the contracts only (ala unit integration test)?
There are already. Software engineering formally defines the integration testing that is end to end, system tests. And keep the integration testing name for tests that cover more than a unit. In the video, MPJ cites that you can stub out components inside integration tests.
I should add that "contract" testing is not that rare. "Growing oo software guided by tests" book uses it with spy/mocking frameworks (these containing expect, record and rewind), and last few chapters are examples and discussions about how this couples your test and implementation. Your test may become a copy of the protocol between the two objects that is already better expressed in code. Again, as MPJ talk the costs of such tests are higher than unit testing.
Some would argue that end to end testing and unit testing should have cost effective coverage enough for most cases (where your software is not going through formal validation or is safety critical).
While being in Java, "Growing oo software guided by tests" is a nice book about design smells that appear when you try to test your code.
Very clear video. I was taught about unit testing on my onboarding for junior dev job, but integration testing was very much a black box for me.
This helped a lot!
Awesome video! I'm a senior developer and couldn't think in a way more didatic to talk about both kind of tests. Congrats and keep making those awesome videos! You won my sub.
@24:00 when you said "network errors" youtube bugged out lol! First testing video I've ever been able to watch and enjoy. Thanks FFF!
One of your best episodes by far !!!
love this, Now I just need to convince my team to keep writing unit tests :)
Same situation here..!
I was able to make automated testing one of our teams written goals (it's now part of our quarterly performance review) by leading by example. Show them why they need to write tests by demonstrating what they do!
Very excellent breakdown of these two testing methods. If anything I've taken away from it so far is that the first step is just to get it in there and get dirty and write trial and error tests until it starts fo make sense. Looking forward to the more in depth examples to come. Thanks for the food for thought, as always.
thanks again for another hilarious and instructive video!
maybe you could consider doing the summary at the beginning of the video. 30 mins is kinda long and your 1 min wrapup gets your point over pretty well too.
I have been watching all the episodes so far. I have noticed you are becoming serious day by day !! Please don't loose your funny bone!!
+Surajit Mukherjee thanks for this feedback. My mood has not been great lately for a number of reasons. I should probably address that.
Autumn is coming, hope your mode gets better soon :) And thanks for sharing those nice thoughts in videos :)
this guys hair rocks.👍 I'm gonna screenshot and tell my barber, do this
Really awesome video. Answered a lot of questions I had regarding tests.
Im leaving a comment cz this video is significantly helpful and to tell you my "thank you"
The summarization at last is fantastic!!!
I agree both are needed. I usually describe the reason for both as the unit tests help prove the functionality works as the developer envisioned it and the integration tests, provided they were written from a user perspective prove that the whole behaves as desired.
I also just like the code structure you end up with as a byproduct of making it testable.
Thanks for a great video as always!
Another very, very great video. You are extremely talented in teaching via video!
I watched this 20 minutes before a technical interview in which I was asked "Can you tell me the difference between a unit test and an integration test?"
Needless to say, I had no problem with that particular question, even if I bombed the rest of the interview :-D
11:28 downside of unit tests
11:36 1. they do not test the contract
14:14 integration test
14:34 2.
17:56 downside of integration tests
18:01 expensive
18:18 meaning of expensive:
18:24 1. slower to run
20:12 2. *brittle*
21:13 3. harder to write
18:07 cannot simulate errors
18:15 they don't tell you where the problem is
They say in order to grow you need to find a excellent teacher and you sir are more than excellent. Thank you for the insightful video. Could you please create a video series on integration testing just like you did for unit testing
i was waiting for this episode :)
me too ;)
me three
me hungry
Great video explained the difference between ut and it, keep posting good stuff like this.
Make a video with some examples of both testing mthods pleaseeee
Yes, maybe a code session creating an app (like a todo list) using unit tests and integration tests.
There are so many examples already available in the net..
Aside from execution time, writing integration tests for applications or services in data-intensive or complex environments takes a lot of development time. Spinning up a dummy database is one thing, populating it with all of the data needed by your application is quite another. And keeping that dummy data in sync with future changes to the production database schemas is a major commitment.
Another great talk thank you, with a well balanced presentation of the issues. Many of the projects I've worked on have had raging arguments over testing - the die-hard unit testorists on one corner who want to burn the integration tests and the integrationists on the other side (not many in favour of no testing though).
My own experience has been that, generally, it is mostly integration tests that catch regressions more than unit tests. I think that is because most unit tests I've seen are too low-level and brittle: they just test every individual internal function with all context removed or faked (which is then a huge maintenance cost when you have to change the internal details for some reason). (I even saw one unit test for a function that calculated the distance between two points; it mocked the call to Math.sqrt to make sure it was called with the right number, but didn't bother checking the actual final answer was correct.)
Hey mpj would you do a video on commonly used error handling patterns?
Yes, please!
Just thank's to clearly explain tests types as well !
You know a great deal more about javascript than I ever will and I frequent a lot of your videos.
However, I am proud to say that I know a little bit more about automated blackbox testing I believe (you labeled this as integration testing throughout the video). This is because I work as quality assurance, rather than as a developer for corporations, so it's usually not my responsibility to write the app, but test the app. I'll highlight some other things for other people interested:
Selenium grid is one method in which you can run integration tests in parallel. It turns a 8 hour nightly test run into sub 15 minute runs at best for individual tests. The only limitation is the amount of browsers you can physically open on the machine.
Page object model is the approach I go about to writing solid integration tests. It is a hefty amount of boilerplate, but it is tried and true, and it gives an approach as if you're writing -an app to test an app-, rather than tests of an app. The philosophical approach of using page object model requires knowledge of the concept of scraping, which is what the "brittle" parts the op was describing and he is without a doubt correct.
Lastly - screenshotting, or grabbing stack traces isn't too hard from the client, however getting serverside (or even db) stack traces is difficult or even impossible if you stick to a blackbox model for your integration tests.
I may be wrong, but I think I'm seeing excellent FUT results here. Great video!
I've always felt that most people get scoping on integration tests wrong. While it is important to test unit interactions (contracts), it is not necessary to test them in the context of a holistic "day in the life" of some sort of data transformation. If you're doing proper unit testing, you've already got tons of facade/mock implementations of your system's components lying about, so you should be putting them to good use and limiting the scope of any given integration test to specific contract behavior. Done correctly it effectively eliminates two of the downsides you cited as it gives you the granularity to both simulate errors and detect where problems exist. It increases how expensive integration testing is as you will no doubt have to generate/spin up excessive amounts of boilerplate data/or components, but I view this as the price of sanity in testing. That said it does potentially afford you significant savings by limiting the combinatorics of mutli-component/platform testing. I've never been in an organization with the (mis)fortune of trying it both ways, but it isn't too hard to imagine the small scope approach's expense being similar to that of a test suite with too large a scope defined when all is said and done.
I felt you could have simplified it better, but definitely appreciate it. It's clear to me what it is though, it took a few times to watch this video though.
Good describing and clear understanding.
Impressive video. Thx from France !
I got distracted for a moment when I saw the Simon Stalenhag painting in the background.
Good topic. Would love to see something about UI testing too in you show. Until next Monday. Keep up the good work!
Love this explanation, thank you!
I created a new list in my account called Amazing Dev just for this video :)
On the subject of integration tests... I would recommend CasperJS, which is basically an assertion library on top of PhantomJS. It's very nice, and also compatible with SlimerJS if that's your thing.
Even better is nightmare (github.com/segmentio/nightmare). Nightmare uses electron under the hood. The plus side of this is that you can actually integrate with your existing node code fairly easily. This would be much harder to do in Casper / Phantom, because they run in their own execution environment.
E.g., you can combine mocha (or your testing library of choice) with Nightmare to write your integation tests. Which is really neat. One of my beefs with Casper is that you have to learn a bit extra casper-specific stuff to get rolling.
Has anyone used nightwatchjs.org/ it's using Selenium's JSON API you can write tests in Node. Compared to Phantomjs and Electron shell it's quite battle tested solution. I have experience with Casper.js(Phantomjs engine) but it's quite difficult to debug if something breaks and it usually does.
couchdb fauxton uses nightwatch github.com/apache/couchdb-fauxton/blob/master/test/nightwatch_tests/nightwatch_README.md
Great video! Just one thing about the downsides of integration tests. One of the most downside is they don't design your components. This is very often to be forgotten that main purpose of the unit tests is to design your components. Just my 2 cents.
+Anton Kalcik great point!
Nice topic! I had the unit test tab for my CMS already open this morning ;-)
Watching this video feels like I'm watching my flash back life :D. I like this. GIVE THUMBs!!! Thousands times!! haha
Excellent video! But where is the coffee break?
+MindBodySoul Developer haha, I only do them sometimes. ;)
Awesome way of speaking.
And great that you've included the summary at the end.
Hi MPJ, a big fan here..
What are your thoughts on ORMs? (such as elloquent etc.)
1. Use almost-pure ORM?
2. Use database
3. Use a combination of both?
What do u use?
(maybe an inspiration for a video here)
Great video, for real man, easily explained!
Great video, and I love your way of explaining trade-offs. But I have one critique. It would've been a lot more helpful to answer the "which kind of testing is more useful/important?" question (EDIT: after discussion, I realized I'm actually re-stating the question as "where should I start learning testing?") by saying something like, "unit tests improve code quality and integration tests test functional correctness - so pick which characteristic is more important to you/your project and start with that". You did touch upon these earlier in the discussion of downsides, and I believe it would've helped a lot to make those explicit here rather than making a funny but ultimately superficial analogy to the human body :)
Btw, if anybody is interested in understanding this further, there's a section in the book "Growing Object-Oriented Software" that explains this *really* well. That book contains lots of fantastic advice for testing although I wouldn't recommend it to a beginner.
+Vicky Chijwani maybe I'm misunderstanding you, but I just don't subscribe to the notion that you can pick or prioritize one or the other, and I don't want to encourage that line of thinking.
+funfunfunction fair enough. But to clarify my point: from the perspective of a total beginner, they have to start *somewhere*, and in that phase all this testing business is super overwhelming. They can't just start writing unit + integration tests right off the bat, especially because they also have to sell it to their teams/managers (which you explained how to do eloquently, for unit tests, in another video). So I guess my point is, it helps to know which aspects of a project are impacted by which kind of testing.
I don't want to pretend I'm an expert though, having just embraced testing recently. Not long ago I was in the phase where I would write tests willy-nilly with no coherent strategy and no idea of what to expect out of it, and it was hard to appreciate the value of tests. Only after consuming many more talks and articles did I get that crucial knowledge.
+Vicky Chijwani oh, then I understand a bit better. If we redefine the question we're talking about as "what tests should I start learning" or "how do I get my team started", then that is a much more interesting and valuable question to me. I agree the learning curve for getting started with automated testing is very problematic, and that also goes for organizations learning it as well (selling to inexperienced software managers etc). I don't have a good answer for that, but I think it merits exploration.
+funfunfunction true. I admit I was subconsciously redefining the question, as you've said. I've updated my original comment to reflect this.
Testing Javascript is hard at the beginning. You need to understand a lot of concepts. Testing Frameworks, asserting libraries, BDD vs TDD, headless browsers, test runners. Also there are a LOT of those in the wild. Its very confusing to know if use Mocha or Jasmine, or maybe Karma? Also if you need to test the DOM, this is another complexity layer, would you use Phantom JS, or maybe JSDOM? @MPJ could you please make some videos regarding these topics? Your videos are AWESOME!! You encourage us to do testing in our applications, and we REALLY want to, but setting up the right environment to make them work is not easy.
Good video as always but I don't quite get the "doesn't test contract" part, in unit test you have to test the public interface, that's what unit test is for, and even if you're doing integration test (or end to end test) it won't point out where you got wrong, it's just test the public contract of the entire app (or the usecase).
Thank you soooooooooo much
F# has contract checkers for the database and others through Type Providers (apparently Idris has recently added them too). This makes for one less test that needs to be done! The downside, as your app grows and connects to more and more services your app takes longer to open/compile than before. So, you have to put that part of your data into other projects, AKA modularize your app. Or, even just create a smaller app to begin with.
so cool. :D i am learning a lot from your channel. keep it up sir mpj.
Containers to mitigate >some< integration test expense. Docker might have that thing you don't want to spend time on. Can also spawn a cloud of bee's to stress you out.
I tend to write my unit tests defensively, e.g. I tend to write them in a way that expects someone to change the code I'm testing in a way that messes things up.
Also, what the deal with QAs obsession with coverage?
I've seen getters and setters removed because they "weren't being used or tested, and so were removed to increase coverage" -but their placement in the object were there to signal what fields could be changed without f-ing up internal state. *sigh*
Excellent video, thank you so much!
Like the explanation, and the german accent :)
Thanks man, brilliant explanation.
Please upgrade your microphone!! Great videos man
Our software has 800 lines of tests. They are all UI tests and have certain starting states, so it's very difficult to automate them >
This was great! Thanks, MPJ! Now a question. Because integration tests are so expensive, how should I go about prioritizing which integration tests to perform against a given application? Should I just be testing core integrations - like payments, to touch your example - and work "outward"? I'm assuming units become the bulk of the suite, but by how much?
I have a similar question. Because integration tests are so brittle, it seems like you have to draw some limits. Is there a certain range of number of integration tests you've found that are good? 300-400? I know there is no hard and fast answer, but would appreciate hearing what your thinking would be about making these kinds of decisions.
Love your videos!
thank you MPJ!
Amazing vid MPJ, helped me a lot! I liked the fast forwarded summary at the end.
Would you be able to clarify where E2E (end to end) testing would fall in this whole story?
I've read quite a couple of articles talking about the pyramid-like hierarchy that tests should (loosely) follow, which, from top to bottom is: E2E, integration and unit tests.
What would an E2E-test entail in a JS context? Is it just a nomenclature thing and does your definition of integration tests already cover this?
Hi, I have just started with javascript. Please help me understand testing process. I am new to programming.
I love listening to your Videos! :D
very helpful👍
Do not test contract? If you have done it deeply then how you ensure some foreign point was called or something logged without accessing the database without forgetting to test whether a webservice or a log to the db was called? You might let it run inside but then you threaten yourself to stumble upon many additional mocks needed. Test testing a unit from 2010 failing when adding a change to 2020 code. It sounds easy like dont do this cause its difficult, but you dont support it by facts.You can rely on a specific curl handler to be called, but it might be conditioned by ten settings which you must mock for a simple call then, one method breaks tests of 10 different classes
In JavaScript, how should I do integration test for my Controllers' code and Database/Payment component without having a View?
What about functional tests? I work with the three approach, for the win.
What equipment do you use? The quality on this video is mindblowing!
Check out the behind the scenes video. tl;dr is that it hasn't too much to do with the camera, it's lighting.
Love your videos MPJ. One thing I've noticed, I believe it was since the last episode - the audio quality has gotten worse. It sounds a lot more muffled. Maybe you could fix this?
Sorry about that, I have a new mic, which is actually a lot better but it's also very different. It will get better. :)
I also noticed that the mic is very bad. Sorry I love your videos but the mic is not same quality as the content is :)
The mic he got is, I believe, a Lavelier mic. It's actually a really really good microphone. It just takes some time to get used to - every microphone is different.
I'm sure MPJ will figure out the best way to utilize this new mic. Give him time.
The mic is fine, I'm just using it badly. :)
Amazing content.
Love the retro Casio :)
Hi MPJ
Could you please make a video about reactive programming (RxJS specifically)? Every video I see is always like "ah yeah, u can use events like arrays and do filter and map and all that great functional stuff". Can you come up with some real-world examples to clarify what RxJS is good for?
Many thanks :)
Thanks mate! I had seen that video already and some few more from Ben Lesh. Probably I have to clarify my question a bit further. It's not that I didn't understand the technical part of RxJS, but I don't see that much value add in using such a library.
I was wondering if there are some real-world scenario examples (besides of the cancellable requests), which make me feel like: "yeah, I would absolutely need RxJS for that".
But thanks anyway for your help, I appreciate it very much :)
Kind Regards, Steve :)
I think this is a great question! (Something I've been wondering about myself). Did you ever find a good answer?
Yeah It's Monday !!
What about a test that clicks on a button, but mocks payment system to simulate error?
Will it be an integration or unit test? or a hybrid? how would you call it?
+Juan Pastás it's just words, really. It's like developer or programmer or software engineer. I use integration tests as a catch-all term for non-unit tests. At Spotify we called the tests you refer to as integration tests.
Real application where unit testing is working with integgration tests?
Dear guru MPJ, a completely unrelated question: is it true that in old Nordic slang "to take an arrow in the knee" means "to get married"?
+Niccolò M. No, I've never hear that, ever, and I like those old sayings. I think that one is an urban legend constructed in modern days.
Pardon my ignorance, but what's this accent called? I'd never before heard "just" pronounced "yust". (I'm from Central Europe.)
@funfunfunction MPJ you coming to web summit next month here in Lisbon? :)
Hey MPJ, I can't tell you how long I have been waited for an episode like this. Thanx a lot to cover this topic. My Question to you is:
I do not really have the possability to setup fake databases to run integration tests with fixed expectations. Since I use angular, I can easily unit test directives and check something like this:
expect(myDirective.find('h1').text()).toEqual('1.75€)
But without being able to set the value to 1,75€ in the database, I can only check something like this in an integration test:
expect(myDirective.find('h1').text()).toMatch(/\d.\d\d€/)
Am I right with this assumption or how should I integration test it without a mock database?
CaRDiaK You are so right, but please tell that my organization. As front-end developer I am several layers far away from those guys who are programming the access layer that not even have the permission to check if they program against an abstraction or not. of course I can mock the layer next to me, but then this is no full Integration test anymore.
Do you consider integration tests and end to end tests to be the same thing? Because it seams like you are more talking about end to end tests in this video.
As I say, the definitions are fuzzy. :) I would say that end to end tests are a subset of integration tests.
We use the term functional test when a browser is involved. Our integration tests run the business logic without wrestling with the UI. In our case server side.
But in any case, this was a spot on summary of my experience. Although I am a bit sad that there is no easy way to create fully covered tests without building the boilerplate, this was really a great video. It will help our devs understand the motivation to have both layers. So thank you very much!!
Great video, but poor sound quality. keep itu up!
Is Robot Framework tests considered as Integration tests then? :)
@5:59 patabase?
WHATS A PATABASE
I really love your videos but i want more! Do you know any channels similar to yours?
+Andrew Smith Not really, sorry, that's why I started it. Wesbos is pretty close, and maybe devtips (more design heavy)
funfunfunction I'll check them out thanks :)
Keep up the great work!
when to write one or the other?
+Juan Pastás hmm, I thought this was what the video was about ;)
funfunfunction hehehe I don't know what I was thinking about
I couldn't disagree more about integration tests having all these issues. I agree integrations are substantially slower and harder to setup,. I also think unit tests are much more important than integration tests. HOWEVER, that doesn't mean that integration tests commonly suffer from something like inability to simulate errors. Integration tests are from the highest level you can test. You can absolutely mock up systems to generate errors. Yes mock up. You can mock up fake processes, services, databases at a high level. It doesn't need to be an in code contract for you to mock up a fake external script, processes, web service, etc. The only thing about integration tests that is substantially more costly is the initial setup of code and a dependencies. After that it might be easier to write than unit tests. Integration tests can also replace QA in some fields. Writing firmware for an MRI machine? That needs QA. Somebody can die from a bug and therefore the failure acceptance rate asymptotes at none. Writing a flight booking system website? The tests here can be fully automated and don't need QA. What's the worst that can happen? Somebody books a flight and it doesn't work? OK well fix that unlikely bug and then take $5000 from the $300,000 that you saved on a QA team and pay the customer for the inconvenience.
Unit Tests are FOR the developer
Integration Tests are FROM the customer
Do both, but if you're stressed for time do unit tests and leave the rest to QA.
Where'd you get the pictures in the background from?
+Dylan Field Simon Stålenhag and @pappasparlor
Great video as always! I don't feel you made a great case for integration tests though. If you have to write more complicated tests, in greater quantities, and they don't tell you where errors are (which is kind of the point of tests) how come we need them just as much as unit tests?
Excuse the noob question, have a great week.
Part of the argument he made is that Integration tests do a better job of testing the contract between the db, controller, and view layer. When you write a unit test, a lot of assumptions are made about what input the system will receive. The controller could have been refactored and as a unit works just fine still but may have broken the view layer. So you need something that takes a look at the application as a whole.
The whole "not testing the contract" thing is a very big deal. Using the example in the video, if in reality your view doesn't actually make the same calls to the controller that the controller's unit tests are mocking, then your controller's unit tests have zero value, but they will keep on passing and will never inform you that there's a problem.
I'm not sure he said we need both to equal degree, just that we need them both.
Once your application works fine on the system you typically use, you deploy integration tests to see if it also works on other hardware, OS, browser, etc.
When doing unit tests, you dig into the dirt to find specific bugs. Hope this helps.
Tim Hooker Fair point, but it doesn't sound sufficient. Why not test only said contracts and avoid all the input and state permutations?
gregg4 It does! thanks for replying. I got a bit confused by the heart and brain simile. Hadn't thought about the runtime environment. Still, integration tests sound like they require lots of planning an maintenance and I'm extremely lazy.
What this late?
"If you don't have any form of automated test, you code must suck..."
You didn't need to attack me like that dude :(... Cool video tho
I don't think it's reasonable to run integration tests on the fly.
I prefer having them on the build server and let them run when you submit a PR.
Sure - it's annoying if you submit a PR that breaks a test, but it's quite rare, so I prefer not having to run them while developing.
you said "agression-test" :)