Yeah ads is fine, I trust your integrity enough that ads would probably be a sort of "recommendation" or like "hey take a look at this cool stuff" It helps pay the bills for my favourite creator as well so, do what you do mustache man. Love your content
I'm loving this new content format. Succinct and informative. Testing is a blind spot for me and I'm appreciative of you sharing your experience with them. Definitely throw in their some ad reads in these videos. I feel you respect your audience enough to make sure that the tech you're promoting is authentic and legit.
I do both types of testing. Unit testing to me is all about documenting my assumptions of the other systems my code needs to talk to. That way, when the integration tests fails, I can match the failure to my assumptions and know how I need to change my code. Also, unit tests are usually the only way I can tests some error states that could come from the other system if the other system doesn't give me a way to simulate that error state.
Mocks are for unit tests to test specific scenarios on small chunk of code. For integration tests you should use testcontainers not mocks. Usually you would use both for different things. Like testing storage implementation with testcontainer and testing service with mock of your storage.
I appreciate you making these videos and not only streaming as I don't have time to watch streams. Also the feel is different than in clips. So thanks! Also, I'm cool with the ads, I think you're a man of integrity :)
Love this format, the past few videos on this channel have been super insightful and gotten me to think pretty deeply about the topics you've covered. Also super OK with ads.
Mhhh. Unit testing? Mock. Integration tests, do not mock. A pull request or commit should have unit test. I think that’s where mocking is the most useful for me. At the end of the day, you need to throw your product out in the wild in connect it with every service, do it immediately or later. It’s all up to you.
So you end up with thousands of so-called unit tests that test themselves, and now your program is impossible to refactor without also rewriting thousands of lines of tests just to conform to the refactored code structure. Unit tests are dead code. Write integration tests instead.
@@MrCatgroove Great artists learn the rules, then break them. This is the curve meme. Dumb people don't write unit tests, mediocre people write unit tests, smart people don't write unit tests.
@@jonathan2847 For sure. This is true for everything in life. Novices and Pros do a lot of the same things, but for wildly different reasons lol. It's like learning to paint like a child, like Picasso said.
Hey prime, thanks for the video! I am 100% with you here 😄 Also, for the ads, please do! You are NOT the person of selling something you don't believe in, so I would definitely be interested in find out more cool stuff (that is what ads are for, right?)
For testing app paths with database connections I've found Docker to be a great tool to simulate databases. Set up a postgresql docker image, create some scripts to create tables and insert mock data, then swap to the database image when testing with a config option. It means you're writing a lot less code and spending a lot less time setting up mocks for your test files
The usage of mocks in tests, the degree of acceptance and the degree of strength behind the reasoninh to use them, depends on the type of test, imo. For unit tests, mocks can (and probably should) be used and you don't need a very strong reason to do so. Their usage should be easily acceptable, as unit tests test if the code behaves as you expect it to in regards to the assumptions you make about the world. For integration and e2e tests, however, the degree of acceptance for the usage of mocks should be much lower, and only be used in very specific situations (i.e. have a strong reason to do so, larger for e2e tests).
I agree! Mocks are good and preferred for unit testing. As you mention, it seems that two types of testing are being conflated or forced into a single test. Unit tests are meant to test code, while System testing is meant for end-to-end testing. Mocks simulate how code handles messages/responses outside of its scope. In this case, connecting to a db as part of the test adds complexity without improving the test. Imagine how complicated unit testing would be for distributed systems such as Pulsar, Kafka, or Cassandra if every system had to be online for unit tests. Every developer would have to be able to compile and run the entire application for every unit test.
I built a testing tool for our ios app that ran a local http server and redirected all of our requests to that instead of the actual server. Then the http server could be told to return a specific JSON response for each test.
In a project I've been working on, I made a dummy database handler, that instead of using a database it uses slices and maps for the data. And it has been working great, because in the constructed world I have perfect control over what everything does, I of course use it in combination with end to end tests, but instead of spinning up a full database when I just need to give inputs and outputs to the application, I know what the input should be and I know what I want in return. With that I can easily check that my application work as intended, since I am programing against a defined interface, and once that breaks I know something is wrong. This works because when I made the dummy I did make assumptions on how the world works, and made something that worked with my assumption. I then of course learned that I was very wrong, but that also meant that what I was doing in my application was wrong. So I had to fix my assumption and now both the dummy and the real database works as intended. It will of course break, but that also means that I have been walking away from the original goal and I need to find out why, and either fix it by going back or redefine my goal. But yeah nothing can replace end to end tests....
Its not hard to do similar mocks in other DBs, at work we do the same with postgres, we have a seed file that resets on each test suit, coincidentally the same seeds we use for local development. It works fantastic, I'm amazing to see actual SQL errors show up on the tests, that didn't happen to me before in the "mock everything" mindset
I program in a very different world but our maxim is if we can do it for real we do. Unfortunately this can only go so far and mocks / emulators become essential when you dont have spare hardware to facilitate dev testing.
Mock makes total sense for unit tests. It is not testing the behaviour of how a function integrates with something outside, but rather the code written in the function. Removing mocks and setting up the connected services/infra is integration test. Unit test is easy to run and good for quick iteration while developing
I started really disliking that way back when I was doing silly things like building a math library without minus, divide, or multiply. Because every time I tested my stuff, I'd test it with like 1, 2, and 3, and it would return the expected answers. Then it would just explode somewhere or everywhere along the logic chain when I went to use it for anything useful. Obviously that's skill issues as much as anything else, I shouldn't have only tested the most trivial cases, but it's why I personally don't do mocks.
The code that does IO and deals with IO failures is not worth unit testing; the returns are way too low for comparatively high investment. Extract the logic, and maybe do unit tests for the logic.
Sqlite doesn't solve the mock problem, it just moves it. What if your schema changes slightly over time, or the format of fields stored, and you don't update your db test snapshot, then you are in the exact same situation where your assumptions of the world don't match with reality, your tests pass but don't work for real. The key imo is write good narrow interfaces, and have good narrow types and write your tests and mocks test as widely as possible within those bounds. That way you can know "my function behaves in an understandable/expected way in a variety of possible cases". Then as things change, as long as you stay within your interface / types then you know your tests are still good. If you have to change the interface or type, you know you need to revisit the test. There are two things at odds with each other. You want your code to be narrow, and limit possibilities for it to be used unexpectedly, but you want your test to be wide and test those limits. This of course is why you should only write generic code when absolutely necessary and the way you limit generic code is by only doing very small focused things in a generic way (like, idk, a custom fold on a generic iterable )
10 seconds ad is completely fine. Due to the nature of your viewers being very tech-savvy, I wonder how many of them have SponsorBlock installed and would just skip the promotion anyways.
If it means money for you, and you stand behind the product, I’m all for it. If you could maybe avoid the whiplash sensation that often comes with ad reads though, that would be nice. If anybody’s going to endorse good products and make the ads enjoyable I would expect it to be you.
I mean this is unit-ish test vs integration test. I do agree with you though, I always prefer the latter approach, I don't see much value from the first iteration. I use mocks only for third party api's and such that I cannot do anything about
Well, if reality changes, and this might come as a shock: you need to change your mocks. You do it at the same time as you change your code that this mock is for. If you don't need to change your code, then you also don't have you to change your mocks. It's not like you write these mocks once and then never think about them again. They are part of your living code documentation. Btw on the topic of ads: I think they'd be fine here.
sqlite is just a file that can be several kb big if you are storing quite a bit. my auto-scaling game server only needs shapes of servers to be stored and i can hydrate that back pretty easily
Yea that makes sense. We have a larger db and how to properly test without mocking datasets has been a mental exercise without burying the team when they just want to make a small change and run tests.
Short, quick ads that focus on pointing out that a cool thing exists are completely fine. The more the ad feels like a sales pitch, the more annoying it gets. That being said, the truth is that I wouldn't quit watching if it was over those boundaries.
Timeline Summary: 0:00-0:35: The creator initially felt that mocks were a good way to test complex systems but later realized they can cause issues. 0:35-1:57: The creator explains how mocks are useful for testing specific functions but can give a false sense of security. 1:57-4:13: The creator argues that using real data and real systems leads to more accurate results in testing. 4:13-5:35: The creator compares mocks to documentation, noting both can become outdated or inaccurate over time. 5:35-6:15: The creator emphasizes the importance of testing assumptions and warns against relying solely on mocks. 6:15-7:57: The creator shares their approach to testing: using real data and systems, with a mocking library for specific functions. 7:57-9:06: The creator concludes by asking for viewer feedback on seeing short ads for a database service the creator uses. Why does the speaker prefer real-world testing? The speaker prefers real-world testing because it provides more accurate and reliable results compared to mocks, which may lead to false confidence and could become outdated or misleading, like documentation.
I really love your content and this new format. However I would say (as a mockist) that I would not use mocks here but the real thing. I like mocks for very simple components where call order and inner behaviors does not matter. Your test is too close to an integration test. It knows too much about the intricacies of the game server. Your mock does not constraint call order, which can hide bugs. Hard to say just with what we see but I change the design of GameServer so that it only expose one function that returns a server, whether it's a new one or not. I think it's not MatchMakingServer responsibility to negociate for a server. IMHO by doing this change, your whole code will become much simpler to test (and no more complicated mock with multiple functions to call for this test).
And also, mocks must be exactly as the world is. If other function calls are made, the test MUST fail. expect is usually not a cause for failure, but asserts are
When to mock: when you rely on external third-party services and need to test behavior around it. Mock their interface according to their published API and code to match that API.
I find this one hard. You could also write your functional code inside the imperative shell. Once you get the data from their API, write functional(-ish) code that operates over that data. Then, when you test, all you need is static test data. If you exercise the code that's calling their API, you run into the view-of-the-world divergence that Primeagen says here - you're ostensibly testing that your code reaches out to the service correctly and gets the data back correctly, but since it's all mocked out you are getting nothing in terms of test coverage and it's trivial to have an error in your mock mean that your test passes but production code is broken.
Isn't a snapshot of the database ALSO an assumption of the world? How is it any different from writing mocks? Over time, the snapshot you are using in your test is going to diverge more and more from the actual database. Unless you mean to take a snapshot each time you run the test? Then how can you verify the results, if you don't know what the test data actually is?
this is very good point, but i would argue that the shape of the data is a verifiable item that will cause JSON parse / Table parse issues. thus will cause at the moment of using the issue whereas mocks can / will miss types of errors (what do you do if you get a 504 from a service? lots of people don't plan for this but it does happen rarely) so a db changes (hopefully) less often than the order in which call your service functions and their arguments or their potential return values
mocking is what happens when you try to test a systems sw like it is small module. unit testing works great when they test something small, concrete, like a policy engine, json parser, etc systems testing needs to be e2e testing. otherwise you are just testing mocks. when dealing with complex systems, most bugs will be in the seems - where one component does something that the other does not anticipate. mocks will not help you with that. systems e2e testing must be something that is thought of from day 1. if you needs your system easily e2e tested from day 1. this is not something you can (easily) bolt on later
I understanding testing the "happy case" using integration tests. But testing the error cases is way, WAY easier with mocks. Figuring out exactly what inputs or environment states cause which specific errors, especially if it's some ephemeral error, is impossible. Mocks test the behavior that is independent of "reality," and most well-designed functions have a lot of this kind of independence. Like I don't care how the server determined port 42069 was the best port, but given that it did that, how does this unit handle that? I can test that. It can't test everything, integration tests are still vital, but mocks are great at ensuring you're able to hit all of your branches and have them reasonably smoke-screened.
this is where simulation testing / tiger style stuff comes into big time play (sim testing be equal to fuzz testing) here is a great example: i have a bug that creeps up at 1 million connections added to my game servers. i would definitely have missed that during regular testing, caught on sim testing
I hate mocks, they are no better than a wet blanket. Over the last few months, I have had to deal with so many bugs that wouldn't have happened if the tests had run against a real system instead of mocks.
But brah end to end testing takes ages. If you would write end to end test for everything that a develop i wouldnt do anything else. Unit test gives me abílity to quickly develop and test my solution and cover my edge cases and you cant do unit test without mocking
I would be cautious about ad reads and sponsorships. Make sure you actually really use the company through and through. For example, when you promote kinesis, its clear you fully use their products, so why would anyone care? Do you use Turso this much? Are you an actual supporter of Turso? Obviously there are degrees of support. But people will have different threat levels to sponsorship. Look at Theo and Vercel.
Yeah ads is fine, I trust your integrity enough that ads would probably be a sort of "recommendation" or like "hey take a look at this cool stuff" It helps pay the bills for my favourite creator as well so, do what you do mustache man. Love your content
second this
Third this
Forth deez, oh no this not twitch 😮
fifth this
sixth this
Mocks - poking holes in reality for short term convenience and hoping things will be close enough.
I'm loving this new content format. Succinct and informative. Testing is a blind spot for me and I'm appreciative of you sharing your experience with them.
Definitely throw in their some ad reads in these videos. I feel you respect your audience enough to make sure that the tech you're promoting is authentic and legit.
I do both types of testing. Unit testing to me is all about documenting my assumptions of the other systems my code needs to talk to. That way, when the integration tests fails, I can match the failure to my assumptions and know how I need to change my code. Also, unit tests are usually the only way I can tests some error states that could come from the other system if the other system doesn't give me a way to simulate that error state.
YEAH, exactly
If it's 10-20 seconds, I don't care. I'll happily watch it to support the channel
Mocks are for unit tests to test specific scenarios on small chunk of code. For integration tests you should use testcontainers not mocks. Usually you would use both for different things. Like testing storage implementation with testcontainer and testing service with mock of your storage.
I appreciate you making these videos and not only streaming as I don't have time to watch streams. Also the feel is different than in clips.
So thanks!
Also, I'm cool with the ads, I think you're a man of integrity :)
should probably do an ad read for a really cool podcast, called Top Shelf
Love this format, the past few videos on this channel have been super insightful and gotten me to think pretty deeply about the topics you've covered. Also super OK with ads.
10 second ad reads sound fine. I have some faith you'll do it right.
Please, continue to do this kind of videos, i love to learn and open my mind with you everyday. Good stuff.
Mhhh. Unit testing? Mock.
Integration tests, do not mock.
A pull request or commit should have unit test.
I think that’s where mocking is the most useful for me. At the end of the day, you need to throw your product out in the wild in connect it with every service, do it immediately or later. It’s all up to you.
Unit test libraries.
Integration test binaries.
So you end up with thousands of so-called unit tests that test themselves, and now your program is impossible to refactor without also rewriting thousands of lines of tests just to conform to the refactored code structure.
Unit tests are dead code. Write integration tests instead.
this is a regarded rule lol
@@MrCatgroove Great artists learn the rules, then break them. This is the curve meme. Dumb people don't write unit tests, mediocre people write unit tests, smart people don't write unit tests.
@@jonathan2847 For sure. This is true for everything in life. Novices and Pros do a lot of the same things, but for wildly different reasons lol. It's like learning to paint like a child, like Picasso said.
Hey prime, thanks for the video! I am 100% with you here 😄 Also, for the ads, please do! You are NOT the person of selling something you don't believe in, so I would definitely be interested in find out more cool stuff (that is what ads are for, right?)
I like mocking but if its to excessive its usually a indication of bad software design.
For testing app paths with database connections I've found Docker to be a great tool to simulate databases. Set up a postgresql docker image, create some scripts to create tables and insert mock data, then swap to the database image when testing with a config option. It means you're writing a lot less code and spending a lot less time setting up mocks for your test files
absolutely, feel free to add some ads. You are great, man! Thanks for the video
Go, PHP, Mocks… What’s next? Ruby?
Believe in DHH
The usage of mocks in tests, the degree of acceptance and the degree of strength behind the reasoninh to use them, depends on the type of test, imo.
For unit tests, mocks can (and probably should) be used and you don't need a very strong reason to do so. Their usage should be easily acceptable, as unit tests test if the code behaves as you expect it to in regards to the assumptions you make about the world.
For integration and e2e tests, however, the degree of acceptance for the usage of mocks should be much lower, and only be used in very specific situations (i.e. have a strong reason to do so, larger for e2e tests).
I agree! Mocks are good and preferred for unit testing.
As you mention, it seems that two types of testing are being conflated or forced into a single test. Unit tests are meant to test code, while System testing is meant for end-to-end testing.
Mocks simulate how code handles messages/responses outside of its scope. In this case, connecting to a db as part of the test adds complexity without improving the test.
Imagine how complicated unit testing would be for distributed systems such as Pulsar, Kafka, or Cassandra if every system had to be online for unit tests. Every developer would have to be able to compile and run the entire application for every unit test.
Totally fine with the quick ad reads. do what you gotta do
cool new format! 10 mins, focused, calm.
I built a testing tool for our ios app that ran a local http server and redirected all of our requests to that instead of the actual server. Then the http server could be told to return a specific JSON response for each test.
Prime's Vim theme is all different shades of purple.
i have a specific theme for zig vs other languages for streaming purposes
@@TheVimeagen zig deez nutz
In a project I've been working on, I made a dummy database handler, that instead of using a database it uses slices and maps for the data. And it has been working great, because in the constructed world I have perfect control over what everything does, I of course use it in combination with end to end tests, but instead of spinning up a full database when I just need to give inputs and outputs to the application, I know what the input should be and I know what I want in return.
With that I can easily check that my application work as intended, since I am programing against a defined interface, and once that breaks I know something is wrong. This works because when I made the dummy I did make assumptions on how the world works, and made something that worked with my assumption. I then of course learned that I was very wrong, but that also meant that what I was doing in my application was wrong. So I had to fix my assumption and now both the dummy and the real database works as intended. It will of course break, but that also means that I have been walking away from the original goal and I need to find out why, and either fix it by going back or redefine my goal.
But yeah nothing can replace end to end tests....
I don't see how that would work with a properly used relational database, unless your queries are no more complex than 'select * from table;'.
Its not hard to do similar mocks in other DBs, at work we do the same with postgres, we have a seed file that resets on each test suit, coincidentally the same seeds we use for local development. It works fantastic, I'm amazing to see actual SQL errors show up on the tests, that didn't happen to me before in the "mock everything" mindset
I use mmock btw. It's so nicely tied to actual code and it enforces to regenerate when you change the interface so it stays fresh. Highly recommend.
Great video, already use and love Turso, so I don't mind ads personnaly.
No problem with the ads!
I trust you not to sell out
I program in a very different world but our maxim is if we can do it for real we do. Unfortunately this can only go so far and mocks / emulators become essential when you dont have spare hardware to facilitate dev testing.
Get that bag, wouldn't mind the ads :D
Ads are fine, your content is awesome and I trust you won't make it annoying to watch
Loved the video! I don't mind about the small ads
love this and i dont mind an ad but wouldnt prefer it
Ads are fine with moderation. Get that paper baby.
Go for it, be good to hear what things are catching your attention, and you should earn a dime or two
ads but with primeagen excitement energy
Ads for stuff you use are great
Best non-ad ad
Mock makes total sense for unit tests. It is not testing the behaviour of how a function integrates with something outside, but rather the code written in the function.
Removing mocks and setting up the connected services/infra is integration test.
Unit test is easy to run and good for quick iteration while developing
I started really disliking that way back when I was doing silly things like building a math library without minus, divide, or multiply. Because every time I tested my stuff, I'd test it with like 1, 2, and 3, and it would return the expected answers. Then it would just explode somewhere or everywhere along the logic chain when I went to use it for anything useful. Obviously that's skill issues as much as anything else, I shouldn't have only tested the most trivial cases, but it's why I personally don't do mocks.
The code that does IO and deals with IO failures is not worth unit testing; the returns are way too low for comparatively high investment. Extract the logic, and maybe do unit tests for the logic.
Sqlite doesn't solve the mock problem, it just moves it. What if your schema changes slightly over time, or the format of fields stored, and you don't update your db test snapshot, then you are in the exact same situation where your assumptions of the world don't match with reality, your tests pass but don't work for real.
The key imo is write good narrow interfaces, and have good narrow types and write your tests and mocks test as widely as possible within those bounds. That way you can know "my function behaves in an understandable/expected way in a variety of possible cases". Then as things change, as long as you stay within your interface / types then you know your tests are still good. If you have to change the interface or type, you know you need to revisit the test. There are two things at odds with each other. You want your code to be narrow, and limit possibilities for it to be used unexpectedly, but you want your test to be wide and test those limits. This of course is why you should only write generic code when absolutely necessary and the way you limit generic code is by only doing very small focused things in a generic way (like, idk, a custom fold on a generic iterable )
Can we get a video on your custom logs ? I love this kind of content, thanks prime
Waiting for prime to find out about testcontainers
The ads thing sounds fine.
10 seconds ad is completely fine. Due to the nature of your viewers being very tech-savvy, I wonder how many of them have SponsorBlock installed and would just skip the promotion anyways.
What is even harder than setting up 50 servers, is causing 50 servers to error out in specific way so you can assert the error is handled correctly.
Mocks in go are best implemented with interfaces
Ad me up baby! Turso is great and I love to know about their new features.
Sure, for me ads are mostly annoying if it's for things I'm not interested in so I wouldn't mind turso ad reads at all.
If it means money for you, and you stand behind the product, I’m all for it. If you could maybe avoid the whiplash sensation that often comes with ad reads though, that would be nice.
If anybody’s going to endorse good products and make the ads enjoyable I would expect it to be you.
No ads for UA-cam Premium members, give em the ads!👿
do the ad of everything you can think can be REALLY useful.
Go ahead with the turso ads, get the bag
LGTM
Turso is good, go for it. Would like a turorial on their multi tenent db pattern in go maybe ;)
i mock your mocks
I mean this is unit-ish test vs integration test. I do agree with you though, I always prefer the latter approach, I don't see much value from the first iteration. I use mocks only for third party api's and such that I cannot do anything about
Mocks are good when you are in early development. SQUEEL LIGHT for the win.
Go ahead. I think good products must have to be shared with everyone
Well, if reality changes, and this might come as a shock: you need to change your mocks. You do it at the same time as you change your code that this mock is for.
If you don't need to change your code, then you also don't have you to change your mocks. It's not like you write these mocks once and then never think about them again. They are part of your living code documentation.
Btw on the topic of ads: I think they'd be fine here.
no problem, if it helps you keep going all for it. But will they match Theo's advertisements?
You checking in your state of the world database into Git? If your service is large enough to require Postgres?
sqlite is just a file that can be several kb big if you are storing quite a bit.
my auto-scaling game server only needs shapes of servers to be stored and i can hydrate that back pretty easily
Yea that makes sense. We have a larger db and how to properly test without mocking datasets has been a mental exercise without burying the team when they just want to make a small change and run tests.
Turso ads would be ads i would watch
The adds are fine!
Short, quick ads that focus on pointing out that a cool thing exists are completely fine.
The more the ad feels like a sales pitch, the more annoying it gets.
That being said, the truth is that I wouldn't quit watching if it was over those boundaries.
doesn't the database file is my mock, get a bit tedious, when the database schema changes?
Ads are fine I trust you that it would not be something useless.
Keep it up! Looks good!
yes adds, i really like your videos and wish i could support you, so adds are great
does anyone know what theme he is using?
Timeline Summary:
0:00-0:35: The creator initially felt that mocks were a good way to test complex systems but later realized they can cause issues.
0:35-1:57: The creator explains how mocks are useful for testing specific functions but can give a false sense of security.
1:57-4:13: The creator argues that using real data and real systems leads to more accurate results in testing.
4:13-5:35: The creator compares mocks to documentation, noting both can become outdated or inaccurate over time.
5:35-6:15: The creator emphasizes the importance of testing assumptions and warns against relying solely on mocks.
6:15-7:57: The creator shares their approach to testing: using real data and systems, with a mocking library for specific functions.
7:57-9:06: The creator concludes by asking for viewer feedback on seeing short ads for a database service the creator uses.
Why does the speaker prefer real-world testing? The speaker prefers real-world testing because it provides more accurate and reliable results compared to mocks, which may lead to false confidence and could become outdated or misleading, like documentation.
Bro summary by some AI tool? Let us know which one?
I really love your content and this new format. However I would say (as a mockist) that I would not use mocks here but the real thing. I like mocks for very simple components where call order and inner behaviors does not matter.
Your test is too close to an integration test. It knows too much about the intricacies of the game server. Your mock does not constraint call order, which can hide bugs. Hard to say just with what we see but I change the design of GameServer so that it only expose one function that returns a server, whether it's a new one or not. I think it's not MatchMakingServer responsibility to negociate for a server. IMHO by doing this change, your whole code will become much simpler to test (and no more complicated mock with multiple functions to call for this test).
I use mocks for testing non-happy path.
And also, mocks must be exactly as the world is. If other function calls are made, the test MUST fail. expect is usually not a cause for failure, but asserts are
W ads I trust you
I don't know what turso is but get payed if you can
As long as you make the ads Tim Dillon style, go ahead 😅
Have you considered contract testing?
Mocks for most unit tests means the code is too coupled. Fake data should be enough for the vast majority of the code
When to mock: when you rely on external third-party services and need to test behavior around it. Mock their interface according to their published API and code to match that API.
I find this one hard. You could also write your functional code inside the imperative shell. Once you get the data from their API, write functional(-ish) code that operates over that data. Then, when you test, all you need is static test data.
If you exercise the code that's calling their API, you run into the view-of-the-world divergence that Primeagen says here - you're ostensibly testing that your code reaches out to the service correctly and gets the data back correctly, but since it's all mocked out you are getting nothing in terms of test coverage and it's trivial to have an error in your mock mean that your test passes but production code is broken.
If you're mocking a lot, you're not testing anything useful.
@@natescode Look up "unit testing".
Ads is fine, sir
Isn't a snapshot of the database ALSO an assumption of the world? How is it any different from writing mocks? Over time, the snapshot you are using in your test is going to diverge more and more from the actual database.
Unless you mean to take a snapshot each time you run the test? Then how can you verify the results, if you don't know what the test data actually is?
this is very good point, but i would argue that the shape of the data is a verifiable item that will cause JSON parse / Table parse issues. thus will cause at the moment of using the issue whereas mocks can / will miss types of errors (what do you do if you get a 504 from a service? lots of people don't plan for this but it does happen rarely)
so a db changes (hopefully) less often than the order in which call your service functions and their arguments or their potential return values
ever heard of test pyramid?
don't write mocks, write fakes.
oh wow, welcome to game dev, I guess :D
mocking is what happens when you try to test a systems sw like it is small module.
unit testing works great when they test something small, concrete, like a policy engine, json parser, etc
systems testing needs to be e2e testing. otherwise you are just testing mocks. when dealing with complex systems, most bugs will be in the seems - where one component does something that the other does not anticipate. mocks will not help you with that.
systems e2e testing must be something that is thought of from day 1. if you needs your system easily e2e tested from day 1. this is not something you can (easily) bolt on later
4:37 - Same thing John Carmack said about print debugging and actual debuggers: ua-cam.com/video/rhUwOBg4TyM/v-deo.htmlsi=E5YspS3S7hK-bDmm&t=277
ADS ADS ADS ADS
you can't really avoid mocking a database unless you have an environment for that specifically
I understanding testing the "happy case" using integration tests. But testing the error cases is way, WAY easier with mocks. Figuring out exactly what inputs or environment states cause which specific errors, especially if it's some ephemeral error, is impossible. Mocks test the behavior that is independent of "reality," and most well-designed functions have a lot of this kind of independence. Like I don't care how the server determined port 42069 was the best port, but given that it did that, how does this unit handle that? I can test that. It can't test everything, integration tests are still vital, but mocks are great at ensuring you're able to hit all of your branches and have them reasonably smoke-screened.
this is where simulation testing / tiger style stuff comes into big time play (sim testing be equal to fuzz testing)
here is a great example:
i have a bug that creeps up at 1 million connections added to my game servers. i would definitely have missed that during regular testing, caught on sim testing
turso ads are ok ads
I hate mocks, they are no better than a wet blanket.
Over the last few months, I have had to deal with so many bugs that wouldn't have happened if the tests had run against a real system instead of mocks.
But brah end to end testing takes ages. If you would write end to end test for everything that a develop i wouldnt do anything else. Unit test gives me abílity to quickly develop and test my solution and cover my edge cases and you cant do unit test without mocking
With real server requests, your tests may become non deterministic and tests become flaky.
Do ads
I would be cautious about ad reads and sponsorships. Make sure you actually really use the company through and through. For example, when you promote kinesis, its clear you fully use their products, so why would anyone care? Do you use Turso this much? Are you an actual supporter of Turso?
Obviously there are degrees of support. But people will have different threat levels to sponsorship. Look at Theo and Vercel.
1st
just don't sound like you're reading stuff.