Another cracking video well done!!. Love to see your view on unit testing services that use Entity Framework. So much confusion in this area as to best approach. I'm using interfaces and mocking it but it does feel quite fragile, however it is very fast and each test is well isolated from the others.
Oh, don't mention that :) Testing the infrastructure is particularly hard because of mocking an external system, which has its subtle annoyances that the tests often have no way of mimicking. That turns into the source of fragility.
Probably from years of my experience in deploying and servicing accounting, erp and general business application this fact of clearing curency at 0 bothered me in other video too, i thought of other reasons at first but still bothered me.
I think there is another issue with this example. What if there was other code that relies on the currency property to become empty just like the brittle test did. How do we account for that? Wouldn't a brittle test at least give us a hint that there might be a problem we need to address? What do you think?
While this cannot be taken as the answer in general, in this particular case I don't see why would any other piece of code effectively depend on either value stored in the money object. Shouldn't value transforms remain encapsulated? I am trying to emphasize the value of encapsulation both ways - in tests and in production code equally. Once you get hold of an instance, you would use its public members to transform it, not question the values of its properties. By the way, if we insisted on pure OOP, the class wouldn't even have the properties. They are there to help separate responsibilities, such as persistence, serialization, or UI. Property values are normally viewed as a given in any behavior that consumes them.
Outside of this very specific example, I will admit that I struggle to see how this could be applied in a more general sense. This also seems like it's not always that appropriate? Wouldn't there be occasions where I want to be sure that the algorithm implementation is working in a manner that I expect it to? You could also argue that in this example, the issue is partially due to using the implementation to guide the unit tests, but it's also strongly to do with the test looking at a lot of information that isn't relevant to what the test is actually intending to look at. i.e. Assert.Equal(0, diff.Amount) is perfectly valid (we're checking that the amount should in fact be zero). The stated intent of the test is to confirm that fact, not to check whether the currency type has changed or not.
You are onto something here, but I would return your thoughts one step back and look at the problem again. Proving that the algorithm is doing a specific thing can be a poisonous thought. Why not test that the result of a specific transform is as expected, and leave the implementation open to ideas? Constraining the implementation that way turns tests into control freaks. Consider this thought. What if the class were an interface and someone handed you an instance? How would you verify that the effects of a certain action have happened? That is your unit test which is resilient to change in implementation. Back to the question of having a piece of code where it is really important to know that something happened. If that is so, then apply formal analysis and prove it, rather than test it. I made the whole series of videos explaining techniques we apply to that end: ua-cam.com/play/PLSDYwLgFqaX6rSZSEVpaI4mjFXTWZqBrM.html
@@zoran-horvat Thank you for your insightful answer. That definitely helps me to better wrap my head around it. And thank you for the playlist! I'll be sure to watch it.
Great advice Zoran! How do you handle testing of void methods though? Since they usually modify internal state or notify someone else, there's little choice but to inject mocks or check an objects state, but this often requires much more knowledge about internals than I want to have.
In my opinion, every method must correspond to a request for a certain externally visible effect. If you can formalize the request and turn it into a proposition, then the test would assert that that proposition is true after making a call. This may sound overly abstract, but once you get to the bottom of it, you will see that you can establish a process of writing tests that are natively resilient to implementation changes.
@@zoran-horvat So basically, instead of having a specific output as a concrete return type, your method has a concrete output but in the form of a state change or a method call down to another component, and you'd still observe this behavior without caring about internals and how the method gets there?
@@allinvanguard That is the idea. Imagine it is an interface with no concrete implementation. What is the expected observable change that must happen if interface were implemented according to requirements? Keep in mind that this method is not applicable universally. But it can easily cover 90+% of requirements in any business application, i e. application which is not heavily algorithmic in its nature.
Can the currency type of the money really be considered an implementation detail? From a code perspective, any public method or value is part of the interface, so the fact that currency can be accessed by our "brittle" test at all means that this test seems to be testing the interface of Money. if currency type were a private variable, then i'd agree with you that we shouldn't test for it directly. But then we wouldn't be able to test against it at all because we have no way of accessing it from a test. Secondly, if your business logic fundemantally changes (IE: the types of currencies that can be added and subtracted changes), then why wouldn't you want a unit test to fail? If I fundamentally changed business logic and didn't see a test fail, that would be a red flag for me because it means that the old business logic didn't have any automated tests that were reinforcing behavior. I could understand if by "implementation detail" you meant wheter we persist to a sql database vs a nosql database, but currency type is a fundamential part of the domain, and busienss users expect this to have a specific behavior regardless of the underlying implementation.
There is wisdom in what you are saying, but in the sample code I have taken a step further. Though every public member is part of the contract in its own right, you don't have to depend on unnecessary parts in every test. Whether clearing the currency was an express requirement or programmer's way of solving a technical problem, either way it is irrelevant to the test that checks that the result is zero, whenthere is the property that defines zero. I have used the public interface of the class to express a property that must be true in that test with least assumptions. Regarding the note about wanting a test to fail, there you are right. However, that would be the change in requirements for that test, and it would be by design for that test to fail. Again, this case from the demo is not of that kind. The substantial properties of addition and subtraction did not change, and so I would expect all tests that used to be green to remain green.
Brittle tests are written after the fact, write them before the implementation. Legacy code is a challenge to test, often there's nobody around who can tell you what it should be doing.
Tests before implementation don't warrant much if the one who writes them lacks understanding. I would say it goes the other way around - those who understand how to avoid brittleness will tend to write tests before implementation, because they see how that works for them.
@@zoran-horvat You write one test, make it pass, refactor, repeat. Not write all the tests in one go. You learn as you go, covering the use cases, thinking up also unusual use cases, with some help from the domain expert and colleagues. And you still don't get it all covered, the use cases nor test coverage. You know how this goes. Hard to see how that brittle case could have been avoided without understanding. It's not enforceable at the Interface contract.
@@zoran-horvat Very much enjoying these videos. I wish that I could afford the sub but I'm a pensioner, still fascinated after all these decades. Our industry surely needs the education, if you think that you have frown marks then you should see mine! Keep up the good work.
if your test isn't testing an implementation what is it testing? The "unrelated" test failed because it tested an implementation and failed because the implementation was no longer valid. So good test
No. You got it all wrong. The true test verifies externally visible attributes so that you can reimplement the method in a different way later and not break the test, so long as the attributes are preserved. Testing the implementation is precisely what gives you brittle tests - the principal reason why so many teams abandon tests mid-project or start ignoring them.
@@zoran-horvat You seem to be thinking of units like pure functional methods. Unit tests are written based on an assumption and there's usually an assert of that assumption. The assumption/implementation that the test is based on should fail if that assumption/implementation changes, otherwise it's a useless test. If a test was written with some twisted concept of zero money, then it should fail when that twisted concept of zero money changes and a new test should be written. The worst kind of unit test is the one that passes all the time even when the implementation it is based on changes. It means either the test wasn't written correctly or it never worked well in the first place.
@@auronedgevicks7739 You are then making more than one assumption per test, if the assertion depends on whatever the test days it is verifying, and on the internals of zero. That I find to be the cause of brittleness.
I agree with the fundamental thesis of the video, but the example is not well chosen. It is actually very inappropriate to make the different currency zeroes equal. Comparing different currencies should always throw an exception, it should be forbidden just like doing arithmetic on them, zero or not.
If you think it through, you will see that in this system there really exist two definitions of zero. It becomes obvious in an operation which aggregates a sequence of money objects - what is the seed?
@@zoran-horvatSumming different currencies is not good idea in general... sums grouped by currency ok, but scaling to single currency can be from not easy to extreme complex depending is this a accounting or banking use and complexity of accounting laws and tax system
Always happy to see a video about unit testing!
You're in the minority and that makes me welcome your coming here twice as warmly :)
Another cracking video well done!!. Love to see your view on unit testing services that use Entity Framework. So much confusion in this area as to best approach. I'm using interfaces and mocking it but it does feel quite fragile, however it is very fast and each test is well isolated from the others.
Oh, don't mention that :)
Testing the infrastructure is particularly hard because of mocking an external system, which has its subtle annoyances that the tests often have no way of mimicking. That turns into the source of fragility.
Probably from years of my experience in deploying and servicing accounting, erp and general business application this fact of clearing curency at 0 bothered me in other video too, i thought of other reasons at first but still bothered me.
Thank you for posting this video!
I think there is another issue with this example. What if there was other code that relies on the currency property to become empty just like the brittle test did. How do we account for that? Wouldn't a brittle test at least give us a hint that there might be a problem we need to address? What do you think?
While this cannot be taken as the answer in general, in this particular case I don't see why would any other piece of code effectively depend on either value stored in the money object. Shouldn't value transforms remain encapsulated?
I am trying to emphasize the value of encapsulation both ways - in tests and in production code equally. Once you get hold of an instance, you would use its public members to transform it, not question the values of its properties.
By the way, if we insisted on pure OOP, the class wouldn't even have the properties. They are there to help separate responsibilities, such as persistence, serialization, or UI. Property values are normally viewed as a given in any behavior that consumes them.
Outside of this very specific example, I will admit that I struggle to see how this could be applied in a more general sense. This also seems like it's not always that appropriate? Wouldn't there be occasions where I want to be sure that the algorithm implementation is working in a manner that I expect it to?
You could also argue that in this example, the issue is partially due to using the implementation to guide the unit tests, but it's also strongly to do with the test looking at a lot of information that isn't relevant to what the test is actually intending to look at. i.e. Assert.Equal(0, diff.Amount) is perfectly valid (we're checking that the amount should in fact be zero). The stated intent of the test is to confirm that fact, not to check whether the currency type has changed or not.
You are onto something here, but I would return your thoughts one step back and look at the problem again.
Proving that the algorithm is doing a specific thing can be a poisonous thought. Why not test that the result of a specific transform is as expected, and leave the implementation open to ideas? Constraining the implementation that way turns tests into control freaks.
Consider this thought. What if the class were an interface and someone handed you an instance? How would you verify that the effects of a certain action have happened? That is your unit test which is resilient to change in implementation.
Back to the question of having a piece of code where it is really important to know that something happened. If that is so, then apply formal analysis and prove it, rather than test it. I made the whole series of videos explaining techniques we apply to that end: ua-cam.com/play/PLSDYwLgFqaX6rSZSEVpaI4mjFXTWZqBrM.html
@@zoran-horvat Thank you for your insightful answer. That definitely helps me to better wrap my head around it. And thank you for the playlist! I'll be sure to watch it.
Great advice Zoran! How do you handle testing of void methods though? Since they usually modify internal state or notify someone else, there's little choice but to inject mocks or check an objects state, but this often requires much more knowledge about internals than I want to have.
In my opinion, every method must correspond to a request for a certain externally visible effect. If you can formalize the request and turn it into a proposition, then the test would assert that that proposition is true after making a call.
This may sound overly abstract, but once you get to the bottom of it, you will see that you can establish a process of writing tests that are natively resilient to implementation changes.
@@zoran-horvat So basically, instead of having a specific output as a concrete return type, your method has a concrete output but in the form of a state change or a method call down to another component, and you'd still observe this behavior without caring about internals and how the method gets there?
@@allinvanguard That is the idea. Imagine it is an interface with no concrete implementation. What is the expected observable change that must happen if interface were implemented according to requirements?
Keep in mind that this method is not applicable universally. But it can easily cover 90+% of requirements in any business application, i e. application which is not heavily algorithmic in its nature.
Svaka cast
Can the currency type of the money really be considered an implementation detail? From a code perspective, any public method or value is part of the interface, so the fact that currency can be accessed by our "brittle" test at all means that this test seems to be testing the interface of Money. if currency type were a private variable, then i'd agree with you that we shouldn't test for it directly. But then we wouldn't be able to test against it at all because we have no way of accessing it from a test.
Secondly, if your business logic fundemantally changes (IE: the types of currencies that can be added and subtracted changes), then why wouldn't you want a unit test to fail? If I fundamentally changed business logic and didn't see a test fail, that would be a red flag for me because it means that the old business logic didn't have any automated tests that were reinforcing behavior. I could understand if by "implementation detail" you meant wheter we persist to a sql database vs a nosql database, but currency type is a fundamential part of the domain, and busienss users expect this to have a specific behavior regardless of the underlying implementation.
There is wisdom in what you are saying, but in the sample code I have taken a step further. Though every public member is part of the contract in its own right, you don't have to depend on unnecessary parts in every test.
Whether clearing the currency was an express requirement or programmer's way of solving a technical problem, either way it is irrelevant to the test that checks that the result is zero, whenthere is the property that defines zero. I have used the public interface of the class to express a property that must be true in that test with least assumptions.
Regarding the note about wanting a test to fail, there you are right. However, that would be the change in requirements for that test, and it would be by design for that test to fail.
Again, this case from the demo is not of that kind. The substantial properties of addition and subtraction did not change, and so I would expect all tests that used to be green to remain green.
Brittle tests are written after the fact, write them before the implementation.
Legacy code is a challenge to test, often there's nobody around who can tell you what it should be doing.
Tests before implementation don't warrant much if the one who writes them lacks understanding.
I would say it goes the other way around - those who understand how to avoid brittleness will tend to write tests before implementation, because they see how that works for them.
@@zoran-horvat You write one test, make it pass, refactor, repeat. Not write all the tests in one go. You learn as you go, covering the use cases, thinking up also unusual use cases, with some help from the domain expert and colleagues. And you still don't get it all covered, the use cases nor test coverage. You know how this goes.
Hard to see how that brittle case could have been avoided without understanding. It's not enforceable at the Interface contract.
@@zoran-horvat Very much enjoying these videos. I wish that I could afford the sub but I'm a pensioner, still fascinated after all these decades. Our industry surely needs the education, if you think that you have frown marks then you should see mine!
Keep up the good work.
if your test isn't testing an implementation what is it testing? The "unrelated" test failed because it tested an implementation and failed because the implementation was no longer valid. So good test
No. You got it all wrong.
The true test verifies externally visible attributes so that you can reimplement the method in a different way later and not break the test, so long as the attributes are preserved.
Testing the implementation is precisely what gives you brittle tests - the principal reason why so many teams abandon tests mid-project or start ignoring them.
@@zoran-horvat You seem to be thinking of units like pure functional methods. Unit tests are written based on an assumption and there's usually an assert of that assumption. The assumption/implementation that the test is based on should fail if that assumption/implementation changes, otherwise it's a useless test. If a test was written with some twisted concept of zero money, then it should fail when that twisted concept of zero money changes and a new test should be written. The worst kind of unit test is the one that passes all the time even when the implementation it is based on changes. It means either the test wasn't written correctly or it never worked well in the first place.
@@auronedgevicks7739 You are then making more than one assumption per test, if the assertion depends on whatever the test days it is verifying, and on the internals of zero. That I find to be the cause of brittleness.
I agree with the fundamental thesis of the video, but the example is not well chosen. It is actually very inappropriate to make the different currency zeroes equal. Comparing different currencies should always throw an exception, it should be forbidden just like doing arithmetic on them, zero or not.
If you think it through, you will see that in this system there really exist two definitions of zero. It becomes obvious in an operation which aggregates a sequence of money objects - what is the seed?
Currency.Empty for 0 Money... fucking genious! 😲
It is the seed for any Aggregate operation that sums up a sequence of Money objects.
@@zoran-horvatSumming different currencies is not good idea in general... sums grouped by currency ok, but scaling to single currency can be from not easy to extreme complex depending is this a accounting or banking use and complexity of accounting laws and tax system
Let's go extreme OO, make every currency subclass of Money 😉