What a Recent Project Taught Me about Acceptance Testing
Вставка
- Опубліковано 18 гру 2022
- Granted, this one is a little unusual, relating more to software development rather than computational redstone. But this is something I wanted to share because, as a self taught programmer, I've always struggled with feeling like my skillset was good enough to be called a proper programmer.
Having discovered the usefulness of acceptance testing, I wanted to share what I learned, as well as how I had learned it, as I figured it might help drive the idea home.
Don't know if this helps anyone, but it's still something I wanted to share.
LINKS:
Unit Tests in Python by Socratica: • Unit Tests in Python |...
Unit Testing your Code with the unittest Module by Corey Schafer: • Python Tutorial: Unit ...
What is Unit Testing? Why YOU Should Learn It by Andy Sterkowitz: • What is Unit Testing? ...
Clean Code Part 4 by Uncle Bob and UnityCoin: • Clean Code - Uncle Bob...
Smallest Redstone Computer Tutorial by ct5k: • Smallest Redstone Comp...
Redstone Hex Computer Ep.3 by ct5k: • Redstone Hex Computer ...
Acceptance Testing for Continuous Delivery by Dave Farley and GOTO: • Acceptance Testing for...
My Discord: / discord
Thanks for being the person to finally describe testing to me in a way that doesn't make it seem like a massive time sink
Yeah, that's the one thing I also found to be annoying about other tutorials, they didn't really do a good job explaining why this stuff is useful.
The only exception being uncle bob and his clean code lectures. But that's like an 8 hour ordeal.
Anywho, glad I could offer some clarity :)
The testing you are describing is integration testing, testing the entire functionality of your application with normal dependencies. Unit testing is different in that each test should only test one "unit" usually method/function. If when unit testing a class has dependencies; when instantiating the class its dependencies can be mocked and injected into the object. This pattern is dependency injection, and can also be helpful with integration testing as well, for example running the application in a different environment or with a different module.
Well, I certainly get that.
That was the style of unit tests I had used initially, and when testing classes, mocking and dependency injection was what I would use to isolate the object.
Hell, even with my new style of testing, I'm still mocking system calls to ensure my tests aren't writing to disk or anything like that.
Thing is though, while I get what unit tests do, I don't get why people consider them useful.
They seem too tightly coupled to the code under test, making things rigid and difficult to change.
Plus, I don't see why we need to tests units in isolation when testing everything together still shows that everything works.
Maybe I'm misunderstanding something, but if you have some insight, I'd be more than happy to listen.
I love your videos, they are very educational and explain things very well!
awesome video! Its always interesting to discover good techniques for programing with more success, especially testing techniques 👍
Oh, I'm really happy with this one. Because it seems with this one I can throw all the others out the window :D
In all seriousness though, it is very nice to not have to worry so much about how you're going to structure your code, which seems to be the focus of most paradigms, and to instead just let things flow... sort of allow whatever comes of it to happen.
It may not be the best looking code, or even the most efficient, but the tests say it works.
One of my colleagues wrote a lot of "procedural" spaghetti code, and now I have to rewrite all of it, it's hell :D
Great video btw, glad to see you!
I'd be curious to hear his justification... I don't particularly agree that that is the best way to write code, but I'm always willing to see if someone knows something I don't know.
Thanks for the great video!
I also just recently have experienced the same fate; gaining the necessary knowledge to call myself a real programmer, despite having been programming for about 6 years.
But my means of gaining that knowledge were very different.
I started programming in Python, and about 2 years ago I began learning C.
The inherent characteristics of compiled languages helped me reduce runtime errors; compile-time error checking was a true blessing.
The lack thereof stood in the way of progress when I was using Python.
Obviously, primarily type confusion errors were pretty much completely eliminated. These were so pesky to eliminate, since they were only catched at runtime. Therefore, they sometimes only occured on edge-cases which were hard to catch. Now they were all printed out with defintiive 100% coverage.
But programming in C obviously introduced memory errors, which were caused by errors in the program logic itself, as well as integer overflows. Meaning I still accumulated way too much technical debt; it was just that the compiler got rid of some of the symptoms. But the lack of modularity in C didn't make things easier.
Then I heard of Ada. It was mentioned in a video I watched as being very safe and used in the military, aviation, space and infrastructure industry (where runtime errors are potentially deadly). I am very interested in these fields and hence I started to learn Ada. The GNAT compiler is very powerful and catched pretty much all formal errors, meaning I could fully concentrate on the bare program logic. The modularity helped me develop well-wrought software and the strong typing system incentivised me to give meaning to types.
For example, while I previously might have simply stored the time of day as the amount of minutes as an unsigned integer in C, I now declared a new type Time_Of_Day, which was whole number ranging from 0 to 1440, spanning 24 hours in minutes.
GNAT also implements runtime overflow and range checking as sort of a compromise between dynamic and static error detection. Ada gives me enough freedom unlike languages like Rust where mechanisms such as borrow-checking introduce way too much development friction, but still detects most of the errors already at compile-time and displays helpful information when unwanted runtime-errors like integer overflow occur.
Ada also makes design-by-contract easily possible, which I can then use in combination with Ada/SPARK in order to prove the implementation fullfills all the contracts and isn't prone to runtime-errors. In order to be proveable, my code must be written according to a strict set of rules SPARK dictates, which is why I only use it to prove the core logic and design the rest of my system around the proven core logic.
Sorry for the long comment; I mainly wanted to recommend to you formal/static verification (which only really is possible with compiled languages). This performs formal proofs that your implementation corresponds to it's specification, thereby theoretically ruling out absolutely every possible error. This would also cover the 2% you're missing, which could potentially contain off-by-one errors which are very hard to catch through testing, aside from the fact that as systems get larger, there are exponentially more cases to test.
Have a nice day!
Oooh, that's a big comment.
Ada huh, I'll have to check that out.
I can see the usefulness of checking for runtime errors beforehand, though I've never heard of design-by-contract.
That's a new one, I'll have to check that out too.
That's also an astute guess that my 2% missing coverage was a result of runtime edge cases.
That might still be true for maybe 0.5%, but a good portion of that missing 2% is just code that got mocked out of the tests, like functions that wrote to disk :P
@@n00b_asaurus Yes it might not be worth the effort, but if your systems get very large, you might have a look into it.
I'm currently writing a compiler and these methods help me ensure integrity.
banger video
From experience (C and C++ for 10 years ish) I would strongly recommend revisiting procedural/imperative programming (optionally with some pure functions), as it's a fundamentally more scalable and superior way of programming in the long run.
the problem with OOP is that it imposes structure/abstraction boundaries at all- this is 99% of the time detrimental to development long term- that is, no structure is better for software than bad structure. The central idea in programming should be: "What is the least amount of code that i could write to achieve this task?" maintainability/organization then comes only as necessary. (note: this means the use of functions is not for organization, but rather consolidation of duplicate code, and there is an art of when to use functions.)
Also, Test Driven Development and Acceptance testing is definitely very useful, however, even more useful is Unit testing, and minimizing and organizing (only data! not objects) state, and the code which touches state. Also trying to write pure functions whenever possible is helpful. I've found this is drastically more effective at ensuring the application has no bugs than trying to write all the tests prior, and hope they don't have to change, because they always will (or have a bug in them.)
Simplicity is the friend of execution- and this is especially true in programming!
I mean... I'm not disagreeing with you...
As stated in the video, I would agree that writing code procedurally/imperatively is much more effective when it comes to getting an idea into code.
However, at some point, it becomes beneficial to start organizing code to make it easier to understand. You read your code more often than you write it, after all.
Of course, rewriting your code does carry the risk of it not working the same as before, which is where the tests come in.
As for the OOP vs procedural vs functional thing, I think there are uses for each method. Usually when I write code, by the time its at a point that I'm happy with it, it usually contains code of all 3 types.
I think being able to explore different structures is helpful, and maybe procedural is the best way to organize something, but maybe OO is better still.
Again though, you can only verify that everything still works as intended if you have those tests in place.
I think testing at the top with an environment layer inbetween to decouple your tests from your interface details gives you the most flexibility to change your code all you want. And as stated, if the coverage report is showing everything is being covered, I see no need for unit tests.
That said though, I do see a number of people stating unit tests are the bees knees... so I'd be curious to hear what they are to you and why you find them better than acceptance tests.
Again, to me, they seem too tightly coupled to the codes implementation details to be of any use, but maybe I'm misunderstanding them... this wouldn't be the first time :)
Fair warning though, I prefer chatting on discord because youtube doesn't notify me when comments get replies.
If you prefer youtube comments, that's fine. It just might take me a second to get back to you.
Another issue I have with unit tests that I don't think you mentioned is that unit tests are very good at finding trivial bugs (typo in my add method) and very bad at finding difficult bugs (when component A passes data to component B it expects component B to now be in state 1 but actually because B received data from C afterwards it's now in state 2). It's in the complex interactions between various components of your system that usually the difficult bugs are, and unit tests are basically not able to reason about that at all, while various end to end tests like what you described are.
That's a good point. I hadn't thought of that.
I worked at a firmware developer for 3 years. There I did a lot of documentation, testing and bug fixing and barely any development of new code. There we had devices which communicated with other devices. And we had a setup which connected to all inputs, outputs and communication busses and checked if their behavior is correct. And the behavior we checked is anything which was in the requirement sheet and some default behavior. Additionally the test got extended every time we found some bug. In our case the test setup took about as long to build as the devices themselves.
In my private project of having the game I don't really do any testing at all. At least no written out tests. I don't know how I should do that in such a graphical project. But when I implement a new change with changeable variables I implement a few possible problematic cases and do basically a manual test. And I guess I run the game and if it runs then the basic function is tested as well.
I also don't really consider myself a programmer. I do enjoy it and I have some basic understanding in it. But I avoid anything which I is a bit more complex. I'm just doing the simple stuff. By now classes also count to being in the doable range, but programming patterns are too much for me.
In other words, you're about where I was not too long ago :P
I would expect a good testing framework to probably take just as long to build as the actual product, but I think the time saved in testing is well worth the effort.
Still, if you're curious about testing game elements, Mojang has some very interesting test automation ideas. Look up 'testing minecraft in minecraft', I think you might find it fascinating.
Of course, at the end of the day, they are your projects, do what you want :)
@@n00b_asaurus I couldn't find the video. Neither by titel nor on the Minecraft youtube channel. Could you give me the link or channel?
I've never really done unit testing. I always write a bunch of the code, figure out what I really want everything to look like, ultimately rewrite the code, then I actually start testing it.
I tend to write janky garbage as a proof of concept / prototype, and I only do surface level tests to make sure the concept could work. Once I actually know what I am doing in terms of that project, I rewrite the whole thing (sometimes in a completely different language), with new knowledge of how all of it is going to work.
I've done that too, never really felt confident in my tests when I did that though.
I'm not sure if you caught my testing hierarchy there, but that whole "identify features, write user stories, create acceptance tests" really helped when it came to identifying not only the happy path, but the not so happy path too.
For example, my save and load profiles feature has some user stories that cover the basic "save profile and load profile" scenario, but it also covers things like "the user shouldn't be able to do anything until they create or load a profile" or "closing the program without saving the profile should result in a warning".
With each of these user stories having their own set of acceptance tests that verify these stories are being played out by the code, I have much more confidence in my code than I ever would have had if I had done after-the-fact testing, or no testing at all.
To each his own though.
That said though, I'm also not above jank proof-of-concept code. Sometimes you just need to see if something works.
@@n00b_asaurus yeah, to each their own I guess. I probably should do more extensive testing though. Sometimes I find bugs in my code that could have easily been found earlier with some testing.
I've always found that what I usually do is make what I think is correct, forget a semi-colon somewhere 🤦♂️, run the program, play with it a bit, then if I see a problem, then I'll scour through the code. usually I'll have a rough idea of the problem based on what occurred, though most of the programs I write are interactable with visual output, so it's easier to catch if something goes wrong, I suppose
Yeah, that's about what I was doing for a while there.
It works in a pinch, but I found myself struggling quite a bit if I wanted to build anything bigger than that.
I threw in some code from an accounting program I am working on to demonstrate this. That was a program I had struggled with for the better part of a year. It was only just recently that I figured out a good testing infrastructure and began making some good headway.
I don't know though... I suppose at the end of the day, the only thing that matters is that your code does what you want it to do :)
ahhh im at school ill watch later
im back