Pro tips for writing great unit tests - Raymond Hettinger
Вставка
- Опубліковано 23 лип 2022
- Pro tips for writing great unit tests - PyCon Italia 2022
There is an art to condensing test concepts into readable, fast, clear predicates.
- We look at many examples and show how they can be improve
- Master the use of any() and all() with generator expressions.
- Expression set() relations to express big ideas clearly.
- Cover the problem space with combinatoric iterators: product(), combinations(), permutations()
- Use subtests for clear error reporting
- Factor-out data acquisition and parsing from the test predicates.
- Learn patterns for testing edge cases.
- Cover class based technique for test reuse.
- Make effective use of setup and teardown.
- We look at many examples and show how they can be improved
- Treating TDD test cases and bug test cases with special reverence.
Speaker: Raymond Hettinger - Розваги
Any topic Raymond lectures, becomes your favorite topic!
I have watched MANY Hettinger talks and they NEVER disappoint. The man is a legend and a gift to the entire community.
One of the best speakers about Python!
I'm a big fan of Hettinger but to me this talk focuses too much on core python syntax. I can watch Hettinger talk about core Python all day so I still enjoyed this, I'd never have thought to use `mode` to get the most common item out of a collection. However I was hoping to get some insight in to which bits to test when, how to keep scope of tests well defined and how to achieve meaningful coverage rather than just pushing up the coveralls metric. If anyone knows a good talk like that could you link it in a reply?
Again, not a complaint, this is a good talk, just not the one I'd hoped to find.
and by the way, this video is not only about tests. It is about clear python code
Absolutely, loved it. Anything on python from Raymond is always learning. Where can I find the slide or the source for this wonderful presentations?
Really motivated me to do Python coding in the weekend 😀
Again an amazing talk by Hettinger, simply a tour de force of modern Python. Would it be possible to share the slides as well?
Amazing!!
Great talk - anyone know where to find the slides ?
Fantastic talk THANKS. Are the slides available somewhere?
Where can I find those exercises/slides?
Can someone tell me what tool is he using for that documentation slide? the one that generates html file. I kinda forgot
1. 24:32 What does he mean "exception world to data world"? Is he talking about the type(e)?
2. Why is the returned list designed to have None in different positions? Doesn't it also work with the existing assert below, without the list and None added?
3. 31:27 shows combination of conditions from multiple assert into single assert. If test fails, wouldn' t this hide which line and which condition failed, making it harder to fix code? Are there features that show you exactly which of the conditions that were logically operated on together failed? Does it work for even shorter syntax of any/all too?
1. The key here is the difference between _raising_ an exception, and _returning_ (the type of) an exception object.
I think with "worlds" he refers to what determines the flow of the program, and what we have to keep in our mind.
Usually, in "data world", functions return data, and based on how that data looks, we decide which pieces of code get executed next (loops, if-statements etc.).
But in languages like Python/Java/C++... code can also raise exeptions, and then the program gets thrown out of the usual control flow and enters a completely separate one (try, except, finally...).
This is often useful, but here it would complicate the testing logic.
By catching the exception right away, turning it into an object e, and incorporating just its type into the return value as normal data, we still get all the information we need, but outside the function we can now always stay in "data world". We don't have to handle special cases, and we can be sure we always get back the same type: _list[Any, None] | list[None, type]_
(Some languages like Rust or (apparently?) Go always stay in "data world" like this, and some people think that this way of handling errors is fundamentally better than raising & catching exceptions.)
2. Because then we can clearly distinguish the return value of the wrapped function (at index 0) from the type of exception that was raised (at index 1):
result, error = capture(...)
You _could_ of course also write a wrapper function (let's call it capture2) that just returns the exception's type if an exception was raised, and just the result from the inner function otherwise.
Note that if capture2(foo) == ZeroDivisionError, that can mean one of two things:
1. Calling foo() _returned_ a ZeroDivisionError class object, or
2. There was an error in foo(), and it _raised_ a ZeroDivisionError exception
I'll admit that I don't often write functions that are supposed to return type objects - but when I do, I don't want my test to say "Yup, your new refactored version of that function behaves exactly like the old one", and then that new function crashes the production server at 4:00 in the morning. :^)
Raymond's version of capture is only slightly more complex to write, but in this case it doesn't make any difference as to how it is called, and you have the guarantee that it works for all functions you may ever want to write, and you don't have to consider weird edge cases.
3. Good question! There is sometimes a tradeoff between "easy to write & read the test" vs. "easy to figure out what caused the test to fail". But pytest (IDK about unittest) indeed has a feature that automatically rewrites "assert" statements behind the scenes to decompose the expression into its parts.
With that, Raymond's shorter version actually _improves_ the error message. Under "short test summary info", it gives:
FAILED my_test.py::test_old_version - assert False
FAILED my_test.py::test_raymonds_version - assert (False or 33 > 50)
And above, in the more verbose traceback above (on the terminal you get syntax highlighting & the error line in bright red), it becomes clear where those values came from:
def test_raymonds_version():
> assert load_limits or load>50
E assert (False or 33 > 50)
In general, it's awkward to see "assert True" and "assert False" - the former does literally nothing (like "pass"), and the latter just gives you "AssertionError" - you'll have to look up the line manually.
One improvement would be to add an explanatory message to the assert statement:
>>> assert False, f"load_limits is off, so we expected load > 50, but we got: {load}"
(And instead of "assert False", I'd much prefer "raise AssertionError('...')" )
A huge plus of pytest is that you _usually_ don't need to add any custom messages like this.
3. (cont.) Unfortunately, pytest isn't great with "all" and "not any". (We don't really care about "any" or "not all", because if those fail, every single value we tested is equally "responsible".)
Here's an example where I'd usually go for all/any:
DEFAULT_SETTINGS = dict(bells=0, whistles=1, knicknacks=0, experimental=1)
def test_everything_off_any():
assert not any(DEFAULT_SETTINGS.values()
This yields a very noisy stack trace, with the whole dict (not just the problematic entries!) somewhere in there.
Worse, if our dict becomes too big, it would get cut off at some point. And if we use use some lazily evaluated iterable, we're out of luck completely: "where True = any()".
So here are two ideas for workarounds, and their pytest output (I prefer the latter):
def test_everything_off_forloop():
for opt, value in DEFAULT_SETTINGS.items():
assert not value, f"{opt!r} should be OFF"
# AssertionError: 'whistles' should be OFF
# assert not 1
# (Note: Stops testing at the first bad one, and doesn't tell you that 'experimental' is also bad.)
def test_everything_off_dictcomp():
bad_settings = {k:v for k,v in DEFAULT_SETTINGS.items() if v}
assert not bad_settings
# AssertionError: assert not {'experimental': 1, 'whistles': 1}
30:35 It is just me? I prefer more explicit if elif because it doesn't make "me think" than these cute one liner logicial operations.
you probably just like it more because that is what you are used to. a lot of his talks are like that.
btw -- you are definitely not alone. a lot of people feel this way and in his videos the audience sometimes interrupts him to object.
isdisjoint, Oh my 🥲