Tip 2 is spot on. I am a religious follower of pep 484, and coming from C++ experience, I use python almost as if it's a statically typed language, with type hints for every variable and function. I find it makes the methods cleaner and their purpose more obvious from the signature. To me, type hints are not optional, they are part of the language.
I just discovered your channel right now while I was searching about "Single Responsibility Principle" Glad you're very much active AND putting out code principles! Great content
11:22 A minor note on the refactoring: If you were to use this function with a set, you might not get the results you expect, since a set does not preserve insertion order like a list or tuple does. So the list of lengths returned would likely be difficult to correlate to the input set. Of course, this is a manufactured example, and in the real world, you'd probably use collections.Counter for something like this.
Such a great educator. Took me years to learn and apply these practices, yet Arjan presents it so concisely. Such good advice. Practice these things, get better at code, make happy maintainers. Thanks Arjan, great video.
In Final thoughts Arjan says "Do not focus on optimization" - totally agree, I spent 2 months creating "perfect solution" which later turned out to be not needed. My advise is to firstly create decent prototype, far from ideal and then refactor once new requirements will come in. Totally agree regarding input arguments -> at first do not try to be too much generic, it requires time and you may never benefit from it. Regarding output, be always specific, never try to be generic.
This great because this is almost exactly relatable to my current python project using geospatial data. I have gained so much knowledge by looking at other people's code (be careful who's code you read), and applying those concepts to my code. Your use of data classes is so simple and easy, and it makes handling some of the errors I received from my first edition of the project much easier.
This video is fantastic as always. Grateful to know about the Sized typehints. Also, the best practices example along with Armstrong's quote is excellent.
Thanks Arjan for another piece of super useful content, even learned something new about the "Law of Demeter"... Regarding the count() function, i would prefer just to write "return [len(item) for item in items]", this saves a lot of boilerplate and is more expressive. Maybe the name count() is a bit counter-intuitive as well, because counting things almost always results in a single integer value. Type hints are a life saver for everybody who needs to dig into code written by somebody else regardless of the value they provide for code completion in IDEs. However, they can get a bit clumsy, especially when it comes to annotating callables or collections of complex objects. Aliases are a nice way to overcome that ;-)
I like the Sized, Sequence, and Iterable over a plain list for type annotations (which most of my colleagues use), though I'm sometime too lazy to bother because of the imports, and my teammates doesn't really see the benefit. I think using type hints like list[str] clashes with Pythons very dynamic approach to typing, and might prompt unnecessary data conversions to satisfy the type checker. I use types most places, even throwaway code, because it helps me to mentally lock down the function signature, rather than introducing a bunch of if statements for handling different data types, just like you mention.
Excellent thought process! Remember that so-called throw away code is being done to prove a concept or prototype a design idea. I can't tell you how many times those exercises have been rushed into production because I did too good a job of marketing them. It is good to develop the discipline that assumes POC code will be in production - tomorrow. Thanks for sharing this very practical perspective!
Arjan, you are by far the best Software Engineering related content creator and I appreciate you so much. Thanks for making complex and challenging topics seem cool and fun for all of us to benefit ❤
Here! Here! As someone who just wrapped up a 40+ years career I totally agree with your assessment with the apparent ease in which he presents topics in a very digestible manner. But as someone who has done a tremendous amount of training myself; I am here to tell you it is a LOT of work. My rule of thumb was 8 hrs of prep for 1 hr of delivery of the training (on average). @ArjanCodes - you are incredibly talented and IMHO have truly found your sweet spot. Please keep it up.
Just came across your video and I just love your flow and style. I am a self-taught new-ish python programmer, and resources like this are invaluable to me. One suggestion I have to help people like me would be to use much simpler, straight-forward code as examples. For this video and a few others, I find myself more focused on trying to figure out the code for URI encryption etc instead of really just focusing on the main point. Other than that, I love what you are doing and keep it up
Hi Paul, thank you! Your suggestion for simpler examples makes sense. The reason the examples may look complex is that I want to make sure they are practical and close to what production code could look like (you won’t find pizza toppings or animal class hierarchies in my videos). It’s a careful balance between keeping things simple enough so it supports the explanation, while at the same time not being a trivial example that’s hard to translate to the real world. I don’t always succeed at that, but it’s something I always strive to improve on.
The tip about using external libraries is such a great one and something I took a long time to follow. As well as the benefits you listed, another one is there's usually better documentation that you can point others too.
I find it useful that a function can return some type like a list when everything is fine, but also I allow it to return eg. None, when sth goes wrong, so that program can still go on. I know I can catch an exception of course, but that approach is also useful to me. I don't like when there is too much of stack trace in the logs. Especially when some failures are expected (up to some point).
I really enjoy your videos. I especially like the code roasting ones as they teach you how to work with unfamiliar code. One small nitpick. Just as you shouldn't make the parameters types too broad, you should not make the return type too specific. The reason is, that the only changes you can do to the function's types without potentially breaking the code that calls it is to make the parameter types even broader, and the return type even more specific. A good rule of thumb is to start on the more conservative side with the interfaces and become more liberal once you're confident in what the code does and how it does it.
For number 2, I would prefer to see a friendly API called to_uri() that accepts and converts the three different datatypes, then calls a _to_uri() function that handles the logic. Just as easy to test, but way simpler to use.
i think that would just add complexity to your codebase as generally you will know what type you're working with from the start so having a function trying to figure out which type you're working with adds complexity instead of just calling from the type directly, then you have a public accessible bare bones one which just takes the bytes, it would leave the user able to create their own functions too if you make your own object which you want to convert to an uri, and if you need the dynamic approach you can create your own, but they will all just use this one function
I usually use those simple type hint as list, dict, str, int, and may be some class in my code, very rare to use custom structure for type hint. Anyway, may I request for 1 ep to show example of libraries that people missed out and would save the time for developers. Thanks anyway for new videos.
Thank you so much for your work, it's very helpful! Could you please consider a topic of Software Architecture? E.g. building delivery app from a scratch, consider how to design components, their relations, DB, deploy(docker), maybe microservices architecture. It's more like advanced level. It might be very helpful.
To the question at the end of the video. I found useful the Postel’s law: be conservative in what you send, be liberal in what you accept". So yes, as example i try to accept types like Mapping (which can be dict or defaultdict, UserDict etc.), but return dict to be precise…
Neat tips. Thanks. If I may suggest, there's Jeff Delaney who runs the Fireship channel. Try to be a bit like him, to be more concise and just stick to the bare minimum most crucial things. That way instead of havint to create a 17 minute video, you could just tell it all in 5 minutes tops.
Pretty much the message is to consider the "life cycle" of the product. During the whole life cycle of the product, i.e., conception through EOL, you typically find that 80% of the effort is done in post-production -> versioning to expand features, optimize speed, debugging and/or general maintenance. The remaining 20% is split 10% design, 5% planning and 5% testing. Yes, most of the time you'll be working in a team, so the amount of code per project per person is really not that much. On the other hand, if you're coding by yourself, then the numbers typically flip -> 80% split between design, planning and testing. The remaining 20% will be complaining about all the features you want to add but can't due to the limited you have (forget about maintenance and testing, takes too much effort). If you are the latter, better listen to Arjan!
Very good tips, especially about type hints at the end. I've used Iterable before, but never knew about Sized. A disadvantage of Sized is that less seasoned Python developers might find them confusing, so maybe it's a bit overkill in a project where the Python knowledge is lower. I think it's neat though. Going from list to Iterable enlarges your options, but Iterables have a feeling that they are fundamentally "like lists" (think a tuple), so conceptually this does not differ very much. But going from a dict to Hashable will allow e.g., integers, which are conceptually very different from a hash map. I think that enlarging a type hint to objects that are conceptually very different from the original might make it more unclear what the function really needs the arguments for. Do you have any thoughts on this?
I really like your video. Just one question about the second mistake. If we raise exception, then we have to handle it at every call to it. Is it the same as we return URI | bool which we have to handle the returned bool value?
Just a question about the third example. We annotate the return type as list[int] a.k.a. the actual python list type. Now my question: which is considered best practice? to use list[int] or List[int], where uppercase List is imported from typing? Could not find a PEP for it.
Hmm, doesn't going to the effort of being more generic than might be required at the time the code is first written go against the YAGNI principle a little bit? Maybe we should wait until we actually need/want to support more generic types?
Absolutely! I think spending some time on making the types more generic mostly makes sense if the code is not for your own internal use but for others.
For mistake 2, I'd prefer to have a bland util function which converts int/str/bytes to str. It could be used anywhere in the codebase and easily tested since the usecase is narrow and clearcut. Then you can just call that on the input and hand the output to the original function which now only takes str and only need to be tested for that.
In your Iterable example, wouldn’t the Iterator type be more appropriate? Trying to wrap my head around the difference between the two, but it seems like you’re not guaranteed to be able to iterate over an Iterable, as you’d have to call iter() on them first? Of course, most Iterables are also Iterators.
Hi Arjan! In VSCode you can use CTRL+D/CMD+D for replacing a recurring sequence of characters. At 4:07 this would have saved you quite a lot of keystrokes :) Just select the code you want to replace (location.geolocation[0]) and hit CTRL+D/CMD+D one time per extra cursor you want. You can then simply type geolocation to replace the code in all places at the same time. The VSCode docs have an entire "multiple-selections-multicursor" section covering similar shortcuts and options to save you some valuable keystrokes. It is an honor to gift this community over 500 of my keystrokes. Fun reading tip: "Do they deserve your keystrokes" by Scott Hanselman
I do a lot of type hints; especially if I have my own internal object models; I am at the point where I am looking for typehinting and validation for DataFrames;
Iterable and Sized and such is a concept used ALOT in C#. It becomes second nature but because its an indirection task, it usually confuses newcomers. Although, I prefer to just explain it and have them understand whats going on..
"How much attention do you give to typing" -- so much. Too much arguably. I can and do spend all day going over old code and streamlining it and adding typing. My old code was *rough*.
👷 Join the FREE Code Diagnosis Workshop to help you review code more effectively using my 3-Factor Diagnosis Framework: www.arjancodes.com/diagnosis
Tip 2 is spot on. I am a religious follower of pep 484, and coming from C++ experience, I use python almost as if it's a statically typed language, with type hints for every variable and function. I find it makes the methods cleaner and their purpose more obvious from the signature. To me, type hints are not optional, they are part of the language.
The good thing about using type hints is that it forces you to use abc's or protocols, which improves quality and structure of code
I just discovered your channel right now while I was searching about "Single Responsibility Principle"
Glad you're very much active AND putting out code principles!
Great content
Awesome, thank you!
Which challenge?
breaking up complex if-else statements in a function to dedicated functions was super useful. Thank you!
You’re welcome, David!
11:22 A minor note on the refactoring: If you were to use this function with a set, you might not get the results you expect, since a set does not preserve insertion order like a list or tuple does. So the list of lengths returned would likely be difficult to correlate to the input set. Of course, this is a manufactured example, and in the real world, you'd probably use collections.Counter for something like this.
Favorite programming channel ❤
4:18 if you had selected ''location.geolocation[0]'' and hit ctrl+d you can easily have selected all of them. Quite a neat trick
That’s one of many things that I learned through your 30 days challenge. Thanks for that!
You are so welcome!
Love your new color scheme Arjan and your videos only get better and clearer. Thanks!
Thank you so much!
Such a great educator. Took me years to learn and apply these practices, yet Arjan presents it so concisely. Such good advice. Practice these things, get better at code, make happy maintainers. Thanks Arjan, great video.
Thank you so much!
Passing an Iterable is a great piece of advice rarely seen/provided.
In Final thoughts Arjan says "Do not focus on optimization" - totally agree, I spent 2 months creating "perfect solution" which later turned out to be not needed. My advise is to firstly create decent prototype, far from ideal and then refactor once new requirements will come in.
Totally agree regarding input arguments -> at first do not try to be too much generic, it requires time and you may never benefit from it. Regarding output, be always specific, never try to be generic.
This great because this is almost exactly relatable to my current python project using geospatial data. I have gained so much knowledge by looking at other people's code (be careful who's code you read), and applying those concepts to my code. Your use of data classes is so simple and easy, and it makes handling some of the errors I received from my first edition of the project much easier.
This video is fantastic as always. Grateful to know about the Sized typehints. Also, the best practices example along with Armstrong's quote is excellent.
Thanks Arjan for another piece of super useful content, even learned something new about the "Law of Demeter"...
Regarding the count() function, i would prefer just to write "return [len(item) for item in items]", this saves a lot of boilerplate and is more expressive. Maybe the name count() is a bit counter-intuitive as well, because counting things almost always results in a single integer value.
Type hints are a life saver for everybody who needs to dig into code written by somebody else regardless of the value they provide for code completion in IDEs. However, they can get a bit clumsy, especially when it comes to annotating callables or collections of complex objects. Aliases are a nice way to overcome that ;-)
I like the Sized, Sequence, and Iterable over a plain list for type annotations (which most of my colleagues use), though I'm sometime too lazy to bother because of the imports, and my teammates doesn't really see the benefit.
I think using type hints like list[str] clashes with Pythons very dynamic approach to typing, and might prompt unnecessary data conversions to satisfy the type checker.
I use types most places, even throwaway code, because it helps me to mentally lock down the function signature, rather than introducing a bunch of if statements for handling different data types, just like you mention.
Excellent thought process! Remember that so-called throw away code is being done to prove a concept or prototype a design idea.
I can't tell you how many times those exercises have been rushed into production because I did too good a job of marketing them.
It is good to develop the discipline that assumes POC code will be in production - tomorrow.
Thanks for sharing this very practical perspective!
Arjan, you are by far the best Software Engineering related content creator and I appreciate you so much. Thanks for making complex and challenging topics seem cool and fun for all of us to benefit ❤
Thank you so much!
Here! Here!
As someone who just wrapped up a 40+ years career I totally agree with your assessment with the apparent ease in which he presents topics in a very digestible manner.
But as someone who has done a tremendous amount of training myself; I am here to tell you it is a LOT of work.
My rule of thumb was 8 hrs of prep for 1 hr of delivery of the training (on average).
@ArjanCodes - you are incredibly talented and IMHO have truly found your sweet spot.
Please keep it up.
Thanks for posting this. Been wanting to refactor my code and tips 2 and especially 3 are going to help. Hope to see more of these soon
Glad it was helpful!
Just came across your video and I just love your flow and style. I am a self-taught new-ish python programmer, and resources like this are invaluable to me. One suggestion I have to help people like me would be to use much simpler, straight-forward code as examples. For this video and a few others, I find myself more focused on trying to figure out the code for URI encryption etc instead of really just focusing on the main point. Other than that, I love what you are doing and keep it up
Hi Paul, thank you! Your suggestion for simpler examples makes sense. The reason the examples may look complex is that I want to make sure they are practical and close to what production code could look like (you won’t find pizza toppings or animal class hierarchies in my videos). It’s a careful balance between keeping things simple enough so it supports the explanation, while at the same time not being a trivial example that’s hard to translate to the real world. I don’t always succeed at that, but it’s something I always strive to improve on.
The tip about using external libraries is such a great one and something I took a long time to follow. As well as the benefits you listed, another one is there's usually better documentation that you can point others too.
Many thanks for your valuable videos from South Korea.
I find it useful that a function can return some type like a list when everything is fine, but also I allow it to return eg. None, when sth goes wrong, so that program can still go on. I know I can catch an exception of course, but that approach is also useful to me. I don't like when there is too much of stack trace in the logs. Especially when some failures are expected (up to some point).
TIL About the Sized type and all the similar ones about specific objects which implement certain protocols. Very nice!
I very much love the Idea of Iterable and Sized. Tbh I didnt even know those existed.
Thanks for sharing. I got 2 tipps very useful - iterable and raise error - I should use this more. Thanks.
Thanks to you I'm getting better with type hints.
You’re welcome ☺️
Thanks, Arjan! This is a fantastic video, packed with ideas we can use.
Glad it was helpful!
Thanks! This is helpful and encouraging going into a refactor! 🔧❤🙏🏼
Thank you so much. Relevant and very well explained.
You're very welcome!
Very nice! Please make more such type of videos!
I really enjoy your videos. I especially like the code roasting ones as they teach you how to work with unfamiliar code.
One small nitpick. Just as you shouldn't make the parameters types too broad, you should not make the return type too specific. The reason is, that the only changes you can do to the function's types without potentially breaking the code that calls it is to make the parameter types even broader, and the return type even more specific.
A good rule of thumb is to start on the more conservative side with the interfaces and become more liberal once you're confident in what the code does and how it does it.
Hi Ivan, that’s a really interesting point regarding the return type - I hadn’t thought about it in that way. Thanks!
For number 2, I would prefer to see a friendly API called to_uri() that accepts and converts the three different datatypes, then calls a _to_uri() function that handles the logic. Just as easy to test, but way simpler to use.
i think that would just add complexity to your codebase as generally you will know what type you're working with from the start so having a function trying to figure out which type you're working with adds complexity instead of just calling from the type directly, then you have a public accessible bare bones one which just takes the bytes, it would leave the user able to create their own functions too if you make your own object which you want to convert to an uri, and if you need the dynamic approach you can create your own, but they will all just use this one function
I usually use those simple type hint as list, dict, str, int, and may be some class in my code, very rare to use custom structure for type hint.
Anyway, may I request for 1 ep to show example of libraries that people missed out and would save the time for developers.
Thanks anyway for new videos.
Good picture, clear sound 👍
Such finesse!
+1 from me for always giving me new things to learn!
Thank you so much for your work, it's very helpful! Could you please consider a topic of Software Architecture? E.g. building delivery app from a scratch, consider how to design components, their relations, DB, deploy(docker), maybe microservices architecture. It's more like advanced level. It might be very helpful.
Great video. There is a singledispatch decorator in standard library which overloads the function based on type of passed argument.
Good point!
To the question at the end of the video. I found useful the Postel’s law: be conservative in what you send, be liberal in what you accept". So yes, as example i try to accept types like Mapping (which can be dict or defaultdict, UserDict etc.), but return dict to be precise…
Very good video! I totally agree with everything!
Sound advice.... and not just for Pythonists..... I could apply plenty of this to JS, Rust, even PHP.
Neat tips. Thanks. If I may suggest, there's Jeff Delaney who runs the Fireship channel. Try to be a bit like him, to be more concise and just stick to the bare minimum most crucial things. That way instead of havint to create a 17 minute video, you could just tell it all in 5 minutes tops.
Fibonacci number 🤣😂😂🤣
Thank you I needed that laugh!
Very much informative video! Thanks for making such quality videos! 😊
Glad you like them!
Your videos make me happy
Your comment makes me happy! :)
Pretty much the message is to consider the "life cycle" of the product. During the whole life cycle of the product, i.e., conception through EOL, you typically find that 80% of the effort is done in post-production -> versioning to expand features, optimize speed, debugging and/or general maintenance. The remaining 20% is split 10% design, 5% planning and 5% testing. Yes, most of the time you'll be working in a team, so the amount of code per project per person is really not that much.
On the other hand, if you're coding by yourself, then the numbers typically flip -> 80% split between design, planning and testing. The remaining 20% will be complaining about all the features you want to add but can't due to the limited you have (forget about maintenance and testing, takes too much effort).
If you are the latter, better listen to Arjan!
Love this
Thank you, Casey, glad you like it!
This video was a gem! 👏🏻
Thanks man, glad you liked it!
Very good tips, especially about type hints at the end. I've used Iterable before, but never knew about Sized. A disadvantage of Sized is that less seasoned Python developers might find them confusing, so maybe it's a bit overkill in a project where the Python knowledge is lower. I think it's neat though.
Going from list to Iterable enlarges your options, but Iterables have a feeling that they are fundamentally "like lists" (think a tuple), so conceptually this does not differ very much. But going from a dict to Hashable will allow e.g., integers, which are conceptually very different from a hash map. I think that enlarging a type hint to objects that are conceptually very different from the original might make it more unclear what the function really needs the arguments for. Do you have any thoughts on this?
I really like your video.
Just one question about the second mistake. If we raise exception, then we have to handle it at every call to it. Is it the same as we return URI | bool which we have to handle the returned bool value?
Thanks!
Thank you! ❤️
Just a question about the third example. We annotate the return type as list[int] a.k.a. the actual python list type. Now my question: which is considered best practice? to use list[int] or List[int], where uppercase List is imported from typing? Could not find a PEP for it.
Very nice!!
Thanks!
I tink some of the type stuff comes down to YAGNI, but if really depends on your specificscenario.
I always read the type: "List[str]" like "Iterable[str]".
actually never used sized, but used iterable before.
Nice neon!
Thank you! 😎
Hmm, doesn't going to the effort of being more generic than might be required at the time the code is first written go against the YAGNI principle a little bit?
Maybe we should wait until we actually need/want to support more generic types?
Absolutely! I think spending some time on making the types more generic mostly makes sense if the code is not for your own internal use but for others.
can u tell name of extension for importing anything to my code in vscode?
For mistake 2, I'd prefer to have a bland util function which converts int/str/bytes to str. It could be used anywhere in the codebase and easily tested since the usecase is narrow and clearcut. Then you can just call that on the input and hand the output to the original function which now only takes str and only need to be tested for that.
In your Iterable example, wouldn’t the Iterator type be more appropriate? Trying to wrap my head around the difference between the two, but it seems like you’re not guaranteed to be able to iterate over an Iterable, as you’d have to call iter() on them first? Of course, most Iterables are also Iterators.
Hi Arjan!
In VSCode you can use CTRL+D/CMD+D for replacing a recurring sequence of characters. At 4:07 this would have saved you quite a lot of keystrokes :)
Just select the code you want to replace (location.geolocation[0]) and hit CTRL+D/CMD+D one time per extra cursor you want. You can then simply type geolocation to replace the code in all places at the same time.
The VSCode docs have an entire "multiple-selections-multicursor" section covering similar shortcuts and options to save you some valuable keystrokes.
It is an honor to gift this community over 500 of my keystrokes.
Fun reading tip: "Do they deserve your keystrokes" by Scott Hanselman
I do a lot of type hints; especially if I have my own internal object models; I am at the point where I am looking for typehinting and validation for DataFrames;
Some people advise function names to be specific, while others suggest they'll be more concrete. I am confused
For item 2 it is such a shame that Python doesn't have templates as C++ does
What is shellroast-py?
I love python
Da quando anche in italiano? 👀
Father of code
Wait, aren't tip 2 & 3 sounds a bit contradictive? 🤔
I pay a little more than an appropriate amount of time implementing type hints.
Iterable and Sized and such is a concept used ALOT in C#. It becomes second nature but because its an indirection task, it usually confuses newcomers. Although, I prefer to just explain it and have them understand whats going on..
thanks again, specifically for tips nr. 1, 2 and 3 😂
Haha, you're welcome!
"How much attention do you give to typing" -- so much. Too much arguably. I can and do spend all day going over old code and streamlining it and adding typing. My old code was *rough*.
Tip #4: when you walk away you don't hear me say pleeeease, oh baby, don't go
Simple and clean is the the way that you’re making me feel
1. ask for more than you need. 2. not being specific enough. 3. too specific
List comprehension missing in eg 3
Go with the flow, because Clean Code isn't written by someone self anymore.
Off-topic: man, you definitively lost a lot of weight!
Yes, I've been working out and prioritising my health.
thank you!
here is a cat for you : )
/ᐠ.ꞈ.ᐟ\
Thank you too!
the second example seem a good use for functools.singledispatch