Python's 5 Worst Features
Вставка
- Опубліковано 15 тра 2024
- Hello Bob! Today I'm going to be sharing with you 5 of Python's worst features (in my opinion).
▶ Become job-ready with Python:
www.indently.io
▶ Follow me on Instagram:
/ indentlyreels
00:00 Learning Python made simple
00:05 Intro
00:22 Implicit str concatenation
03:35 Else block
08:16 Star imports
12:05 Mutable defaults
15:14 Shallow copy
18:33 What are your thoughts?
I read this title as Python 5 and thought I woke from a coma
Python is learning from iPhone and just skipping numbers that are bad for marketing, like the unlucky number 4 in Japan xD
I thought it said "Python 5," too, and I was initially like, "Wait a minute, what the hell?"
Wait till you hear about Python 95 and Python 98 😂
Yes, David Hilbert said if he were awoken in 100 years, his first question would be -has the Riemann Hypothesis been proved- what version of python is in beta?
commenting to share that was my immediate reason for clicking this video too lol
"It will print nothing because we didn't print anything"
-Python development in a nutshell
Regarding the `else` statement. Raymond Hettinger once mentioned he had proposed renaming it to `nobreak`, but in hadn't been accepted. In any case, I consider it the best Python feature with the worst name.
Agreed
I learned something today.
ooo... I like that.
It's not the worst name, it was inspired by assembly loops, where you have an if(generally a jump too but whatever) block which executes iteratively using jumps and we can kind of use an else here
I agree very much with this sentiment. I've used it in many scenarios where it made sense to use it. The feature is great, but the naming could be better. Cool that "else" makes sense in the context of assembly jumps, but it just doesn't make any sense in the context of Python.
To avoid confusion, you have to think that the "break/else" are working together, which means if there is no "break" statement in the loop then there shouldn't have "else"
The for-else thing caught me off guard. I never used it but I assumed it got triggered only when the body wasn't run, since in most languages the for loop is a while loop with batteries included, and the while loop is an if with a hidden goto. Very, very unexpected behavior!
completely agree, that's the one that's totally unintuitive. It should be for ... then ...
@@francoismolinier6924 this makes a lot more sense!
I don't mind "else" with "try" since it would naturally follow an "except".
Unless they've changed this behavior, you _can't_ have it without an "except" even though you can have a "try" without an "except" ("try . . . finally"). Thus, it's really "except . . . else", because either "except" or else "else".
@@MAlanThomasII Exactly. So it actually makes sense in that context.
It makes sense to me because i think of except as "if exception." I never knew it worked with loops though, that behavior is weird.
@@Fanta666 This. The alternative is `except` being replaced by `if except`, though the suggested alternate syntax of "noexcept"/"nobreak" is also an agreeable compromise.
@@MAlanThomasII What would you use "try...else" for? If this were valid syntax, I would just remove it because there's no difference between "try: print(1) else: print(2)" and "print(1); print(2)". Don't be silly.
The real issue with `import *` is not shadowing in the way you showed, because you can understand that kind of shadowing statically from your environment.
The real issue is actually that you may be deploying your code in an environment where each module has different versions, and if they are using semantic versioning then *adding a new feature* to those modules only bumps the *minor* version, which is assumed to always be backwards compatible. If anything changes that is not backwards compatible, the module would have bumped the *major* version instead, and package managers on the deployment end will use this standard to automatically get the most up-to-date but still compatible version of the dependent modules.
However, if you use `import *` then this new feature will be imported into your program, possibly shadowing part of another module that you could not have possibly known about at the time you wrote the code, which turns industry standard backwards compatible updates into automatic code breakage!
Sometime you’d use a for loop to go through some data looking for a feature, then if you don’t find it you’d exhaust the loop and drop to the else block. In that case it’s not success, it’s failure.
In case someone still doesn't know, R language does not have the import as syntax at all, so functions from various libraries often override each other. You often need to use syntax like base::mean(), which means using the mean function from the base library.
The dummies behind tidyverse have created some tools, like forcing users to explicitly specify the library origin for each function when namespace conflicts are detected. It's just replacing one nightmare with another nightmare.
R is used a lot in my field. I do my best to avoid it like the plague.
Else block has imo fairly solid intuition: You often loop things to find something. Once you find it, you'd break out of the loop, and be happy.
However, sometimes you don't find what you were looking for, so you now have to do something... else.
With exceptions likewise, the intuition seems clear enough, you expect an exception of some sort... But if you don't get that exception? You do something else.
I find it a bit underused syntax tho, and as such, maybe it should be removed. But it's very helpful syntax for many common use cases.
Personally In day to day code; If I have a try...except block I'm NOT expecting a exception to happen, I PREPARE my code to RESPOND to something in the case it happens, but sure enough I'm not expecting it to happen, because the proper and complete execution of the code depends on the program not triggering an exception (I guess there are special task that the normal execution path expects or requires an exception to be triggered... I would never write code like that tho); but depending on the exception to execute to achieve a task seems counterintuitive on 99.99% of tasks. It's like paying insurance: You don't pay because you expect to crash your car, you pay just to be able to handle the UNEXPECTED event of a crash... Just thinking on having to debug code, that depends on a exception to happen to achieve the normal program flow gives me a headache. They are called "exceptions" for a reason: They respond to EXCEPTIONAL conditions in the program, not to the normal and desired conditions. I LIKE the else in the try block, as it allows to handle the correct execution path in a very elegant way, but NOT to respond if there where not an exception that I was expecting for. That's extremely counterintuitive.
@@setsunaes For loop iteration for example depends on the iterable sending "loop over" exception. As an example of expected exception
I don’t think it’s fair to say Python’s string literal juxtaposition causes concatenation is “poorly thought out”, because this was a feature of C. In C, it made more sense in the context of macros and automatically generated code. And Python has borrowed a lot of other syntax from C, so at the time *not* having this feature would’ve been more conspicuous.
C doesn't have a string concatenation operator, Python does. Python breaks with tons of C traditions (it's one of very few who put bit operators &, | above comparisons ==, > etc in the priority table!) - and it has a philosophy of 'one correct way', so making the + concatenation optional goes against its core values.
@@sharpfangThe "one correct way" has been broken many times; it is merely a preference. It is in no way to be followed.
PEP 584 directly addresses this.
"In practice, this preference for “only one way” is frequently violated in Python....We should not be too strict about rejecting useful functionality because it violates “only one way”."
The language "violates" the philosophy when there's a good functionality, multiple times throughout its history.
@@moho472 Except this functionality generates very hard to catch bugs, so it's very arguable if it's a good functionality.
@@sharpfang That could be said for every single language, and is not unique to Python.
I do agree, implicit string concat is just unnecessary, and it kindof forces one to use a linter to catch stray cases like strings inside a list, for the expense of code looking a bit nicer in some cases.
This topic is very close to my heart. I love Python as a programming language but I have faced these issues. Since I code in multiple languages, I have been gravitating more towards syntactically rigid languages.
Way back when python first started catching on, there were some variants which added back in (optional) typing, blocks denoted by curly braces, ect. I liked that. But alas, most folks didn't...
The lack of strictness is a bit of a tradeoff between ease for small stuff and scripts, and making it harder for large/complicated things. However, the real brilliance of python IMO is being able to fairly easily include lower level C and C++ code as modules. It also beats the hell out of perl
@@travcollier Python is almost as bad as Excel in that people keep abusing it for stuff it was never intended to do.
What?! No mention of package dependency management? :)
Big hater of implicit string concatenation - recently caused a large amount of calculations to silently not run for me
My most terrible feature is that a += b and a = a + b can have different semantics in Python if a and b are objects.
Such as builtin arrays
One thing close to copies is when you try to initialize a 2d array like this: a = [[0]*5]*5, it wouldn't do a proper 2d array (an array with multiple different arrays in it), but an array with multiple references of the same array, so if you were to go a[0][0] = 1, it would change the first elements in all of the rows, not only the first one
This made me screw up a leetcode problem
Oh dear, I did not know that... although I think the only time I ever used that, was when I was creating a numpy array, which I'm pretty sure creates a deepcopy.
Luckily the list comprehension for this isn’t too hard; it’s just [[0]*5 for _ in range(5)]
this is good, because lists aren't arrays, and you should not be using them as arrays. Use an array, otherwise you are violating POLA.
@@DrDeuteron well, what are arrays in python?
The real problem with “try … except Exception” is that python does not document what exceptions a function can raise, which encourages the use of Exception… 😢
Yes. The exception may occurre at a really deep level.
In the Rust language, when a function is able to error, it returns an enum Result with Ok(value) or Err(Error). Yes, enums have values inside in rust.
@@denizsincar29 That's because what Rust calls an "enum", languages with saner naming conventions would call a "sum type". Calling them enums is _really weird._
And yeah on the main topic, catching Exception is _good practice._ What's the alternative, just allow your program to blow up when it comes across an exception type you didn't anticipate? Exception is a base class of the other exceptions for a very good reason.
@@isodoubIet If there's an exception of type you didn't anticipate, it seems the only sane way to handle it is to allow it to blow up the program.
@@gJonii And then you'll never find out because it won't be logged, your customers will call asking why the service is down, and you'll have no idea why. "Allow it to blow up the program" is never acceptable.
@@isodoubIet This is not necessarily true. Unless the exception is expected and can be handled in some way (maybe how it should be handled is logged and forcing the user to redo the previous step), allowing a program to continue in an invalid state that caused the exception in the first place can lead to problems like security leaks, bugs, etc. In many cases, it is better for a service to be down and fixed rather than broken and running.
I know when you're using default mutables for a dataclass it requires you to use a function that returns the mutable to get around this, would that work in an ordinary function call as well? I don't think it's any easier to read than the boilerplate you have, but it would be a different way of doing it
The mutable default is the closest we got to C's static variables inside functions, so i think they are a valuable tool, despite being limited to only lists and dictionaries (no, global variables don't count as they can be accessed from anywhere).
Shallow copies are spain without the p
sain?
A trip abroad where you aren't allowed to use the toilet?
without the “s”?
obviously a programmer as they made an off-by-one error
@@ShunyValdez shallow copies are:
>>>func = functools,.partial(filter, 's'.__ne__)
>>>"".join(*func(''Spain'.casefold()))
'pain'
is safer. Why index?
You see, try, except, else works for me.
I would agree with you however that in the case of 'for' and 'while', it does seem unintuitive... but, hey, at least I learnt something more about looping!
😊
the 'else' in 'for' and 'while' I would expect means 'if there were no elements reached by the loop' which is nearly the opposite of what it actually means
@@MagicGonadsThe idea is that you'd often loop to find some particular element. If you find it, you break out of the loop and continue from there. But if you reach the end of the iterator... Well, now, you need to do something else. This something else in case of this failure, you'd put in the else block, knowing it's only ran if you failed to break out of the loop.
@@gJonii but semantically 'for all of these things, otherwise this' is what a construct 'for-else' would mean intuitively, is what I'm saying. For loops may often be a search, but not every for loop is a search.
Yeah, its very confusing in the for and while loops, and even for the try/except i feel like its not even worth it. A "nobreak" or even a good old "then" would make it much clearer
But the whole thing could be much less ambiguous by explicitly setting a boolean variable (e.g. found, error, etc) before the loop and changing that variable in the same line as the break/exception, then using an if after the loop to explicitly run some code if the variable was changed.
You don't need a new keyword for every possible scenario, or else we'll end up with a "noop" keyword for when the loop is iterating over an empty list or something
@@sutirk yeah my interpretation of how 'else' would work would also be called 'empty' (the case in which the iterator is empty) and often you just handle this explicitly
the else block is like the exact opposite of what you would think
it doesn't even make sense compared to how it works with if
if anything it should run only when broken out
i think it shouldve been named 'also' block
The use case presented for it is element search. You loop over an iterator, searching for some element. If you find it, you'd have "if element == target: do stuff; break"
But now you'd write code after the loop. Can you trust you've found the element? Perhaps not. Perhaps your loop just ended naturally, and your cool break logic never ran. What to do then? How would you even know that happened? Enter else-block. It's only ran in this scenario, so you know your break-logic was never ran.
You'd have absolutely no benefit from this also-block that runs if broken out from loop, since you could put this logic manually to the "if condition: break" section for much more readability.
FYI if you want to run code when a for loop is broken, the way you would do that is to put the code before the break. Something like:
for x in xs:
if x is None:
print("Got unexpected value, breaking loop")
break
else:
print("Processed all values successfully")
You can also kind of see how it DOES make sense with the if. In this example, which is how for...else is usually used, the "else" only runs if the "if" never runs. In expanded form, the above code translates to something like this:
if xs[0] is None:
...
elif xs[1] is None:
...
elif xs[2] is None:
...
else:
print("Processed all values successfully")
@@gJoniithat makes sense but it’s weird to me that python cares about this very niche use case but doesn’t have named breaks to allow breaking out of multiple nested loops. Rust lets you break out to any scope you want by name and even “return” a value with your break statement which can be used to solve this problem too.
I mean I get it, python is much older and is full of tons of design decisions that we wouldn’t choose again knowing what we know now. But it’s just a bit frustrating when a “low level” language lets me often write higher level code than a “high level” language.
The else in try block makes sense to me as I've always understood it as "(if) except: ... else (no exception: ...
The else in the loops is less intuitive to me. It seemed to me like it should run if there were no iterations at all
Yeah I agree, the for-else blocks require you to think of a for loop as a series of checks for which "breaking" is the sign you've found what you're looking for. Most people are taught that loops are for doing something, and break is for when you want to stop doing that thing early...which is kind of conceptually the opposite.
I don't read it as applying if there were no interactions at all though, I read it as "if any of them failed (had a break)"
it does run the else block if the loop had no iterations. It's useful if say you count successes:
for count, item in enumerate(container, start=1):
break
else:
count = 0
print(f"Found {count} items)
w/o the else block, you get a NameError, and to prevent that you would need to predefine count=0, which is U G L Y, and unpythonic.
Great video, thank you. I am mainly a C# and JS dev but like to see what other languages share/do differently.
What about else with 0 runs? So while (false) else print('yes') or no?
Why would you use "else" in a real example? Just to avoid checking count/state afterwards like if count < length / isAllGood = false?
I thought that you goinng to say, that else is worst feachure bc you can mistakenly make else not for if, but for for,
like:
for i in range(10):
if i == 5:
print(five)
-else:-
-print(i)-
_else:_
_print(i)_
and you get an error
Messing up your nesting is just a skill issue
@@SonOfMeme It's always baffled me how people will complain about the whitespace in Python, but then if you don't use whitespace "properly" in their language of choice they bitch about it.
Vestigial semicolons and meaningless whitespace... why?
I am starting to learn Python and even if the subjects are more relevant to people who already have mastery of it, I found it very interesting to follow the video (by reproducing the examples, because I learn better by doing it even if it's shown, that way I can test a little more)
And I already liked seeing certain practices often seen in tutorials which could go against the good practices that you mentioned and therefore avoid getting into bad habits and in addition I learned some things with the video that I I don't know enough about it yet but it will probably be useful to me one day.
Dude got some serious issues with Bob 🤔
I personally use for else in my code, but i you are also right, it doesn't justify for what it actually means. I used to use from module import * but then i got to know the importance and i don't use it. And btw i never knew the difference between shallow copy and deepcofee until i watched this video 😅
because of this video I've just realized that I might have a mutable default problem in one of my private libraries. Thanks!
I think the most irritating part about else block is that for "if" statement it means that "if" *did not* work
If I make a shallow copy, is there any way to display the list that displays the references so that I _know_ I'm dealing with a shallow copy? (I figure this might be useful in debugging.)
map everything into `id` if it's not a primitive
Hey, fun video! What editor are you using? It looks like VSCode but not quite... I'm really curious.
It's PyCharm from Jetbrains
List comprehension usage would be useful for a lot of these vs what you do like in mutable example . Or using inline defaults
Time and performance gains too
Thanks for helping us on these😊
An addition to the star imports:
Not using star imports also benefits to the speed and the file size of your application. If you use a big library like PySide6 (for creating GUIs) and you import everything, your compiled app will be roundabout 200-300 megabytes. If you only use the Widgets, Gui and Core (which most applications do), then you will end up with like 20 megabytes and a MUCH better startup time AND in addition to that it also helps your IDE, as it doesn't have to index dozens of docstrings and functions.
But if you only use small libraries like colorama it doesn't really matter, but still a good habbit to not do star imports :)
Thanks for the useful video; I learned some things. For implicit string concatenation, that comes in very handy when needing to do deeper escaping of strings containing a mix of single and double quotes. Used right, it's a huge help. I agree that used wrong, it's a mess. Maybe best to not use it unless it's actually needed. Great video overall, though. Thanks!
Excellent video! Very useful. 🤯🔥👋
import * and mutable defaults are both caught by pylint, at least. But yeah these aspects of Python all have sharp corners.
Also, when did the | syntax for type hints show up? I use typing.Optional and typing.Union since I wasn't aware of that bit of syntax sugar.
The last feature is quite expected. After all, a list in the memory of a computer is just a pointer to a memory address :)
Actually I mostly write code in php and its copy-on-write behavior was confusing me for a long time in the past.
I think shallow copies make more sense than you'd realize. Generally speaking you want to copy the least ammount of memory possible and be very explicit over deep copies.
It's not that it makes no sense, its just that it makes no sense to call the function copy when it's just creating a reference to some data and not a copy of it.
@@craftylord3336
It kinda makes sense when you realize that all these languages work with reference types and primitive types. A reference-type is just a pointer to the real data, so when you copy the shallow copy is justified by this same reason (the pointer is copied).
You generally don't want deep copies when you have this kind of reference-type structure as it would blow your memory and also your GC, and because most of the time what do you need is a shallow copy.
@@diadetediotedio6918 No it still makes no sense. By that reasoning when ever you use = to set one value (int, string, bool, etc) to another it should also be a reference.
Or strings should be returning and/or require ascii/Unicode values by default then.
It's also basic linguistics. A function called copy should create a copy of specified data and a function called pointer should create the pointer to specified data.
And how much you "blow your memory and also your GC" is up to the programmers abilities, the tools at their disposal and the task at hand.
Python making it difficult/confusing to copy lists is just poor design.
@@craftylord3336
You use = to set these types that are what I so called primitive types. And even if they were not primitives it would still make sense because their memory layout is entirely flat.
When you write the number 100 the number is fixed and cannot be changed, every single bit required for it's identity (which is value-based) is already here, so you use = to set it to another number (the same for the other ones).
Also, about the function called copy vs function called "pointer" I don't get it, no single language has a function called "pointer" because it does not make sense, a pointer is just a reference to an object somewhere in memory, underlying it is a number (like an 'int') so when you call 'copy' you are really copying everything that is flat there (including this int), just not the pointed object itself which is on another place in memory.
If you want a language that does deep copies of your lists use some that don't have reference types like C, C++ or Rust, most modern languages that have references (including C#, Kotlin, Java, Dart, JS and Python) suffer from the same thing you called a "poor design" that in reality just makes sense if you think about it for a minute.
@@craftylord3336= does copy the reference, not the object. Test it with the `is` operator!
I disagree with the shallow copy vs deep copy part. That would be really unexpected coming from any other GCed language. How does it even handle ref-loops? Haven't tried Python's deep copy, am on a phone, but does it crash, fill up memory, somehow track all objects and try to recreate the loops? How does deep copy handle custom objects? There's no one way to copy objects in Python, there is no copy interface/protocol last time I checked. Seems like a deep copy function would be full of hacks.
Regarding splitting a long string over mutliple lines, i was gonna say i would write this in a triple quoted string and then unfold it with some stdlib tools. But when i looked for such tools i realized that neither textwrap nor shlex have them ready to go. You would have to do it in two steps (like textwrap.dedent().replace("
", "") ) or use a full on regular expression. Or i guess you could make a proper tuple of your string lines and use str.join(). There's a lot of option, but they all seem like a lot (ish) of work just to make your source code prettier :)
Your videos are helping me learn and giving me also idea how to present my videos
I mean... For try/except/else, the else is after the exceptions. So it's like, "else, if there are no exceptions, do this". Makes sense to me.
For for loops, it makes even more sense. You break out of the loop because you are done with whatever you were doing. If you reach the end of the loop, you're probably not done, so you fall through into the else block.
Best example is searching for a particular element. Once you find it, you break out of the loop. If you reach the end, you didn't find it.
Calling the else branch of the for or while loops a success is somewhat questionable. One could consider the break statement execution to be a success instead, actually. For example, it might mean we found something we looked for.
I agree with you when it comes to "else" blocks on loops, but try/except/else actually makes perfect sense to me (first try this, then if there is an exception do this, otherwise do this), and it can be very useful, especially in cases where you want to do something with a result that won't be there in the exceptional case, without accidentally catching errors you didn't mean to. I'd even go so far as to say that if you find yourself writing more than a single function call inside a "try" block, you should consider whether some of it should go in an associated "else" block instead.
So you'd rather cleanup fails in your code because you didn't realize that some library that used another library that used another library didn't document one of the errors that can be raised deeper in the call stack, right?
At least catch an Exception and log if the type of exception is unknown, unless you're absolutely sure the function raises only what you know it raises.
Or unless your code doesn't have any important cleanup to do.
to the star imports:
Namespaces are one honking great idea -- let's do more of those! - the zen of python
Absolutely valid criticisms for the string concatenation and shallow copies.
There is also a problem when you're trying to make a list with multiple copies of the same thing (ie. lista = [[item1], [item2]] # if you do [[item1] * 2, [item2]] * 2 and then try to adjust item 1, it will adjust all of the first elements of all of the copies of lista. There is a way around it, but to find out the easy solution you have to go to the Q&A section of the documentation -_-
I disagree on you star imports point, if you are making a function that is already defined... I feel like you're setting yourself up for failure! Why would you do that?!
But yeah, good video.
In the Ruby programming language, the role of the "else" keyword, as described in the second section, is performed by the "ensure" keyword. I think it's a much better name. It's also slightly different, because the "ensure" code block is always executed.
regarding star imports: pylint will throw various warnings at you about them, such as to not use them in general, but it also has a lint for star imports shadowing each other.
pylint, with a lot of configuration about which issues you care about, is actually a very nice tool to use in any project. there really isn't an alternative for it yet.
The shallow- vs deepcopy is new to me. What would be a useful use case for a shallowcopy?
Mostly to drive people away from the language.
You'd almost always want to use a shallow copy on a list containing immutable data. Like a list of strings:
A = ["1", "2", "3"]
B = a.copy()
B[1] = "c"
print(A) # ["1", "2", "3"]
print(B) # ["1", "c", "3"]
Strings are immutable in Python, so you never have to worry about the pitfalls of modifications to B propagating to A. This means that A and B require less memory to store than if B deep copied A, because they both have the same references to elements 0 and 2. So only 2 new objects have to be created (B and "c"). A deep copy would require 5 new objects be created (B, "1", "2", "3", and "c").
When you want the elements in the list to be reference identical. Perhaps they're being used as dict keys, or will share mutations.
Nice video !
Usually i put inside the try: some lines to be executed after de dangerous code, if nothing triggers an exception that code will execute, otherwise it won't... so i don't really get the purpose of the else: at all...
If the line you thought was dangerous doesn't throw, but a line afterwards does, you may be catching different behavior than you expect. So using the else can help you be precise about what you know how to catch and handle vs what should propagate an error
Sometimes, for-else block is very useful!
he agreed, his only 0xDEADBEEF was the name in the for/while context. I do love the construction, and have no problem with the name...for me it's "BREAK or ELSE"....
Good list learnt something new today.
1.) Didn't know this, can't really see a use for it and can see how that would be annoying.
2.) Didn't know this either, could be useful.
3.) Did know this, but never use star imports personally.
4,) Bit by this before, when my editor didn't warn me. I spent hours trying to figure out why something wasn't working.
5.) Come up against this before but don't think it's too bad.
What about global and nonlocal?
Which developer’s version of Python do you recommend ?
Which version has the fewest inherent 🐞 🐛 🐜 bugs ?
Another alternative for the mutable default list, is create a function that returns an object of the given type (in this example, called "new").
def func(target: list[str] = new(list[str]))
dunno if this is a good idea, but one that came to mind nevertheless
Doesn't fix this particular problem since new is called only once at function definition.
String concatenation is very useful, especially when creating error descriptions or string templates for complex terminal interactions, a good formatter usually is enough to detect those problems.
"Explicit is better than implicit." literally, from python zen, lol
@@viktor67990 python is literally cluttered with implicit features, this doesn't mean we must not use them. The string implicit concatenation is useful in some contexts, like the ones I mentioned, and it's also performed at compile time, so it's more efficient than joining string constants at runtime.
triple quote strings though
@@viktor67990 The Zen of python is like literally a list of things Python designers decided _not_ to do.
Totally agree on the else for the reason; if you don't go into the if you go into the else. So it would make intuitive sense if you don't go into the loop block you go into the else rather than its present logic. And there is far more cases where it would be useful to use else if you cant loop, rather than if you can.
I always import the entire module/package instead of importing single functions, is this bad practice? I prefer to access copy.deepcopy() than to access it as deepcopy() because if someone is reading or glancing at the code they will think deepcopy is an independent module/package
It was a very informative and helpful video, I just feel like the 'else' is actually intuitive, and that it is one of the things that any programmer should read the docs about anyway, besides that I have the same opinions.
the else block for while is kinda weird.. the else block for the "for" loop is extremely unintuitive, since I would expect "for i in items .... else:
6:13 f.. didn't know about this else block.
Xd, this is something I didn't expected
I'm totally with you with the first 4 features, but the last one do you have in any language i know, because of the reference type of the nested list (or to be clearer in python because of the mutable type, because in the end everthing is a reference type in python). Therefore, copy behave as expected in my opinion. What would be nice on the other hand, an additional deepcopy method for example.
Doesn't work that way in C++.
@@isodoubIetC++ is a lower level language where you are usually preoccupied with memory management and performance. In higher level and usually interpreted languages it's much more common to see pass-by-reference as the default, at least for object types. That would include JavaScript and consorts, PHP, Ruby, C#, Java,... Problem is, it always comes with an overhead, usually either reference counting, garbage collection or both, because you have to keep track of where the object is still needed or not. That's not an acceptable tradeoff for a systems level language like C++ or Rust, but you can always implement your own if you so desire.
@@eldonad It has nothing to do with C++'s focus on performance. It's just a conscious design choice based on the idea that it's much easier to reason about programs where your objects behave just as the built-in types.
@@isodoubIet Ok, I've thought about it for a bit, and I can imagine a weird version of C++ where objects are passed by reference by default, so I stand corrected. However I still think passing by value as a default is more natural in runtimes with unmanaged memory, since in that case specifying the flavour of reference you use can provide you with information you wouldn't care about in a garbage collected runtime. But eh, at the end of the day every language is kind of pass by value at heart, only that the value can be a magic handle to an object, or a shared_ptr...
Also star import: you import anything that was imported in the imported module. I saw logging being imported and used this way. Definitely agree this is bad feature.
Thanks for a great video. Coming to Python from Julia I can tell that most of these problems have better solutions there.
My least favorite thing in python is the bytes() constructor because it has one notable inconsistency with the str() constructor that is inconsistent with the other constructors in the same space. Here's an example:
A = "1"
int(A) # => the integer 1
str(int(A)) # Now we've roundtripped back to the string "1"
A = "1"
bytes(int(A)) # this is b"\x00", ie the null byte.
Unlike the str() constructor which turns an integer into a decimal string representation of the number, the bytes() constructor creates a byte string with as many null bytes as the integer specified.
bytes() is explicitly made to work with ASCII text, why would you pass in an int?
I assume that passing an int works as a handy way to get x number of NUL bytes because otherwise it would be incredibly ambiguous.
In your case, should bytes(int("1")) be parsed as 1 in hex (\x01) or as the string "1" (\x31)?
What if we pass in bytes(int("111"))? Do we expect it to give us the character "o" (\x6f) or the character "1" three times (\x31\x31\x31)?
I guess you can see how it would be useless either way because you're either limited by only outputting the bytes 1-9 over and over again; or your input would have to be made of a concatenated mess of a bunch of decimal values for characters making a truly meaningless int, and which would be even more ambiguous to parse if you consider multiple characters, and then extended ascii and encodings like UTF-8...
Your first question can be answered with a similar question: str() is explicitly made to work with abstract text, why would you pass in an int?
bytes(int("1")) => b"1"
bytes(int("111")) => b"111"
Rationale:
int(b"1") => 1
int(b"111") => 111
For non-ascii::
int(u"一") => ValueError, only characters 0-9 are recognized so bytes doesn't have to handle that either.
My view on this in general, which should address your arguments that I did not explicitly mention above: mapping an int to bytes *is* ambiguous but it is the same amount of ambiguity as mapping an int to a str and mapping bytes to ints. The decision as to which of the possible outcomes Python will use for those values has been made. So for roundtripping with int and symmetry with str(), bytes() should have been implemented with the same choice.
does the shallow copy method not do the exact same thing as just setting a_copy = a? Since in both cases the variables both point to the exact same data if I'm understanding correctly. So then why is there an explicit copy method when it does the same as just assignment, you would expect it to do more than that, such as actually deep copy
What I'm understanding is that list.copy() DOESNT fully copy the nested list. It only fully copies the outer layer elements, and creates a reference to the inner list (which comes from the same memory location as the original list) and thus, editing the nested list of the copy created with the list.copy() method would actually be altering the original nested list that is referenced in the copies list.
@@valerielboss yes, so a_copy[0] *= 2 will also change a[0]. One way to test is evaluate the boolean:
>>>a is a_copy
False
which compares the id()'s.
So in his 1st example:
>>>a_copy == a, a_copy[1] is a[1]
(True, True)
while:
>>>a_copy is a
False.
(but I didn't test those in an interpreter, but all pyhtonistas should try it out to learn it).
It gets easier to understand the behavior if you understand memory and pointers/references.
In his example, you have
a = [1, ["a", "b"], 2]
What that actually means is (in some very generic pseudocode notation):
mem0 = ["a", "b"]
pointer0 = address(mem0)
mem1 = [1, pointer0, 2]
a = address(mem1)
So what "a_copy = a" does is that both variables point to the same memory address:
a_copy = address(mem1)
In this case both a and a_copy are exactly the same, they point to the same memory and have the same content.
The shallow copy "a_copy = a.copy()" copies that region of memory along with every *value* in it:
mem2 = [1, pointer0, 2]
a_copy = address(mem2)
In this case, a and a_copy have exactly the same content, but one of its values is a pointer to another object, so both a and a_copy share this object, and when this object changes it will affect both a and a_copy.
Finally, the deep copy "a_copy = deepcopy(a)" actually goes into every pointer and copies that memory region too, saving the new memory addresses in the list:
mem3 = ["a", "b"]
pointer3 = address(mem3)
mem4 = [1, pointer3, 2]
a_copy = address(mem4)
In this case, a and a_copy have the exact same values, but are completely independent, whatever you change in one will never affect the other, because a references pointer0, whilst a_copy references pointer3.
Calling the else block "success block" I consider confusing, as I would use it specifically in case of failure. For example if using a loop to find something, the else block will be executed if it was not found.
What is entirely pointless, however, is an else block on try. Just put those statements at the end of the try block.
On the impot * the problem isn't the ability to use that, but the fact that Python silently overwrites the value from the previously imported module. Instead you should get an error when trying to access the ambiguous name unqualified.
On string concatenation, I think the Python developers got spoilt by C which also does that.
14:32 the if statement creates unnecessary branching, which could make the function run slower. a better way to do it is `target = target || [];`.
forget about speed, just reducing cyclomatic complexity is a win.
I also don't like how enums work, the fact that you have to call auto() for each of them when in 90% of applications you are going to use auto anyway. Why not make that the default, while still allowing overwriting with a value in the rare cases you need a specific value.
Also having to import packages for such basic features always seemed a bit hacky. Almost like its not part of python but you had to rely on someone else's implementation.
Regarding the mutable defaults, when you fixed it why did you write “target | None = None”? Inside the function you checked to see if target was None anyway, so why not just make it “target = None”?
It's the appropriate type annotation according to the docs.
I love the first feature (using parentheses) you just need to be careful, but it declutters the code so much!!! Also a feature of C/C++ if you are wondering from where that came.
I’ve written thousands of lines of Python over 10ish years using it, and have never encountered that missing comma issue.
I'm almost ready to bet, for the people that did encounter it, that they probably didn't notice it. It's not something that messes up your code as much as the user's experience when they read those typos.
But I am curious to hear if anyone did experience a major bug because of this?
@@Indently 2 years in programming. Sometimes I make this mistake myself. Although I always manage to notice it before running the code. But I can easily imagine that it will be difficult for others to notice this mistake. In general, it often happens if I edit an existing block of code. Usually this doesn’t happen to me if I’m writing code from scratch
I think by its very nature, the bugs will be minor. If you are dealing with a list of strings, it's more likely to be loaded dynamically and bypass this language behavior entirely. If it's not, then it should be caught quickly by developer testing or it's going to crop up as a small formatting error with minimal impact to the program's behavior.
I've written hundreds of lines of python over 10ish weeks, and at least twice f'd by that.
@@Indently I did encounter a serious bug in C with the same implicit concatenation
I had a big enum for each error case in my program. In my main function, I would get the final status code, and if it wasn’t a success, it would simply do printf(ERROR_TEXTS[ERROR_CODE]).
This works only because my ERROR_CODE enum has the same number of values as my array of error strings.
Now guess what would happen if, by mistake, you forgot a comma a the 7th element ?
Well the error codes 7 now prints 2 errors, everything above error 7 prints the wrong thing, and the last error code just prints garbage (again, C, not python)
I guess the problem with the "success" case else is the indentation typo with if block. It can happen easily when you copy and paste from other parts of the code. else in if..else and else in for...else mean two completely different things, so they should be named differently.
Shallow copy is a common thing across programming languages, and makes sense, but default values being created statically for each function is unforgiveable.
How many time have people needed to check whether a loop has reached its natural conclusion? The else clause to a loop is in principle a great idea! It's just that using the else keyword because it's already there is a lazy implementation of this feature that can cause confusion. What if there is an if statement in your for loop?
I only create basic scripts that help me with work, but I have come across the deepcopy issue myself. (Work not related to programming)
The last one caused me hours of pain over multiple days.
The deepcopy can yield unexpected behaviors when it acts on objects without recursive memory calls, which may be another reason for why it's not default. I don't know what the cause of these unexpected behaviors are, but I've run into situations where performing a deepcopy on an object makes it unusable, while performing a shallow copy works perfectly. My guess is that more complex integrated objects are more likely to have internal parameters that you don't want to copy, and so are more likely to want to be shallowly copied instead of deepcopied.
From a Python beginner:
• Are there any benefits of using deepcopy vs a_copy = a[:]?
• There’s no need to import when using a[:].
• Could this syntax be a fairly new addition?
a[:] also returns a shallow copy
@@IndentlyAs said, I’m a beginner, but would there be any benefit in the supplied example (17:41)? The contained list - [‘a’, ‘b’] - is hard coded. I understand that there would have been a difference if the list in the variable “a” were to have contained another list variable.
Example:
a = [1, 2]
b = [a, 3]
b_copy = b[:]
Then b_copy would, in my understanding, be affected by changes in a, but not by changes in b, nor b be affected by changes in b_copy.
By the way, thanks for your informative videos.
That's correct. The slice operator creates a new object with shallow copies of the objects in the sublist you picked. Shallow copies of basic data types are just a new copy of the data. Shallow copies of complex data types are not (Technically it's a new copy of the pointer pointing to the object, but then I'd have to talk about pointers).
@@bjorn_ It depends on what the type is of the value you're operating on in any given list.
When you make a shallow copy of a list, you create a new list containing references to the same elements held by the original list. This means that if the original list contains primitive types (like integers or strings), they appear to be copied. But in reality, the new list simply points to the same memory locations. If the original list contains mutable objects (like lists or dictionaries), these are not copied; both the original and copied list refer to the same objects. So, if you modify a mutable object in one list, the change is reflected in the other.
On the other hand, when you make a deep copy of a list, you create a new list and also create new copies of every item contained in the original list. This includes creating copies of all mutable objects. So, if you modify an object in one list, it does not affect the other list.
Here's an example:
from copy import deepcopy
# Original list
a = [1, 2]
b = [a, 3]
# Shallow copy
b_copy = b[:]
b_copy[0][0] = 'x'
print(a) # Output: ['x', 2]
# Deep copy
a = [1, 2]
b = [a, 3]
b_deep_copy = deepcopy(b)
b_deep_copy[0][0] = 'x'
print(a) # Output: [1, 2]
What is your addons list?
This all looks like problems with dev environment not language itself. It would be 'nice' to have warnings for these (in vs code, maybe someone can program a plugin for it, or already did?), but then again, all these features work in a way one would expect them to.
- string concatenation - somewhat useful I guess. Would be nice if it worked with variables: 'some text ' variable ' more text'.
- the 'else:' should probably be called 'finally:'. The 'break' skipping that is reasonable, as it terminated the loop.
- star imports - well, it does exactly what you asked it to. Same thing happens in javascript mixins, last one wins.
- mutable defaults - I hoped to see some 'overwrite core language values' stuff here. It was fun to replace console.log in js.
- copying - The problem exists in many languages, to the point of some of them (c#) not containing any built-in copy interfaces because of possible misunderstanding between shallow/deep copy.
In R, star imports are pretty much the defacto standard. However, you get a warning when a library masks a function from another imported library
I would love if there was a way in Python to get an error message for functions that are overriding others in the same script. Usually we get some squiggly lines if they're re-declared in the same file, but would be nice if during the reading of the script, Python would say "Warning: the following functions were re-declared during execution" or something.
I think the worst feature is the copy, maybe is made it with shallow copy because Python itself is heavy. But I think they should change to specific copy like a.shallow_copy() instead of only copy method.
it mentions coffee, it gets a thumps up. simple
On the uses of `else`, i think the uses in `while` and `for` are better than the use in `try`.
In the case of `while`, a condition gets evaluated to `False` and because of that, the else block run. It's unusual to have a structure retry the same condition over and over until it turns false but that the idea of a loop and `else` prividing code to execute then is a bit of a stretch but mostly fit.
`for` is a `while` loop tied to an iterator so the same logic applies.
`try` is different because the condition that would have to be false for the fallback analogie to works would be having the try block raise an exception.
IMHO, you tend to look as the code execution as the "normal" path and an exception as being unsual. Having the "normal" path tied to the `else` keyword feels like a double negative (if not ok: except(); else: success()) and these tend to be awkward to work with.
Also, in languages allowing valued break (giving a value to a loop construct), the else block can provide a fallback value which is also its behavior in a `if ... else ...` in such languages.
(e.g. if a loop gets a value from its `break` but no `break` gets triggers during execution, the `else` can provide a fallback value and avoid not having a value)
I think you can target=list() for mutable defaults
Implicit string concatenation would probably make more sense if used with variables
damn, that target = [] "feature" gotta be the dumbest thing ever
counter point:
funny enough when i taught my ex programming she intuitively guessed the else feature for for loops. i couldnt understand why her code run correctly and ended up finding it in the docs
most of us that already program find it unintuitive because no other language does it. but i here its us who cannot think outside of the box and the feature imho actually in wonderful because you actually have to do this quite often and i always hated flag invalidating loops when i had to write them in any language
There is a great linter called Ruff. It's a combination of all popular flake8 etc. and it's really fast because written in rust. And it warns the import *
I only agree with two of your features being bad (import * which the documentation notes is mostly for trying things out at the REPL rather than for using in scripts [although, I have another valid use case for this...]) and mutable defaults.
It would be nice to go into why miracle defaults behave the way they do... I don't think it is so much of a "feature" as a product of semantics of the language. When the function is created, its function definition is processed and the defaults specified are created. This is why that same container type is used every single time the function is called. Knowing why this happens can help you remember to avoid it ;-)
The use case for import * is niche: when you are creating a wrapper around another module, import * is the most robust way to ensure your wrapper handles whatever you are wrapping, now and into the future. These types of wrappers are especially useful when writing code that will run on multiple versions of python. An example from my distant past:
try:
# modern python
from json import *
except ImportError:
# old python that doesn't have the json module in the stdlib
from simplejson import *
I like the else block, and want a "Success" and a "Fail" block. But the name of it is indeed the worst.
"When we print this, it's going to print nothing because we didn't print anything"
very wise words
15:35 can someone explain to me(very green and curious programmer) why not to declare this list like a = [1, [a, b], 2]? Tbh didn`t even know that you can declare variables in python like you did
These are type hints, they are relatively new to the language and optional. I'm pretty sure that they are not enforced at runtime, but the editor can use them to point out mistakes that you might have made.
11:49 Does no one make a ruff extension for pycharm
I think you’re more or less right, except the last part. I needed a deep copy maybe 2 times in my life, compared to 27286373 times of shallow copies
My thoughts on all of this:
Implicit string concatenation is fine, it does have unfortunate things that can happen with missing commas, but those are revealed if you run a formatter like black. It is also nice for separating strings across lines without the indent behavior of multi line strings.
While poorly named, I have used for…else a couple of times, and it is nice to not have to use an additional variable to store that state. It should have a better name, but I think that python would be worse without it.
Star imports are terrible.
Mutable defaults are definitely a curve ball when first learning python, but once they are understood that’s it. They are also fun for golfed caches. Unmentioned in the video, but there is a second, harder to explain stage when using lambdas since they bind late.
Shallow copies by default are also a learning barrier, but again it’s a thing that you only have to learn once. There is also the unmentioned tuple interior mutability, which feels like the same sort of issue.
Regarding for-else and while-else, I use the else often, but comment it as 'iterator finished'