ChainMap - very interesting. Have to think of use cases for it, though. As in the example in this video it might be done with simple dictionaries. Look: >>> default: dict[str, str | bool] = { ... 'theme': 'Light', ... 'language': 'English', ... 'notify': True, ... } >>> user: dict[str, str | bool] = { ... 'theme': 'Dark', ... 'notify': False, ... } >>> choice = default | user >>> choice {'theme': 'Dark', 'language': 'English', 'notify': False}
For example, if you change the original dicts after the ChainMap is created, then the ChainMap will dynamically use the changed values. Creating a merged dict will create a snapshot copy but will not follow the original dicts. The video mentions other use cases, like changing the ChainMap's first (empty) dict but shielding the others.
I had the same thought initially; then I thought of how to represent variable scope when writing a compiler/interpreter, and it clicked. This is exactly what you need for representing a stack of closures where your current scope can access variables from the parent closure and the global scope; though in those scenarios you would need the ability to update those variables, so it’s not exactly right, hmm..
I have needed it before, it is very useful for configurations and settings. Because it lets you save all the history of the settings. Imagine mid way through a program you want to override some settings for a particular task but don't need or want it to remain that way through so you gotta revert back to the original settings.
As a only occasional python user who never heard of this, from the name I was expecting something which chains the mapping functions - i.e. given a key, it looks it up in the first dictionary, and then uses the value as the key for the second one.
Love hearing about all the esoteric components of the collections, itertools, functools, etc. libraries!! Thx for your reliable quality and effective communication🎉
Any code using a tool you are unfamiliar with will be difficult to read. The tradeoff is that once you understand the reason the ChainMap is used, the code will become straightforward (assuming it is used appropriately; obviously, in many cases a dictionary will suffice). A huge advantage of ChainMap compared to merging dictionaries is that it effectively provides a way to continuously revise the merger of 2 or more dicts. There is no equivalent method that is not as hard or more hard to read.
one application of this would probably be in writing (simple) interpreters. this could store the variables and it's values; on every block-scope start, the chain map could have a new child, and update the keys on encountering variable declaration/assignment.
When will this be used? I can't think of any case that you want to merge dictionaries while keeping the reference to the original dictionary, just to accidentally mutate it / try to pop keys that don't exist in the first dictionary
@@しめい-l4mthat's actually not true at all. They just have a different approach to performance. The language itself is pretty slow, but there are a bunch of powerful libraries that are written in C and thus fast, so you get performance by offloading the majority of computation to those libraries.
Any case in which you want to keep a history of revisions. Imagine you are a user and you temporarily switch your theme to light mode, then switch back to dark mode. Then the application updates to default to light mode. Case A where the user preferences are stored in a merged dictionary results in the user being defaulted to light mode. The screen brightness overpowers the users frail eyes, blinding him permanently. Case B where the user preferences are managed by a ChainMap recognizes that the user intentionally set the value of the application to dark mode, and he preserves his eyesight. Lawsuit avoided.
Looks like exactly what is possible in JavaScript with prototypical chains. I know, Python has much more (than JS) powerful standard library, but this particular thing is exactly like prototypical inheritance. Also it's even simpler in JS with { ...default_settings, ...settings } for one-time computation, if one doesn't need dynamic combination. Also in generic CS terms, this is the chain of responsibility pattern.
Python has something quite similar with its inheritance model (see MRO), but it makes a distinction between index/subscript notation, like obj[x], and attribute access, like obj.x (even though objects are still normally backed by a dictionary!) So you could probably emulate most of what JS does using python’s language primitives but it would be quite a lot more cumbersome; i think this is one instance where I prefer JS’s “less is more” approach
Hi there! ChainMap saves memory when compared with the unpacking method at the expense of slighty slower lookup. Also, is also looks cleaner in the code to process missing keys hierarchically, for example, if a key is missing on user_preferences, go with the default.
ChainMap would not build a new dict but keep references to the two original ones, so if you modified them afterwards it would still stay in sync. Whether that is something you would want is up to you :)
Kind of going overboard with the type annotations don't you think? Like yeah ["abc"] is a list[str], does that really need an annotation? I find them most useful for function signatures/class fields and occasionally a local variable in a complex function. But using it literally everywhere just makes things unnecessarily complicated imo
Recently i've tried to find a solution for merging default settings and user settings. The problem is nested dicts. I want to override a setting which is inside a dict inside a dict inside a dict. So I've written a recursive function which iterates over each item of the tree and updates them if needed.
Thank you for your videos, they have titles in Portuguese, the only thing missing is AI dubbing, when will UA-cam make it available globally? It would be amazing.
This smells a lot like the Java Properties class. But Properties has capability for only a single parent. I struggle to find a real use case for multiple parents. I doubt that it would be reasonable to model a tree structure using ChainMap. hmmmm Insightful video - got me thinking... Thanks
I can't imagine that the ChainMap would be so useful. It might performs a mapping for keys and values inside dict, taking the key and iterating over each dict for values associated with this key and also by the reverse, returning one Key and all other values associated which is in the first and in any other. Instead, ChainMap stores each dict and if you want access any key this only returns the first
I might have bad imagination, but this structure looks pretty specialized to me. I can’t imagine use-cases besides before-mentioned settings. Does anyone have good ideas how to use it?
But why would you need this, the merge syntax of two dicts would solve this easily? In your example around minute 8 if you just put default before custom and anything where the default conflicts with the custom will be overridden
UA-cam presents your videos with automatic translation. As a viewer I have no chance to generally choose the original language. I have to switch from translation to original by hand. This is bullshit. Please consider changing your configuration.
@indently sure, but: For d in my_dicts: If key in d: Return d[key] I feel like Python devs have an obsession with convenience, and chainmap is a prime example
It's true that you can deconstruct any built-in function and manually write the functionality. But personally I would never re-write something that's already included in Python. I'm not going to compete with the professionals that spent years on optimising these functions for everyday use. Convenience is why most of us code in Python (and money probably too).
To me, it makes absolutely NO sense why the first map is the "child" and all the other maps are "parents". Shouldn't it be the other way around? Or is it called "child" because it is the node that is always updated when inserting something? Still conceptually the names are kind of horribly chosen.
Well, perhaps it’s by analogy with inheritance in OOP. Python itself first looks up a method on an object’s class, and if it doesn’t exist, it then looks at the _parent_ class (or classes) for a method with that name. If there are attributes/methods with the same name present in both the parent and the child classes, then the child takes precedence ( _overriding_ the parent)
- To all new Python coders who might be watching this: don't EVER call your dictionary "D1" or you will be fired. - Also, he says at 4:13 that type annotations are "...just telling Python that this is of type 'ChainMap'...", but that is incorrect. Type annotations were introduced in 3.5 and Python does not care about them at all. They are solely for coders to tell other devs what type is expected. You can, however, use "__annotations__" or type conversion ("int(input("Enter an integer: "))") to create enforcement checks, flags, etc.
how is that incorrect? You're hinting the type checker on the type of the variable. Maybe it would've been more accurate if he said something like "you're telling pyright the type of the variable" or whatever type checker you use but that just seems nitpicky. If anything, not telling the audience why and when you'd want to use chainmap over the way simpler alternative {**a, **b} is the real problem
>>>d1 = dict(a=1, b=2) >>>d2 = dict(c=3, d=4) is much clearer python. It directly types a dictionary with string keys assigned to int literals. Are the type hints just for your IDE? imo human readability comes before your IDE. But why would you type a dict anyway? If you want you special dict[str]=int, just subclass the builtin and assert it.
He is doing a demonstration and making us introduce a new concept for people who don't know about it. It's not about you, its about how you teach others.
@@landsgevaerare you really wanna hear the boring story of “pythonistic way” they come about whenever they want to make you believe their way is the best way?
This is not much clearer and is in fact unpythonic. Writing it correctly as d = {"a": 0} for example also correctly types d as dict[str, int] for the IDE's typechecker. Specifying it as d: dict[str, int] = {"a": 0} is just for readability, just like any other type hint in python.
These functions are so ridiculous. Who is this for? People bad at algorithms that don’t care about calling 69 functions just o have some trivial thing done for them? It’s a Node where they are passed by reference (cuz it’s an array ). Could just make a soa in np and get better performance.
Too much overkill with all those type definitions, when these don't do literally anything at the compile side, at least for this demo. The elegance of python programming being succinct is being corrupted bit by bit.
@@denom Yes and there they are useful especially for function signatures. But annotating a little variable in a tutorial with list[str] when you assign ["abc"] to it 3 characters later just makes your code less readable.
As a teacher, consistency is important. I make plenty of mistakes when explaining things and even for what you call "a little variable", I sometimes inadvertently insert the wrong datatype literally 3 characters later, and when you're explaining things, a little mistake like that can take time to find. But with type annotations I spot it immediately if I make a mistake. I annotate 100% of the time, that's my preference and it saves me far more than if I were to skip typing these little variables. The best part about type annotations in tutorials, is that you don't need to copy them, and you can follow whatever conventions you want.
ChainMap - very interesting. Have to think of use cases for it, though. As in the example in this video it might be done with simple dictionaries. Look:
>>> default: dict[str, str | bool] = {
... 'theme': 'Light',
... 'language': 'English',
... 'notify': True,
... }
>>> user: dict[str, str | bool] = {
... 'theme': 'Dark',
... 'notify': False,
... }
>>> choice = default | user
>>> choice
{'theme': 'Dark', 'language': 'English', 'notify': False}
I had the same thought
For example, if you change the original dicts after the ChainMap is created, then the ChainMap will dynamically use the changed values.
Creating a merged dict will create a snapshot copy but will not follow the original dicts.
The video mentions other use cases, like changing the ChainMap's first (empty) dict but shielding the others.
@@landsgevaer Thank you, never had such use cases, but will keep in mind an option for dynamically updated dicts in ChainMap.
I had the same thought initially; then I thought of how to represent variable scope when writing a compiler/interpreter, and it clicked. This is exactly what you need for representing a stack of closures where your current scope can access variables from the parent closure and the global scope; though in those scenarios you would need the ability to update those variables, so it’s not exactly right, hmm..
I have needed it before, it is very useful for configurations and settings. Because it lets you save all the history of the settings. Imagine mid way through a program you want to override some settings for a particular task but don't need or want it to remain that way through so you gotta revert back to the original settings.
As a only occasional python user who never heard of this, from the name I was expecting something which chains the mapping functions - i.e. given a key, it looks it up in the first dictionary, and then uses the value as the key for the second one.
Yeah, this one probably should be named as ChainDictionary or something like that
A case study in over-engineering in standard library. Somone was high on kool aid one afternoon.
Love hearing about all the esoteric components of the collections, itertools, functools, etc. libraries!! Thx for your reliable quality and effective communication🎉
I have never used ChainMap. That needs to change.
Chaing
ChangeMap
Reading a code using Chainmap, and being written by someone else, is going to be very challenging... and probably very painful to debug.
Any code using a tool you are unfamiliar with will be difficult to read. The tradeoff is that once you understand the reason the ChainMap is used, the code will become straightforward (assuming it is used appropriately; obviously, in many cases a dictionary will suffice). A huge advantage of ChainMap compared to merging dictionaries is that it effectively provides a way to continuously revise the merger of 2 or more dicts. There is no equivalent method that is not as hard or more hard to read.
Excellent demo of theory, followed by the default/preferences example. Thanks from an old 'hobbyist'learner!
one application of this would probably be in writing (simple) interpreters. this could store the variables and it's values; on every block-scope start, the chain map could have a new child, and update the keys on encountering variable declaration/assignment.
When will this be used? I can't think of any case that you want to merge dictionaries while keeping the reference to the original dictionary, just to accidentally mutate it / try to pop keys that don't exist in the first dictionary
According to the documentation, the advantage is speed. ChainMap uses a lazy approach to combining dictionaries, that doesn’t require running updates.
@@AWriterWandering I naturally assumed that performance was the last thing python devs would ever concern about.
@@しめい-l4mthat's actually not true at all. They just have a different approach to performance. The language itself is pretty slow, but there are a bunch of powerful libraries that are written in C and thus fast, so you get performance by offloading the majority of computation to those libraries.
Any case in which you want to keep a history of revisions. Imagine you are a user and you temporarily switch your theme to light mode, then switch back to dark mode. Then the application updates to default to light mode.
Case A where the user preferences are stored in a merged dictionary results in the user being defaulted to light mode. The screen brightness overpowers the users frail eyes, blinding him permanently.
Case B where the user preferences are managed by a ChainMap recognizes that the user intentionally set the value of the application to dark mode, and he preserves his eyesight. Lawsuit avoided.
2:30 : wait, so if you try to remove the value at 'c' it will error?
Edit: 3:51 shows that yes.
Looks like exactly what is possible in JavaScript with prototypical chains.
I know, Python has much more (than JS) powerful standard library, but this particular thing is exactly like prototypical inheritance.
Also it's even simpler in JS with { ...default_settings, ...settings } for one-time computation, if one doesn't need dynamic combination.
Also in generic CS terms, this is the chain of responsibility pattern.
Python has something quite similar with its inheritance model (see MRO), but it makes a distinction between index/subscript notation, like obj[x], and attribute access, like obj.x (even though objects are still normally backed by a dictionary!)
So you could probably emulate most of what JS does using python’s language primitives but it would be quite a lot more cumbersome; i think this is one instance where I prefer JS’s “less is more” approach
4:42 I've watched over a dozen of your vids, but never noticed you were Scandinavian until you said "Sandra" in a way only a Norseman would
Wow, I never would have guessed based on the accent… but then I went to 4:40 and heard “Sandra” and there it was, the unmistakable trill of Swedish.
I'm not Swedish, but I read in Swedish and go to Sweden often because I love Sweden.
im curious as to why even for your example with the settings you would use this when you can do preferences = {**default_settings, **user_preferences}
Hi there!
ChainMap saves memory when compared with the unpacking method at the expense of slighty slower lookup. Also, is also looks cleaner in the code to process missing keys hierarchically, for example, if a key is missing on user_preferences, go with the default.
ChainMap would not build a new dict but keep references to the two original ones, so if you modified them afterwards it would still stay in sync.
Whether that is something you would want is up to you :)
Python has the most random things built into it's standard library.
Kind of going overboard with the type annotations don't you think? Like yeah ["abc"] is a list[str], does that really need an annotation? I find them most useful for function signatures/class fields and occasionally a local variable in a complex function. But using it literally everywhere just makes things unnecessarily complicated imo
Recently i've tried to find a solution for merging default settings and user settings. The problem is nested dicts. I want to override a setting which is inside a dict inside a dict inside a dict. So I've written a recursive function which iterates over each item of the tree and updates them if needed.
I never knew this existed. In the past, I implemented something like this.
Fantastic video 👌
Very interesring. Thanks a lot
Thank you for your videos, they have titles in Portuguese, the only thing missing is AI dubbing, when will UA-cam make it available globally? It would be amazing.
This smells a lot like the Java Properties class. But Properties has capability for only a single parent. I struggle to find a real use case for multiple parents. I doubt that it would be reasonable to model a tree structure using ChainMap. hmmmm
Insightful video - got me thinking... Thanks
I can't imagine that the ChainMap would be so useful. It might performs a mapping for keys and values inside dict, taking the key and iterating over each dict for values associated with this key and also by the reverse, returning one Key and all other values associated which is in the first and in any other. Instead, ChainMap stores each dict and if you want access any key this only returns the first
I might have bad imagination, but this structure looks pretty specialized to me. I can’t imagine use-cases besides before-mentioned settings.
Does anyone have good ideas how to use it?
I’m sorry, but why would I ever want to use a ChainMap in real life? None of the examples felt in any way relatable.
Pls tell waht ide you use
Pycharm
But why would you need this, the merge syntax of two dicts would solve this easily? In your example around minute 8 if you just put default before custom and anything where the default conflicts with the custom will be overridden
UA-cam presents your videos with automatic translation. As a viewer I have no chance to generally choose the original language. I have to switch from translation to original by hand. This is bullshit. Please consider changing your configuration.
yeah the new auto translate features aren't thought through properly
I ended up changing my app language to english
@n0150 I ended up switching app locale to EN a couple of years before because of localized titles on 3b1b and other channels.
this is lowkey funny
Huh? Funny how?
Sounds like bugs waiting to happen.
Dictionaries have a union operator? Is this new or what?
Thank you 😊
Why did you choose the names Bob and Sandra?
Only my therapist could tell you that
Can chainmaps return one collapsed dictionary?
That sounds like inheritance
Your example isn’t practical, because you could achieve the same by just merging the user preference inside the defaults and not doing the opposite…
I'm struggling to see the purpose of this feature. Why would I use it over `|`?
Variadic parameters with short circuiting on lookup, also `|` will create a new copy whereas `chainmap` won’t.
type ChainDict = ChainMap
This doesn't solve the problem of the maps attribute.
def get_preference(key):
return user_preference[key] if key in user_preference else default_preference[key]
😕
With two dictionaries you can do that, but now imagine you have three or more.
@indently sure, but:
For d in my_dicts:
If key in d:
Return d[key]
I feel like Python devs have an obsession with convenience, and chainmap is a prime example
It's true that you can deconstruct any built-in function and manually write the functionality. But personally I would never re-write something that's already included in Python. I'm not going to compete with the professionals that spent years on optimising these functions for everyday use.
Convenience is why most of us code in Python (and money probably too).
To me, it makes absolutely NO sense why the first map is the "child" and all the other maps are "parents". Shouldn't it be the other way around? Or is it called "child" because it is the node that is always updated when inserting something? Still conceptually the names are kind of horribly chosen.
Well, perhaps it’s by analogy with inheritance in OOP. Python itself first looks up a method on an object’s class, and if it doesn’t exist, it then looks at the _parent_ class (or classes) for a method with that name. If there are attributes/methods with the same name present in both the parent and the child classes, then the child takes precedence ( _overriding_ the parent)
- To all new Python coders who might be watching this: don't EVER call your dictionary "D1" or you will be fired.
- Also, he says at 4:13 that type annotations are "...just telling Python that this is of type 'ChainMap'...", but that is incorrect. Type annotations were introduced in 3.5 and Python does not care about them at all. They are solely for coders to tell other devs what type is expected. You can, however, use "__annotations__" or type conversion ("int(input("Enter an integer: "))") to create enforcement checks, flags, etc.
how is that incorrect? You're hinting the type checker on the type of the variable. Maybe it would've been more accurate if he said something like "you're telling pyright the type of the variable" or whatever type checker you use but that just seems nitpicky.
If anything, not telling the audience why and when you'd want to use chainmap over the way simpler alternative {**a, **b} is the real problem
@@xetera I'm not sure how I could clarify my original post. Python doesn't care about type annotations; they're for the humans.
@dirtydevotee And python tooling like pyright. Otherwise it would just be a comment.
nce
early comment love your videos
>>>d1 = dict(a=1, b=2)
>>>d2 = dict(c=3, d=4)
is much clearer python. It directly types a dictionary with string keys assigned to int literals. Are the type hints just for your IDE? imo human readability comes before your IDE. But why would you type a dict anyway? If you want you special dict[str]=int, just subclass the builtin and assert it.
He is doing a demonstration and making us introduce a new concept for people who don't know about it. It's not about you, its about how you teach others.
Why is that "much clearer python"?
@@landsgevaerare you really wanna hear the boring story of “pythonistic way” they come about whenever they want to make you believe their way is the best way?
"Your code is not readable and I am the best programmer in the world"😂🥳
This is not much clearer and is in fact unpythonic. Writing it correctly as d = {"a": 0} for example also correctly types d as dict[str, int] for the IDE's typechecker. Specifying it as d: dict[str, int] = {"a": 0} is just for readability, just like any other type hint in python.
These functions are so ridiculous. Who is this for? People bad at algorithms that don’t care about calling 69 functions just o have some trivial thing done for them? It’s a Node where they are passed by reference (cuz it’s an array ). Could just make a soa in np and get better performance.
19 views in one minute
He really fell off
How can I stop getting the german versions of the videos? They are awful.
I honestly don’t know, I didn’t see any settings on my end to change the translations anywhere.
Too much overkill with all those type definitions, when these don't do literally anything at the compile side, at least for this demo. The elegance of python programming being succinct is being corrupted bit by bit.
Have you ever worked in a very large Python codebase, which is not even yours? Typehints will save your life.
@@denom Yes and there they are useful especially for function signatures. But annotating a little variable in a tutorial with list[str] when you assign ["abc"] to it 3 characters later just makes your code less readable.
As a teacher, consistency is important. I make plenty of mistakes when explaining things and even for what you call "a little variable", I sometimes inadvertently insert the wrong datatype literally 3 characters later, and when you're explaining things, a little mistake like that can take time to find.
But with type annotations I spot it immediately if I make a mistake. I annotate 100% of the time, that's my preference and it saves me far more than if I were to skip typing these little variables.
The best part about type annotations in tutorials, is that you don't need to copy them, and you can follow whatever conventions you want.
In the last one: what about cm.parent.parent?