I found that std::variant::visit method in the documentation and was wondering why it might be a good idea to ever use it. After searching around a bit, I found this talk. Very illuminating, I dig this logic and I was always interested in various ways of doing runtime polymorphism.
Absolutely amazing presentation! Thank you Klaus Iglberger, I really enjoyed watching this and learned a good few things along the way. Do not hesitate to hire this man for training.
In relation to the Acyclic Visitor pattern presented, I believe the Visitor classes could virtually inherit from the empty base AbstractVisitor, and therefore both let operation implementers eschew the extra base and potentially obviate the cross-cast in the accept implementations in favor of a simple downcast. Can anyone shine some light on any downsides? Edit: Oops that’s impossible. Have a nice day people
Nice thanks for the talk. I agree visitor works good in some cases. To be honest though I’ve seen it applied in cases where it makes everything very complex, especially when templates start to become involved. It seems to suffer a bit from the issue where if you make a new class you then need to update like 10 different spots to fully implement it. It ends up being a bit similar to having switch statements everywhere for an enum type. Also the issue where you need to know all the types used in a std::variant beforehand, so can’t extend things from different assemblies without adding a lot of templating. But as always, time and place 😊
So, visitor is for adding operations and type erasures for adding types easily. What if we combine and apply these two concepts, i.e., apply type erasure on the "visitor" hierarchy part? Or maybe std::variant is doing that?
@ABaumstumpf 2 simple rules: - if you need indirection, don't pay for it for every data element. Instead of vector, use tuple. This utilizes all the cache lines for the data during iteration. This is suitable for closed polymorphism. - For open polymorphism, use virtual functions that take ARRAYS of elements and not a single element as arguments. This makes the indirection overhead much lower as you need to jump only once for every element type.
@@embeddor3023 that's a nice example of Structure of Arrays (SoA) versus of Array of Structures (AoS) which was well explained in Avi Lachmish's CppCon 22 presentation on data locality and parallelism
@@embeddor3023 sometimes you need to preserve the relative order of things. For example, you need to draw (circle, square, circle) and not (circle, circle, square). In those cases you can't do SoA without an additional indirection array.
Author says that with the visitor pattern it is easier to add operations․ My concern is that actually, instead of overriding those functions in derived classes, we create types of these functions (like Rotate, Draw etc.) and when we add a function, we need to introduce new type like, e.g., Serialize. But this is not enough actually: we have to also override ALL visit functions in that class. So, what is the difference between overriding visit functions and Serialize function in a derived classes? In any case in both cases we have to override K functions (either visit or a function) if there are K operations. Although I understand that we actually might break dependencies from 3rd parties in our actual concrete class (like Circle, Square etc.). For example, if there is a 3rd party tool which helps to draw, then Circle won't know anything about that library. Instead, concrete visitor will know about it, hence we avoid introducing dependencies from 3rd party library for the concrete types (Shapes). I mean this is I understand and accept ,but how we make easier adding functions: that part I don't understand well.
I'm no expert on this topic, but from my understanding there should only be the one member function in each class (type) that accepts any visitor. The visitor knows how to do some operation on a specific set of classes (types). Separating the class (type) from the operation means that if we want to add a new operation to some subset of types, we get to decide which ones we can operate on from the perspective of the operation instead of the perspective of the class (type). With his shapes example, the base is "Shape", and when you start adding operations to Shape, the dependencies are now connected to all Shapes, which may be undesirable. Perhaps you could do multiple inheritance and make the various derived shapes "drawable", "serializable", etc., but this is a different take. Visitor moves the ability to operate on a Shape to outside the Shapes themselves, taking a Shape and knowing how to operate on it to accomplish the operation.
I'm hoping that the issue is actually resolved in this talk, but at 11:30ish the issue of having to touch the base class to add more operations... visitor (as far as i know) has the same limitation
Very good Klaus. Great complement to the type erasure talks. Perhaps would have liked to have seen the overload() idiom for the implementation of the modern visitor, rather than a functor.
21:38, you don't need them, you need a dynamic static array, at the start before you ever use the class you register the classes in use along with their callbacks, you then can have the global visit() etc functions use take registered IDs to identify where in the array to select the actual function from, you destroy the array only at the exit of the program. Still, looking at what you've shown so far it seems the c++ community is taking the long route to learning C is better for control, despite it lack of the awkward semantics you lot introduce to your extension of the language it ends up being both easier to understand and easier to adapt, I can only sit amused by how long you're taking to re-learn what programmers in the early days learnt without fuss. Even I who was just a teenager when I first dabbled in programming realised quickly that C gave me everything I wanted after I learned how inflexible object oriented programming and a lack of types was via javascript
Talk about completely missing the point, like the typical C programmer stuck in the 70s. In the real world, you need to maintain the code, that was brought up repeatedly in the talk. Once you closed down the types and operations, of course you can make a super optimized version with no types at all, not even functions. By the way, did you know C copied some of the C++ "extensions"? How do you sleep at night, knowing C was "tainted?"
Gotta love when people go the long way around and end up with the obvious tagged union approach that we've had since forever. If you're thinking about Object-Oriented Programming and patterns and what's theoretically going to be a good design, you're doing it wrong. You should be thinking about more fundamental ideas like performance, decoupling, systematization, minimalism, etc., and always in the context of solving a concrete problem. The code will tell you what needs to happen to it over time if you listen.
looks all nice on paper but i rather debug a switch statement with a type instead of a std::visit on a variant. std::visit looks only good until u actually use it. imo std::visit was a mistake and is just a very very horrible compensation for not having pattern matching
our processors don't support our software well: that's the implicit issue. The OO solution _should_ be the best solution, but it doesn't scale well with our hardware architecture model, even if much work has gone into trying to keep cache lines filled and branches correctly predicted. It seems that as software engineers we have to do significant work to try and maintain processor efficiency as high as possible, and programmers outside of the C++ community are likely to think less about this stuff, if at all. How many programmers even know what's going on inside the processor these days?
I'm not sure the virtual call case can be handled well by hardware, short of some kind of (likely very brittle) "heads-up" opcode in order to prefetch and pre-pipeline into a call. My intuition is further emphasized by the fact that the Mill guys aren't even trying to optimize dynamic dispatch, despite the whole architecture centering on being able to execute ordinary code as if it were an inner loop.
Unfortunately DRAM is slow but it can be amortized by reading chunks of data in cache lines, that makes parallel arrays of data pre-fetchable while prettily organised extensible code with abstract types using indirection is slower.
So with the visitor - something changes aaaand now what? How to make sure that all new shapes implement all the needed functions? You also limit your self and the compiler to the public interfaces. More complexity will tend to reduce performance of the visitor pattern - at least we have seen that several time sin our codebase and while the local code is easier to read understanding what is happening exactly became harder. But in general I am trying to push the code in that direction cause the computations them self are several orders of magnitude faster than other parts of the system and it would help with getting new guys to understand the system.
> How to make sure that all new shapes implement all the needed functions? You get a compiler error if an operation you declared as a "must have" is not yet implemented. The problem is not in implementing new operations (you'll need to do this anyway), the problem is how to do this non-intrusively, i.e. without diving into the guts of the existing library, but extending it outside, like with plugins. > You also limit your self and the compiler to the public interfaces. This is a price of extensibility. Also, if your class' private members should be used in operations involving external dependencies, like painting using multiple representations, serializing using multiple encoding, etc, then probably your class breaks SRP, and it is time to make it leaner, may be even reducing to a simple value type with a minimum of behavior.
when we're talking about the downsides of visitors, are we going to just keep ignoring the elephant in the room -- the huge dedicated class with a bunch of operator()'s declared for every type? rust and zig do the same exact thing on the language level with no syntactic overhead using tagged enum pattern matching. it's literally the same thing on the lowest level, you just don't have to write a whole bunch of garbage boilerplate code like intermediate classes with operators that makes it happen.
For me I don’t really see a benefit. This seems like the kind of thing you end up implementing to duct tape a solution together for a codebase that has lived for way too long and or grown too big.
Definitely. Ugly, boilerplate-ridden, and a minefield for common OOP-reference pitfalls. Worth using only with a massive amount of understanding and articulation
good, thought-provoking talk from Klaus Iglberger as always. But is the CppCon idea to dribble out sessions until CppCon 23 rolls around? I guess the amount of material is testament to C++ and CppCon's popularity!
Klaus Iglberger is definitely my favorite presenter!
I found that std::variant::visit method in the documentation and was wondering why it might be a good idea to ever use it. After searching around a bit, I found this talk. Very illuminating, I dig this logic and I was always interested in various ways of doing runtime polymorphism.
Absolutely amazing presentation! Thank you Klaus Iglberger, I really enjoyed watching this and learned a good few things along the way. Do not hesitate to hire this man for training.
In relation to the Acyclic Visitor pattern presented, I believe the Visitor classes could virtually inherit from the empty base AbstractVisitor, and therefore both let operation implementers eschew the extra base and potentially obviate the cross-cast in the accept implementations in favor of a simple downcast.
Can anyone shine some light on any downsides?
Edit: Oops that’s impossible. Have a nice day people
I don't think recompilation in result of touching (6:50) the enum is a disaster. Making change all over the code is much more important though
Nice thanks for the talk. I agree visitor works good in some cases.
To be honest though I’ve seen it applied in cases where it makes everything very complex, especially when templates start to become involved.
It seems to suffer a bit from the issue where if you make a new class you then need to update like 10 different spots to fully implement it.
It ends up being a bit similar to having switch statements everywhere for an enum type.
Also the issue where you need to know all the types used in a std::variant beforehand, so can’t extend things from different assemblies without adding a lot of templating.
But as always, time and place 😊
So, visitor is for adding operations and type erasures for adding types easily. What if we combine and apply these two concepts, i.e., apply type erasure on the "visitor" hierarchy part? Or maybe std::variant is doing that?
Whether adding types or adding operations, I think it is relatively frequent. Unless you are very sure, it is better to use the most direct method.
ua-cam.com/video/qn6OqefuH08/v-deo.html
Link to the type erasure talk
No mention of Expression problem… :(
Since I started working in HPC, whenever I see dynamic polymorphism used on data, I cry inside. I think I need a therapy ...
Why? It can also make the code easier to read and it can be better performance - just depends on the design.
@ABaumstumpf 2 simple rules:
- if you need indirection, don't pay for it for every data element. Instead of vector, use tuple. This utilizes all the cache lines for the data during iteration. This is suitable for closed polymorphism.
- For open polymorphism, use virtual functions that take ARRAYS of elements and not a single element as arguments. This makes the indirection overhead much lower as you need to jump only once for every element type.
@@embeddor3023 that's a nice example of Structure of Arrays (SoA) versus of Array of Structures (AoS) which was well explained in Avi Lachmish's CppCon 22 presentation on data locality and parallelism
Is variant indirection? It stores data in place @@embeddor3023
@@embeddor3023 sometimes you need to preserve the relative order of things. For example, you need to draw (circle, square, circle) and not (circle, circle, square). In those cases you can't do SoA without an additional indirection array.
Author says that with the visitor pattern it is easier to add operations․
My concern is that actually, instead of overriding those functions in derived classes, we create types of these functions (like Rotate, Draw etc.) and when we add a function, we need to introduce new type like, e.g., Serialize. But this is not enough actually: we have to also override ALL visit functions in that class. So, what is the difference between overriding visit functions and Serialize function in a derived classes? In any case in both cases we have to override K functions (either visit or a function) if there are K operations.
Although I understand that we actually might break dependencies from 3rd parties in our actual concrete class (like Circle, Square etc.). For example, if there is a 3rd party tool which helps to draw, then Circle won't know anything about that library. Instead, concrete visitor will know about it, hence we avoid introducing dependencies from 3rd party library for the concrete types (Shapes). I mean this is I understand and accept ,but how we make easier adding functions: that part I don't understand well.
I'm no expert on this topic, but from my understanding there should only be the one member function in each class (type) that accepts any visitor. The visitor knows how to do some operation on a specific set of classes (types). Separating the class (type) from the operation means that if we want to add a new operation to some subset of types, we get to decide which ones we can operate on from the perspective of the operation instead of the perspective of the class (type). With his shapes example, the base is "Shape", and when you start adding operations to Shape, the dependencies are now connected to all Shapes, which may be undesirable. Perhaps you could do multiple inheritance and make the various derived shapes "drawable", "serializable", etc., but this is a different take. Visitor moves the ability to operate on a Shape to outside the Shapes themselves, taking a Shape and knowing how to operate on it to accomplish the operation.
I'm hoping that the issue is actually resolved in this talk, but at 11:30ish the issue of having to touch the base class to add more operations... visitor (as far as i know) has the same limitation
Enjoyed it, ty for sharing it
how does this compare to using components?
good talk
Very good Klaus. Great complement to the type erasure talks. Perhaps would have liked to have seen the overload() idiom for the implementation of the modern visitor, rather than a functor.
21:38, you don't need them, you need a dynamic static array, at the start before you ever use the class you register the classes in use along with their callbacks, you then can have the global visit() etc functions use take registered IDs to identify where in the array to select the actual function from, you destroy the array only at the exit of the program. Still, looking at what you've shown so far it seems the c++ community is taking the long route to learning C is better for control, despite it lack of the awkward semantics you lot introduce to your extension of the language it ends up being both easier to understand and easier to adapt, I can only sit amused by how long you're taking to re-learn what programmers in the early days learnt without fuss. Even I who was just a teenager when I first dabbled in programming realised quickly that C gave me everything I wanted after I learned how inflexible object oriented programming and a lack of types was via javascript
Talk about completely missing the point, like the typical C programmer stuck in the 70s. In the real world, you need to maintain the code, that was brought up repeatedly in the talk. Once you closed down the types and operations, of course you can make a super optimized version with no types at all, not even functions. By the way, did you know C copied some of the C++ "extensions"? How do you sleep at night, knowing C was "tainted?"
Gotta love when people go the long way around and end up with the obvious tagged union approach that we've had since forever. If you're thinking about Object-Oriented Programming and patterns and what's theoretically going to be a good design, you're doing it wrong. You should be thinking about more fundamental ideas like performance, decoupling, systematization, minimalism, etc., and always in the context of solving a concrete problem. The code will tell you what needs to happen to it over time if you listen.
looks all nice on paper but i rather debug a switch statement with a type instead of a std::visit on a variant. std::visit looks only good until u actually use it. imo std::visit was a mistake and is just a very very horrible compensation for not having pattern matching
our processors don't support our software well: that's the implicit issue. The OO solution _should_ be the best solution, but it doesn't scale well with our hardware architecture model, even if much work has gone into trying to keep cache lines filled and branches correctly predicted. It seems that as software engineers we have to do significant work to try and maintain processor efficiency as high as possible, and programmers outside of the C++ community are likely to think less about this stuff, if at all. How many programmers even know what's going on inside the processor these days?
I'm not sure the virtual call case can be handled well by hardware, short of some kind of (likely very brittle) "heads-up" opcode in order to prefetch and pre-pipeline into a call. My intuition is further emphasized by the fact that the Mill guys aren't even trying to optimize dynamic dispatch, despite the whole architecture centering on being able to execute ordinary code as if it were an inner loop.
Unfortunately DRAM is slow but it can be amortized by reading chunks of data in cache lines, that makes parallel arrays of data pre-fetchable while prettily organised extensible code with abstract types using indirection is slower.
So with the visitor - something changes aaaand now what?
How to make sure that all new shapes implement all the needed functions?
You also limit your self and the compiler to the public interfaces.
More complexity will tend to reduce performance of the visitor pattern - at least we have seen that several time sin our codebase and while the local code is easier to read understanding what is happening exactly became harder.
But in general I am trying to push the code in that direction cause the computations them self are several orders of magnitude faster than other parts of the system and it would help with getting new guys to understand the system.
> How to make sure that all new shapes implement all the needed functions?
You get a compiler error if an operation you declared as a "must have" is not yet implemented.
The problem is not in implementing new operations (you'll need to do this anyway), the problem is how to do this non-intrusively, i.e. without diving into the guts of the existing library, but extending it outside, like with plugins.
> You also limit your self and the compiler to the public interfaces.
This is a price of extensibility. Also, if your class' private members should be used in operations involving external dependencies, like painting using multiple representations, serializing using multiple encoding, etc, then probably your class breaks SRP, and it is time to make it leaner, may be even reducing to a simple value type with a minimum of behavior.
amazing....
when we're talking about the downsides of visitors, are we going to just keep ignoring the elephant in the room -- the huge dedicated class with a bunch of operator()'s declared for every type? rust and zig do the same exact thing on the language level with no syntactic overhead using tagged enum pattern matching. it's literally the same thing on the lowest level, you just don't have to write a whole bunch of garbage boilerplate code like intermediate classes with operators that makes it happen.
For me I don’t really see a benefit. This seems like the kind of thing you end up implementing to duct tape a solution together for a codebase that has lived for way too long and or grown too big.
We need a new word! Let's call it 'auto procedural programming'.
This is more or leas how Rust works ny default.. Draw would be a trait in Rust
this pattern is a total and complete antipattern.
no
Definitely. Ugly, boilerplate-ridden, and a minefield for common OOP-reference pitfalls. Worth using only with a massive amount of understanding and articulation
good, thought-provoking talk from Klaus Iglberger as always. But is the CppCon idea to dribble out sessions until CppCon 23 rolls around? I guess the amount of material is testament to C++ and CppCon's popularity!