Beyond Clean Code: The Layers of Optimization
Вставка
- Опубліковано 28 чер 2024
- Read the full article: tobeva.com/beyond
In some circles online, “clean code” has become a euphemism for “bad code.” People say that clean code contains piles of useless abstractions, is overengineered, obscures the flow of control with tiny methods and classes that each do very little, and leads to horrible performance.
Robert Martin, aka Uncle Bob, wrote his book "Clean Code" in 2008. It contains OOP advice because the book describes how his consulting company wrote OOP at the time, generally in Java. His book was never meant to be the last word on developing software. It's just him documenting the approach they used at the time, in the 1990s and 2000s.
OOP can indeed have too much overhead to be used in tight inner loops; however, in the broader world of software, only a fraction of code needs to be heavily optimized. Most real systems contain a blend of styles and degrees of optimization, and this is a good thing.
0:00 - Introduction
0:25 - The Clean Code Book
1:20 - Definition of Clean Code
2:16 - Thought-terminating Cliché
2:37 - Casey Muratori
3:17 - The Code
5:54 - Memory Layout
7:45 - Virtual Functions
8:25 - OOP Advantages
9:48 - Casey's Conclusion
10:02 - Casey is Right
11:10 - Casey is Wrong
12:27 - The Chainsaw Analogy
14:49 - Beyond Clean Code
16:23 - Game Engines and Game Code
18:06 - Python
20:16 - Function Programming
21:15 - Fred Brooks and Physical Bits - Наука та технологія
To optimize my code, I declare my main function as constexpr so all my code is executed at compile time and I achieve infinite performance
Incredible. You are literally already done before starting.
great video, found it through the article which was also very good
"Clean code contains piles of useless abstractions, is overengineered, obscures the flow of control with tiny methods and classes that each do very little"
Because clean code has tools to create abstractions, however it doesn't give you an answer to the question why and when to use it, so people apply to everything
if you don't limits then it doesn't have limits
Clean code isn't written with good engineering vocabulary, it is written as gospel
This means that if you remove 2-3 words from the gospel it has different meaning
A good engineering and well written rules don't have that problem
a great example is Fallacies of distributed computing - if you know them you can't misuse the rules, unlike the rules proposed by Clean code/ SOLID or other gospels on the internet
There is an approach that was for some reason overlooked by most in the PLT community: languages with a well defined separation between logic and performance semantics. What our manual optimisations are? Just applications of certain rules and strategies - e.g., scramble and de-scramble data around some heavy computation (turn a bitmap into a z-curve, etc.), turn an array of structures into a structure of arrays, fuse loops, and so on. These optimisations make code unreadable and unmaintainable. So the more reasonable approach is to write a readable high-level code, and then annotate it with optimisation hints that must be automatically applied. You really don't need to sacrifice performance for readability while you can have both.
I generally agree with this assessment (talking about loop bodies is particularly helpful), but you're running into the same problem Casey did when this went viral. The reason people don't like clean code as a book is that you have a bunch of developers using it as a reason to make an abstraction for shapes when they only have one shape type in code.
People don't like having to extend code that's prematurely optimized for essentially the same reason they don't like something that's prematurely abstracted: both are very difficult refactors and probably mean changing a fundamental data structure or a shared function call.
The advice from both Casey and Bob are the same here: start very simple, and when you see shared values, you put them into a data structure. The difference is primarily on how they handle procedures, and how early they reach for an abstraction.
Ask yourself how easy it would be to extend either of these. Since the code hasn't actually gotten the avx implementation yet, it's actually not that hard to make another struct or union for other shapes, and to add appropriate methods to handle that data. The version with inheritance wants to own all possible shapes in its parent class, and so if you run into a shape that doesn't fit the parent, you run afoul of liskov and all your efforts have to be rethought. If you find you have a need to optimize, the work You've done up to the present is also a barrier rather than a help.
Clean code as a concept is a bit vague but like porn, you know it when you see it. Unlike porn, no one's going to complain if they see it on a monitor in the office. The book is more an artifact of the Java era, but can still hold some value to developers who know when they're being too proscriptive. The trouble is that since the rise of OOP and clean code, devs increasingly don't have access to this info. They don't know what Uncle Bob assumed they do - that steps like polymorphism over branching can make complex problems easier to read, but they always come with a sizeable overhead, and so shouldn't be taken until very late in a project when you need extreme measures to keep things manageable.
I'd generally prefer working with Casey's code over Bob's because i think its simplicity makes it easier to work with, but their process that it starts out exactly the same: do the simplest thing that accomplishes the goal. They then both eliminate redundant code through data structures and better procedures. The difference is the goal they're working towards, and my experience is that close adherence to OOP and the Coean Code book simply isn't readable or maintainable in larger projects because of the code winds up being loosely coupled to a lot of things, and too generic to offer much of a guide to its purpose if the original name isn't relevant any more.
In the video I'm not saying we should follow the guidelines of the Clean Code book. I'm saying that we should take back the phrase "clean code". I think that any code can be clean code. Casey's good code is clearly "clean code", for example. It's silly to cede a generally useful phrase like, which has a long history, to mean "the style of OOP recommended by a 16 year old book that was primarily about Java programming".
My second point was that Casey's example only showed OOP was not suitable when the inner loops were tiny: nanoseconds. He did not at all show OOP was unfit when timescales were longer, although he falsely claimed to.
But you raise a much bigger question. Which is given these different styles, what is the best way to develop code? That's a way bigger topic than my video attempted to cover. I could imagine in a future post trying to dig more into that, it's a huge topic.
@@tobevasoftware I can absolutely get behind that. I also think Casey's "don't teach this without caveats" is fine, especially in the context of the course. His intro even covers doing the same stuff in Python, so there's even more agreement here than I think the Twitter debate will ever capture.
What I find very helpful about your video is that this debate often gets framed as a kind of "game devs do this, but 'enterprise' does it this way " culture and practice argument. You very neatly show how thinking about scale should probably change your decision making process, even in the world of game development where people worry about milliseconds. It's one thing to do SIMD operations on a rendering method. It's another to insist that your artists and level designers write in C or C++ and avoid inheritance.
So people don't like the book because some other people have a reading comprehension problem? Really?
@@7th_CAV_Trooper to a certain extent, yeah, but it's kind of a problem with the layout. Most people read it as "do this instead of this" instead of the far more reasonable "start here but you probably want to go here when you get some repetition". I think some of that is the Uncle Bob videos, which I always thought took an even harder stance.
@@MrBreakstuff If it's easy and it meets the customer's needs, why should you care?
I personally began struggling to read Casey's code, there was way too much nuance and cognitive load. As soon as I see pointers I start thinking about managing them.
The name "clean code" has a lot of bad baggage that needs to be cut and proper well though out ideas and methadologies associated with it for people to forget the currently bad "clean code" and improve.
Casey's video just highlighted the currently bad "clean code" with it's on the surface layer concrete, but on the lower layer very poorly defined rules and methodologies. His video on "Performance excuses debunked" is great at showcasing the reasoning of his "attack" on "clean code" and how it's more on the issue that software is doing less problem solving and more wasting time for the users and devs, so being more performance aware should improve the life of a developer and improve the final product for the user.
The clean code "rules" should be associated with examples, then analysed if they are practical and performant in different environments (embedded, games, frontend/UI , server, networking, etc.) and have rules of thumb that say how you should tweak it for your environment. E.g. in games, we figured out that shallow and wide inheritance with composition is generally better than deep and narror inheritance.
I would really like if you and Casey have a discussion. It would make a really good video.
As Obi-Wan once said: "Only a sith deals in absolutes." Same goes for coding😊
I found this to be very insightful. It's very easy to fall into the mind set that clean code means boiler plate - bad performance code. But in reality, OOP, Functional, POP, DOD, etc. They're all tools to solve problems we might face in our programming journey. Refusing to learn and understand one of these would mean (Utilising your own analogy) refusing to use chainsaws due to its failure in a specific task.
The main points I got out of this is to look beyond the code and understand what is really happening behind the scenes. Understand why your code isn't running well and instead choose the best tool for the job.
Keep up the great work with these insightful videos!
11:45 - spending time to optimize it isn't worth it... but is spending time making this abstraction worth the time then? Obviously in candy crush there are no special shapes, so what's the point exactly of spending more time crafting obfuscated code + making it slower? Maybe im not clever enough for oop and solid but it never made sense to me how this is "clean". Caseys code took me a few seconds to grok, the other code requires up to date documentation with arrows pointing all over the place, especially if it gets more complicated over time.
I think it's also telling that enterprise software that is shipped out of these Java shops is usually painfully slow and universally hated by people who use them. Like, it's only recently I realized that Java is actually a fast language, I had the idea that it was a slow and bloated language due to how awful the software is.
What makes you think the word clean means obfuscated?
In the video I don't say which of these is the better solution. And I don't think there really is an answer without knowing the context: the rest of the codebase, the goals of the project, the people involved. None of which exist since it's a toy made-up example.
All I'm saying with this slide is that performance is not a reason to avoid the OOP version here, in this specific case. Even while performance is a absolutely a reason to avoid OOP in other specific cases.
@@tobevasoftware I understood your point, the issue I have is that you made a point of when to use the clean version; the optimized code is still objectively easier to understand, modify and extend - and it still has the benefit of being faster.
There really is no point in managing these classes for something as straight forward as implementing different strategies based on type. And to maintain this abstraction you have to come up with a bunch of more abstractions so as to not break the abstraction. I just do not see the point. I've never seen a code base where switching a fundamental piece out has ever been easy despite all these smart engineers coding in this style.
Thanks for listening to my Ted talk.
Java and C# are extremely popular with the "enterprise" software which tends to have bad performance because the users are a captive audience. Internal employees who are forced to use the system. There's no competitive pressure. Like the restaurant inside a hotel that's in the middle of nowhere. The polar opposite of that are companies where the users can switch to a competitor with a single click. There performance is table stakes. Imagine a new TikTok clone but the videos load really slowly. But as you allude to Java (Minecraft) and C# (Unity) aren't inherently slow. Slower than C/C++, but decently fast if you are careful.
Finally someone who adds substance and nuance to the polemic discussion of capital C clean code. The double logarithmic plot at 19:20 is pure gold and a great view of optimization as a tradeoff.
Also, is 22:11 why it's called C++? 🤔🤯
I love a good log-log plot. An inspiration for them were "population pyramids" if you've seen those:
en.wikipedia.org/wiki/Population_pyramid
A problem with OOP is that both logic and data get scattered all over the place. I'm not sure that by splitting everything into tiny little bits of files, classes, functions and data you really gain in maintainabily in the long run. As to Caseys example: An array of triples (shape,w,h) together with a definition what shape, w, and h are *is* much simpler and much simpler to work with than all these classes. Suppose, for some reason, you need to make a copy of you data structure. It will be one line of code for the array of values, whereas you will need to add a virtual clone method to every class in the OOP case. Conjecture: OOP code almost always gets pretty complex pretty soon, as you do not see the entire picture anymore.
Abstractions have a cost; more isn't better. Now instead of learning the problem I have to learn your little dsl abstraction as well. Unless it's a genuine business concept or *really* gnarly, just write the obvious thing. And even then, save the grandiose design for version 2.0 when we're positive we know what we're solving (this is a lie, you will never know)
It's easier to add code than to change it. Stop trying to accommodate the future; you're actually making it harder.
Thank you for this lecture.
11:51 As you showing mobile game, you forgot about power effeciency and battery use, then Casy is still not wrong.
The time it takes to process 100 shapes is one ten-thousandth of the frame, so the energy expenditure would be equally negligible. So, no, there is no measurable energy advantage in the 100-shapes case.
But in the 1M shapes case, you are right. Going from 88% of the frame time down to 3.6% would save a lot of energy. Even if, to your eye, the game plays the same. So yes, you should use Casey's optimized version if you need to process 1M shapes per frame.
The whole trick is that 100 shapes are such a small amount that it basically doesn't matter how you process them. While 1M shapes are so many, it matters a great deal how you process them.
@@tobevasoftware Generaly may be neglible but It depends how finely tunned is idle detection on hardware as i know on mobile devices this detection is very "agreessive".
But also this exemplary 100 shapes is only a part of this game or part of processes running on this device, if every programmer wave a hand on few cycles or microwatts we end in situation we currently often have: short battery life and/or sluggish system response.
Simillar topic is about cloud.
Backend on node or php may produce acceptable response times in miliseconds,
but we use many times much energy or many more machines.
@@AK-vx4dy This. The mindset of being intellectually lazy at the cost of performance will apply to all parts of the program. And nowadays, it applies to all levels of the code, except if you're lucky enough to use some low-level library written by someone who is actually a good programmer who cares about performance. Saying a single thing can be slow because it is not that much anyway implies everything else in that application is as fast as it should be, and you're being sloppy just this once on this tiny irrelevant part. But that is never the case. Someone who has the discipline to write a good rest of the program, also has the discipline to write that one part properly. And OOP is not even saving you any effort, because it takes longer to type.
I think there are tradeoffs worth making, and tradeoffs that aren’t. Even with the need to support many different types of shape, it would be much more performant to use a switch on the type with static functions calls, as opposed to anything Java offers.
Eventually the functions wouldn’t for in the L1 cache anymore, sure, but my personal CPU (Apple M1) has 12 MB of L2 cache, which can fit a truly staggering amount of functions before it is full.
This is where languages like Rust are interesting, it has the same ergonomics as the Object Oriented code (just a straight method call on the shape) but does compile down to the static jump table, which doesn’t happen with virtual functions (and might sort of happen in Java, but only if the JVM deems your code worthy of optimisation, which should have been done at compile time instead of needlessly pausing your code)
*wouldn’t fit in the L1 cache
13:19 That chainsaw comparison is quite good, however I think it’s kinda funny how the chainsaw was originally invented for use during complicated childbirth situations.
Wow that's amazing, I had no idea. Yes, that's extremely ironic for the analogy.
After watching that video, my only thought is: When did Clean Code was promoted as a way to improve performance? If not, what's the point to make such a claim ? Unless when someone wants to promoting his own course?
Because computers these days are incredibly slow and terrible and your thought process is the exact reason why. I'm glad someone is pushing back against the laziness programmers have been displaying, and actually demanding quality again.
can't tell if this is a troll or not. Moore's law still valid today.
Most code should be created for general use cases unless we need to make a trade off to achieve optimization aspect.
Clean code is good code optimized for maintainability and not every piece of code needs to be optimized for speed. Everything is a trade off and the art is knowing when to trade which for which.
Like in gaming we definitely want to trade readability for speed as most games lifespan are shorter than exterprise softwares whilst we'd want trade speed for maintainability for enterprise softwares in long run.
Dogmatically thinking one type of optimization must be used everywhere is failing to see the full picture. There is no sliver bullet. Every decision is a tradeoff.
Saying "clean code: horrible performance" is like saying "performant code: horrible readability". It is meaningless to say so except someone wants to draw attentions to promote one's courses behind paywall.
I am trolling now:
I am glad Casey video exposed so many single-minded people to worship one-sided opinions as facts.
@@chauchau0825 first, Moores law is not in effect today. Not in a way that's meaningful to the past.
Second, there's no way that having yo go to 5 different objects through multiple function pointers is more readable than all the code in the same spot. That's just a strawman.
Finally, have you ever worked on an "enterprise code base"? They're notoriously hard to work with and extend. Way harder than the if statement casey has.
So I just think you don't know what you're talking about. Fundamentally you might like the oo approach because you're used to it, but that doesn't mean it's any those things you said. It's just pure propaganda.
@@VACatholicgood luck when your 5 star dev that optimizes the f out of everything leaves the company and nobody knows how to read that unreadable code.
For a company the most important thing is maintenance of code, so when errors might happen even the dumbest dev could fix that easily.
Unless you will code the next gta 5 game you will never need those crazy a... optimizations for some indie games or less demanding apps. Nobody cares.
the moment the guy hid the assembly code i knew that rant was bs. Maintainability of software is the most important thing. Speed is irrelevant when every year pc are getting better and better. As you said. Moore's law.
Nice essay, it is pleasnt to listen. Excellent job!
I feel that your title does not match your argument. To extend the metaphor, your argument seems to be: "use chainsaws to cut trees, and scalpels to cut surgical incisions. They both have a place, and it depends what you're trying to cut and why." Meanwhile, your title reads: "Chainsaw Cutting is Good Cutting."
The title doesn't mention chainsaws, even though yes the thumbnail has a surgeon holding a chainsaw. The title is "Clean Code Means Good Good: The Horrible Performance Debate". But yes I think the title is misleading in a way, and I might change it at some point.
The chainsaw analogy would have been clearer if I'd added in "the scalpel" and made the connection that Casey's optimization reflects someone expertly using a scalpel to write great code. Some people assume I'm pro-OOP and anti-Optimization but that's not at all true. Your summary is my actual position: both are tools that have their place.
I'm definitely going to change the title for the written article. I'm not sure yet if I'll change it for the video. I haven't thought of a new title though. It has to be slightly catchy -- this isn't a dry academic article. People won't even click to read an article unless the title grabs them a bit.
@@tobevasoftware If OOP had the massive productivity of a chainsaw compared to other cutting implements, then I could agree. But nothing indicates that OOP code is more productive to write than non-OOP code. You're comparing a well-optimised version of the code to OOP, while even the non-optimised sane (=non-OOP) code is already massively faster than OOP. All OOP does is lower the entry barrier of knowledge required to add new code. It costs productivity due to bureaucracy and it costs performance due to blackboxing.
I love how you use the analogy of a chainsaw for the OOP version instead of for the AVX vectorized version :D
But it's true, you don't do the yard work with your scalpel. It's also true that a chainsaw can cut someone open real fast!
I agree it does at first seem like the AVX code would be the chainsaw, ripping through lots of work quickly. But in my version the chainsaw is an analogy for the approach used to write the code, the chainsaw isn't the running code. So to me OOP is this harry-homeowner general-purpose tool, good in a lot of situations, but it's unsuited for writing really optimized code, it's too clunky and unwieldy. I wanted the failure case of the analogy to be vividly a real disaster, reflecting just how bad 25X is, so the chainsaw + surgery image was good for that reason. But yeah, slightly confusing.
A chainsaw is a power tool that vastly outperforms manual tools at just cutting through stuff. How does OOP outperform non-OOP in any way? The only saving you get is that you do not need to have any idea of what kind of things your program deals with, and what operations on those things entail. When does that ever happen? It basically only ever happens if you have user-provided plugins. But for everything else, the possible types of object you're dealing with are well knowable in advance, as well as the actual instructions each operation on these objects entails. The only sane argument for OOP is: "The programmer is mentally overwhelmed by the amount of different kinds of entities that exist, and has no clue what each abstract operation on a class of entities concretely entails for each deriving type, so we completely blackbox everything that has to do with these entities." It is not faster to write, it is not faster to execute. All it does is lessen the mental load expected of the programmer, but also prevents any real understanding from occurring or paying off. OOP sets a hard cap on program performance and programmer productivity, and thereby caters to unskilled or mentally disabled programmers and artificially confines talented programmers in their productivity and in the quality of code they can produce. It's basically the communism of programming and instead of uplifting bad programmers, it drags everyone down to the lowest common denominator. So your chainsaw example is wrong.
If good code was easy...
Good video. I personally don't really care about the topic because everyone has a different definition about clean or dirty code. I just want the code. 😅
You should care. In the end, we're all trying to expand the codebase while mainaining performance and without obfuscating the logic. You'll never enjoy looking at somebody else's code. That's why you should try to make the experience as painless as possible.
Awesome video.
I'd never heard of the term thought terminating cliche. I'll use that to defend myself against lazy thinking.
I like how you we quantified when each approach makes sense. So much of programming feels like it's based on vibes, and it's good to remember that we can base our coding style on the problem at hand.
So use the right tool, for the right job, yup math checks out.
If you just focus on writing good code, with correctness, performance and simplicity in mind you'll find that you use the appropriate programming paradigm at the appropriate time. Ultimately the problem space is king.
Clean code is a code, where you can easily spot bugs
👏