Always rewrite EVERYTHING! - FunFunFunction #20
Вставка
- Опубліковано 20 сер 2024
- 💖 Support the show by becoming a Patreon
/ funfunfunction
Why do we have this impulse to always re-write? We explore the feeling a bit, thinking about why our intuition is often wrong about re-writes, about the allure of the clean slate, and that rewrite seem a lot better because you're comparing the reality of a current codebase to a utopian fantasy about what could be.
"You have to remember that the existing code base, well, it exists, and your rewrite does not." - MPJ
We're going to need a rewrite of this episode, the weather could be better.
Hahahaha
+Lee Owen The weather looks glorious!
+Peter M for yeti
Also doesn't have any quarry shots.
I wish you could offer a philosophy or programming course through udemy or something. You are the man!
What MPJ was saying to me really felt out in the woods in the beginning but towards the end he really brought it back home. Nice video!
But the code is so much more satisfying after the rewrite :(
I'm in the middle of a rewrite that's taking much longer than expected, and you make some some annoyingly good points!
Great episode. These insight videos are very valuable. Many channels seem to make it about code... code... code... This is what makes funfunfunction unique.
I mean, it depends on the road you're travelling and the size of the application. For instance, if the fundamentals of the application are poorly built and you're scoped for a year of work... I'd rewrite. If you're adding a feature to a project that is scoped out for a month, I probably wouldn't. I found rewriting to help for the long term, whereas learning and squeezing in better suited for the short term. I'd also say responsibility plays a pretty huge factor. As a hired contractor I tend to contribute my work around what the team has already done. Whereas when I'm a lead I tend to rewrite a lot in a direction I feel more suited for the long term. Awesome video as always mpj. Thanks again and enjoy your Monday.
Great vid as usual. I will say that I'm for rewriting. Keeps you fresh, motivated and gives you a chance to evolve with the tech and the ever changing paradigms. The customer benefits from motivated developers who write code, rather than sift through someone else's old junk (as alluring as it may sound :) ). I do take rewriting seriously and carefully map the existing code (like I would for refactor) and I make numerous reality checks and buffer estimations. But then I sit down at a blank vim screen and start typing and that feeling is priceless and keeps me coming back to programming (much like your side projects my friend)
When you take your car to the workshop, would you consider it a positive thing if the mechanics changed your entire engine and charged you for it?
We all wish that the software industry was as mature and standardized as the auto industry (where you know how long every thing takes to make and there aren't a million bugs in a released product) If that was the case i would completely agree. However, the software industry is still very much evolving so I wouldn't use this analogy, plus, rewriting is what enables the development of our industry by improving each iteration. Finally, I'm sure everyone remembers this but always makes me smirk as its still true:
www.windowsnetworking.com/j_helmig/gatesgm.htm
All right, but a house then? Or a garden?
+funfunfunction This is a longer conversation and there is no one answer, but there really are very little areas that are as pervasive and unregulated or standardized as software development is. I guess the equivalent would be if my house was built from some homemade cement that is unsafe and the new builder can either reinforce it with sticks or rebuild it with bricks. Obviously, there are plenty of situation where I opt for keeping the old code base, but I really think rewriting is an important tool that shouldn't be ruled out just because it is something we feel compelled to do as developers and enjoy the deceptive peace of mind a blank project gives us :)
Ever considered next person coming to your code and rewriting it because it is someone else’s old junk?
This video describes my actions two weeks ago. I took a simple program, attempted to rewrite all of it in ES6 and perfectly functional and perfectly organized and modularized etc. and by the end of the week, the program was completely broken and I had to revert the project back to the commit from the previous friday. Wish this video came out sooner to warn me :)
Always appreciate your videos! Thanks again!
I love this episode. I was naively tricked by this impulse on my first ever big project at my first coding job. The fiction going in my mind seriously thought I could do a rewrite in 3 weeks... I really wish I could go back, implement the changes that they wanted and then just slowly refactored the code so it to could scale better and have improved extendability for the future. Quite embarrassing for me but I'm seriously glad to know I'm not the only one! ha ha. God knows why my supervisors didn't stop me! p.s. it's refreshing to talk about coding while walking out in nature :)
Awesome. I love it when you say stuff like "When I find a new codebase, instantly I want to do a rewrite". This is something I can relate to, this is a situation where I've found myself. Then, you process do argue and dissect that thought, with such awareness that makes me remember meditation.
Thank you for making fun-fun-function in video format, instead of podcast, because it's much easier to understand when I can watch your mouth, and there's a big plus for HTML5 players: 1.5x speed :) more like fnfnfnctn :D
Please keep them coming.
I like the exercise of thinking about what would be the benefits of the rewrite in question, what could make the code perfect, I try to imagine this without any constrain, from that I then go back cutting edges for what are the really weak points in using the current code.
Refactoring is always better. Continuous refactoring may safely lead to 100% rewriting of the codebase. That happened very often in my practice. Refactoring, if it's done properly presumes one change at the time. And that's the best thing about it: You can stop it after each portion of changes. And come back to it later. The second pro for refactoring is that it gives the better understanding of the domain, which can lead to further improvements in the domain level (that are not obvious when you are writing code from scratch). You will fell that leap, when you understand the domain much better then before. Such a happy feeling.
Thanks, very true. Next questions that pop in my mind: how to *recognize* this situation where a rewrite is needed (and all these cases where it isn't), and how to prevent yourself ending up in such a situation where a big rewrite is the only way out?
Came across an interesting, in-depth answer here: programmers.stackexchange.com/questions/6268/when-is-a-big-rewrite-the-answer
+Jos de Jong imo, necessary rewrites comes with technology shifts (new db engine, new framework etc..) most of the time.
Very useful read, this video and his latest made me rethink an upcoming company project, but this answer solidified the idea that is really has to be done.
In my experience I found a better decision path on when rewrite a program. I think the point is the if your code touch it's limits, and no matters the effort you put in it, you just can't achieve your next goal with what you have. This is the first point that should drive you to a complete rewrite. But keep in mind that you should always refine your code when you see it is possible, with refactoring small parts time by time. This doesn't save you for the limits of the architecture you have. The second important point is that you have a new solution that are going to fit your next problem (by "next problem" I mean the actual existing requirement you have to implement), while your current code base will not, as said before.
Technology shifts could be a reason, and that's why you should be careful when choose to move with a new technology. The danger you're going to face it to have the same staff in a different taste, with no real advantages and with a lot of time and effort spent. Anyway, I disagree that the "new db engine" should be a drive to rewrite your software as it is a symptom that your code is poorly designed.
+mpj
I suffer from the same awful condition - rewrititis!
The few times where I find rewrites helpful are when I want to introduce fundamental changes to my data-modeling layer because I suddenly realize that my abstraction isn't as elegant and wisely accommodating as I initially thought.
+Tal Weinfeld The same is happening to me this very moment.
What a fantastic points! A long while back I wrote a basic canvas multiplayer drawing app. I then wanted it to do more; fancy brushes, websockets, then try WebRTC, better UI, tiled panning canvas.... All that works. It's jQuery and Backbone slushed together, all whipped together over many very long nights which later seemed to ooze and smell looking back over a few months later. Backbone is no longer flavour of the month for views. A '2.0' rewrite with the dream of it being more maintainable and clean in my mind took shape again. I started tasking myself with rewriting all views to react components, all dom events to flux dispatches and stores (or whatever true flux means). Just loads of tiny mostly half useless .js files empty of all what my current app does are sitting in a shell repo with little real reward for the effort. That allure of a clean start has long faded. Fortunately I saw this video and it certainly reinforces my later doubts about starting completely over.
F. Brooks did say "Plan to throw one away... " but then later retracted saying it was too simplistic and doesn't fit with our waterfall model of making software these days anyway.
I remember watching this as a junior. It was a very important video for me.
This is so true!!! I fall / have fallen for this trap all the time...! Thanks,awesome video again
I think what often tricks people into thinking of rewriting is an idea like "This codebase has clearly grew evolutionary, it was not designed as a whole system from the start, but rather some "core", then some new features, special cases, edge cases, whatnot. So the original developer team did not have a chance to get the architecture right. But NOW WE'll GET IT RIGHT!, right? Since we know how this product was used for last 5 years and we can get the architecture straight!"
Something like that.
I really love that you upload videos in 60fps. I can't wait for a 120fps standard
Nice video .. Rewrites are like ego massage for us :) .. Also we have heard tales in our lifetime that so and so joined the company and rewrote 60% of the codebase and was a genius :)
+funfunfunction
By curious co-incidence, I didn't get to watch this until today because I was so busy re-writing code!
That said, I am replacing code that's hardly changed in 8 years and the replacement includes a whole load of new features that wouldn't fit within the original.
But yes, you're absolutely right, especially when it comes to completion estimates.
I'm fully rewritting a corporative software and having the "It would just take three months!" and taking almost one year to do all the rewriting stuff, but in my case the customer will say "Wow, thanks god they rewritten all this stuff!". In some cases when the codebase is old and full of workarounds, rewritting seems to be a good idea. There are some requests which were taking 4 or 5 seconds to complete, now it takes only 300ms :)
Perhaps the allure of the rewrite is also akin to our tendency to procrastinate as was mentioned in the episode on tooling? The rewrite, perhaps, is what Steven Pressfield refers to as the Resistance, or Seth Godin refers to as the lizard brain? It is that thing in our brain that keeps us from going after the really hard work that will move us exponentially forward.There isn't anything too scary with a rewrite. Our brains are wired to see it as being "safe", which is why we gravitate towards this. It looks like low hanging fruit, but as was pointed out in the video, it rarely comes without a cost.
No, I don't think that's the biggest part of the allure. It's more likely just that the concept that you have in your mind is a lot more attractive than the code that has already been written, especially because it's imaginary.
It is actually really hard to try to talk yourself out of rewrite if a client wants it. Even when you try to convince that refactoring would solve the problem if the client has decided that the only option is a rewrite.
I joined a company who had a project which was rewriting some js from v1 to v2. The rewrite was pretty terrible. Everything from manual concat files to writing things which would overwrite the core files for custom needs. Since I had to deal with it just about every day I advocated a v2.1 effort using best practices since no one would humor yet another rewrite. Took me about 3 months plus another 2 months to do a monitored and highly qa'd deploy. It's hard to say whether it was a rewrite or a very detailed refactoring as the logic does the same thing but looks completely different (and is very well tested). In this case, it was the right decision because now I rarely have to work on that code and very rare edge cases pop up like every 3 months which can be patched in nicely.
done 5 rewrites of my Node cms just to understand cms's chalenges that's a good idea.
The code written by you, me, and every programmer a few seconds ago is already a legacy code.
Great topic/video! I feel like a good, related topic could be something along the lines of "Introducing and Mixing too Many New Technologies w/ the Old".
Example: you are in an old codebase. Things are ugly. Old technology is in place. Maybe there is a big new feature to add. Build the new feature in almost a whole new stack side-by-side the old system. Repeat. In this extreme example, you might end up with jQuery running one feature, React under the hood of another, and Angular in charge of another etc.. etc... (also applies to backends) You now need to know and maintain several distinct sets of technology. You now are having a bad time.
Sometimes re-writing helps you understand the code better. The re-write may not actually be as good, but you understand what is going on.
just wanted to say I really appreciate your content. I've learned a lot and its really fun. thanks for doing this and keep up the good work :)
I think that most of the time, re-write take less mental workload and make me feel less frustrated. So even if it take more time to re-write, my brain still choose to do it. It was easier and make me less stress.
We have to do some reality check. Refactor bad code is hard and take more mental workload but there are about 80% chance that it actually take less time than re-write.
I was just about to do a rewrite. Now I think I'll just refractor :)
This is great, very relatable. Thank you for sharing your experiences
These videos are incredibly useful.
Haha so good!! And indeed so true :) I have precisely the same urge whenever I see a new codebase haha it's like I want to "rewrite it in my own words", write it in away that pleases my own aesthetic ideal XDDD
Thanks so much for your movies!!!
I will go a bit further saying that "rewrite" works only when it's done gradually over a relatively longer period of time. It's like refactoring with the intention of rewrite in mind. I did two rewrites of a POS and a billing application this way. One glorious day you will find there is no old/legacy code remain in the application anymore.
I do rewrites too, and now i know: 'i'm not alone(crazy)'. Great video mpj. :D
Last week i did porting (rewrite) of one module from another system. I did an estimate of 1-2 days but spend whole week and even some weekend days. So I really wish i got this video 1 week ago.
I really enjoy your Videos.
You are a great Mentor.
Thanks.
Imo, you should always try to work on the old code for about a year. That way, you learn the pitfalls of the project. If after that year, you're still not satisfied, you can venture a rewrite. But usually, you'll effectively have rewritten the project on a module to module basis by the time that year comes.
Thanks a lot for all yours videos especially those which express personal concepts! You are very fun fun!!! Next week I'd like to start a new personal project related to spotify, if something good comes out I'll ask for your noble opinion!
Its the little devil on the shoulder. I have learned that juniors always offer this, and not to totally counter it but if the code stack you want to write is too large and NOT core business always depend on others packages as no software, even your software will be good as soon as you are done, stuff will need to change, so lean on others skills too.
I'm in the process of doing a refactor at work. As much as I wanted to do a complete rewrite, I knew it wasn't going to happen. However I am estimating our refactor to take at least a year to get it to where it needs to be.
I think this video came at just the right time for me
Thanks
that's amazing isn't it. Found the fun-fun-channel at the right time for me
Yeah such a coincidence. We're still going to rewrite the code base we have =P
Refactor, not rewrite. it's difficult, but watch a video of/by Uncle Bob (Martin) and you will be more motivated too. MPJ is inspiring and brings confidence , uncle BOB is lecturing and brings confidence.
I know but in this case we just have to. The codebase is Java/Android and we want to write a Xamarin.Forms app to deploy to iOS too. Apart from that the Android app doesn't even work and whoever wrote it abused an eventbus to the point where you can't even make out what happens when.
Trust me I would love having less work and staying on Java but it's not possible in this case. Also I don't call the shots :P
This is insanely accurate.
Another awesome video :). Any chance of a video explaining monads in JS?
Nice episode mpj. Enhanced by being out and about in Sweden!
Wise words! /fellow rewriter
Sometimes it amazes me how much I still have to learn about my brain, and I'm 30!! Psychology...
I have been thinking about this a lot lately. What do you think about taking an existing code base and testing the hell out of it? Then doing the refactoring with a margin of safety. I wonder whether that is the best approach.
I'm in the middle of a rewrite now. I do regret it lol. But the code was not testable the way it was written.
Wise words... why are we so irrational?
Is typing code the only way to tell a computer what to do? Is there a better way?
I wrote a 2d game on sdl2 almost a year ago. It was the hardest thing I could write. Now I am thinking about rewriting it now in libgdx. It is about 2000 lines of code. And I had to make my own (shitty) implementations of Sprite, Button classes and write "physics". Libgdx has a lot of that already. Should I rewrite my game or refactor it?
p.s. you can see my little game on my channel (it is called space tinker)
Maybe its small(compared to real world) scale somehow affect the decision?
+Alen German You should re-write in this case because the tools you used origionally are crap. I think he's talking about re-writing someone else's code because you would like it to be written your way rather than the way it was written.
I think you might have a slightly biased perspective here, MPJ. You seem like quite a senior in the field and I think it's safe to say that you are able to get some jobs at some really cool companies. Companies that had some funds to at least hire a team of somewhat experienced people that could plan together. Now of course that doesn't necessarily mean anything, because even highly funded software projects can go southward. But.. when you're not a senior and end up working for a start-up that's running a software product that has been developed several years ago by people (or one person) that were still learning the basics of the language, with no planning, structure and constantly changing requirements, this really is a different story.
Given by how educating you are I think it's very likely that lots of juniors are watching your channel too. And lots of those will end up working for start-ups first, where they might get into a project that was created in a hassle by a not so proficient person. Therefore, I think your estimate of 1% is way off. Indeed, in most cases the intuition of making a complete rewrite is wrong, but this figure is too low.
I'm in between junior and senior myself, though I still work for a start-up because I like their unique product. But this is a game platform running on a 20+ page app on jQuery. Most variables are named a,b,c, thousands of lines of code in a single file, functions of hundreds of lines with deeply nested callbacks. Over 50 JS files, everything is global, no packages. Countless redirects and backend dependent page loads between pages. Running ESLint through it (without style correction) shows a solid red bar at the right of my VScode window. The configuration dashboard is made in React and is of similar quality with render and business logic deeply intertwined.
After 6 months now I know where to find stuff in this system, but it will never get familiar because it truly is rotten to the core.
Forgive the criticism, but nobody is going to convince me that a rewrite is not due here. And I really don't feel like the unlucky 1% here. There's some intensely bad codebases out there that high-end developers like you are lucky enough not to encounter as often.
Don't get too stuck on the 99% figure to the point of missing the entire point of the video. I should probably have said "almost always" in order to not nerdsnipe people but please try to listen to intent and not get stuck too much on semantics. :) For the record, I've been coding for 20 years or so, and I've been through many different code bases. While some people get a job at at company such as Spotify, I've worked my way up through code bases that are infinitely bad. Some code bases are indeed necessary to rewrite (but in those cases I honestly think it's much much more efficent to create a completely new product, as you will spend so massive amounts of time replicating the exact business logic of the original code, and most likely be forced to surgically import parts of the existing codebase).
Oh yes I agree with you wholeheartedly about the flawed intuition about wanting to rewrite everything.
I'm not sure if creating a new product is also not just a form of rewriting, but hey, I might be nerding too hard here again. But as coders we should focus on semantics, should we not? ;)
Rewrite and new product is very different - the big difference is expectations of feature parity from customers and product owners. Trello is a new product that Fog Creek software created, but it doesn't have feature parity with their existing product FogBugz.
Nice video mpj! So I was wondering, in which cases you think that make sense to do a rewrite?
+Nícolas Iensen The only point in my career that I look back upon a re-write and don't think it was a mistake was when I re-wrote an old binary that was written in Cobol or something it "ported" to .NET. I basically took an input file and emulated the output in a new language. That was a needed re-write. In all other cases (and they have been many) I think that I was actually just doing it for my own sake and not for the best of the project.
Yep nailed it :)
What happened with the Pomodoro timer?
This is just a question. Are you thinking on going back to the old format of videos with you coding?
Dude, I did a 30 minute coding episode last week
I know i know. Don't get me wrong. I just missed when you teach ways of doing things like you used to do.
+funfunfunction are you considering finishing the pomodoro application in an upcomming video? By the way loving the content you're putting out lately!
+Pablo Andres Dealbera Theres already 1000's of hours of people doing code tutorials yet no one talking about programming as a general concept. Much prefer this format. I mean why would I wait a week to learn something when I could google it in seconds
your videos are awesome
\
In what instance *is* a rewrite appropriate?
A very late reply, but I inherited a project that had 4-5 previous full stack developers. From what I saw, none of them even tried to understand the original code base but rather did their own thing.
I told the guy who owned the site that I will not work on it unless I get to rewrite everything.
My reasoning is:
* In current state, I would have to spend a lot of time on figuring out when and where the different developers crossed their path, hacked something together to make it work.
* In current state, debugging would be a nightmare
* In current state, different sections of the site use different technologies creating a unnecessary overhead
* In current state, none of the naming conventions, folder structures etc. made any sense
* After a rewrite, there is ONE mindset, ONE structure, only the technologies that are needed. A clean slate.
The guy said "naw, we put so much time and effort into this, just make it work!"
So, I didn't do jack shit and kept my sanity.
In short, when you are handed a dumpster fire of code mess.
Where is the border between refactoring and rewrite?
+moofymoo This is the way I see it: A refactoring is just a constant improvement of the current application that customers uses today. It will keep working at every incremental step that you make in the refactoring. A rewrite is something that you create side-by-side with the existing application, rewriting from scratch, until it is deemed to be on par with the existing application, and then you do the switch.
What if the application is running on a 'legacy' framework? That will most likely lose support soon? Is it just to do a re-write then? Or should the components just be refactored in to the new framework?
That doesn't happen frequently. At least it shouldn't. I you choose your tools wisely you will probably have tools with long life span. As he said 99%.
That's true, but for example we are re-writing an application from plain js to Angulr 1.x now. And the problem is, if Google decides to drop support we'll have to do the re-writing again!
+Nipuna Gunathillake I agree it is more difficult in JS world. But the decision to write application to framework for with exist new alpha/beta version is not quite wise as you see.
lol you are cold!
could we apply for programming job without certificate ?
Certificates are useful for some things
1. Making money for the people selling them
2. In case you are out of toilet paper
3. Pretty
Not very useful for:
1. Getting jobs
lol .thank you
funfunfunction
thank you hahahaha
does refactoring not considered "rewrite"?
+pluraltest in this context I use the word refactoring as taking a small piece of code in larger codebase and restructuring it in order to gain some performance or maintainability benefit, while still retaining 100% functionality. A competent development team has automated unit tests, system tests, and a small manual regression test suite in place to ensure this. When I say rewrite, I mean re-making a component (such as a complete microservice, for instance) or entire product from scratch while the old version exists decoupled from the main product.
For 1% of instance that you need a rewrite among the 99%. How can you tell that you need a rewrite?
You you need to ask you don't need a rewrite. :)
One more thing. Is there a consideration?
- Like Code based is hard to debug and followed along.
- Developers will be more productive to use new frameworks.
uhm, 1% really? What data do you have to able to say that?
When commenting on a video that is nearly two years old it helps with a timecode. :)
I agree with the video. But the feeling of refactoring almost always is a bad idea, as you said, it feels weird to say that without proof. I dunno.
wise man said :+1:
REFACTORING!!!!!!!!!!!!!!!!!!!! T_T
Hello, MPJ, I think you are giving us a bad example. Walking and talking to the camera at the same time - not a good habbit ;-)
You clearly haven't dealt with much code from contracted outsourced code or using completely outdated frameworks. Code that has no optimizations or testing - it worked once and was sold and forgotten.
This is exactly what I have worked tons with.
The quality of the work you're seeing must be much higher. It's been common for me to see contracted code like a "server side backend" that's 1 PHP file consisting of else-if's with blocks hundreds of lines long, using mysql_query() using direct $_POST variables.
Love the videos, but please, for the love of God, stop using that backing music
Not a fan of that misleading title.