Types Of Technical Debt And How To Manage Them
Вставка
- Опубліковано 27 вер 2022
- Technical debt is a useful and sticky idea. Managing tech debt can be very easy if you stay on top of the 'payments', or one of the most difficult problems in software if you don't. So how to manage technical debt is a big topic.
In this episode Dave Farley, author of best selling books "Continuous Delivery" and "Modern Software Engineering" talks about the different types of technical debt, where the analogy with real debt makes sense, and where it breaks down, and how to really solve the problem of avoiding serious, damaging tech debt.
-------------------------------------------------------------------------------------
🔗 LINKS:
📄 Martin Fowler’s Description of Tech Debt: ➡️ martinfowler.com/bliki/Techni...
📄 Wikipedia’s Description: ➡️ en.wikipedia.org/wiki/Technic...
-------------------------------------------------------------------------------------
Also from Dave:
🎓 CD TRAINING COURSES
If you want to learn Continuous Delivery and DevOps skills, check out Dave Farley's courses
➡️ bit.ly/DFTraining
📧 Get a FREE guide "How to Organise Software Teams" by Dave Farley when you join our CD MAIL LIST 📧
The best way to keep in touch with the latest discussions, events and new training courses, get FREE guides and exclusive offers. ➡️ www.subscribepage.com/organis...
_____________________________________________________
📚 BOOKS:
📖 "Continuous Delivery Pipelines" by Dave Farley
paperback ➡️ amzn.to/3gIULlA
ebook version ➡️ leanpub.com/cd-pipelines
📖 Dave’s NEW BOOK "Modern Software Engineering" is available here
➡️ amzn.to/3DwdwT3
📖 The original, award-winning "Continuous Delivery" book by Dave Farley and Jez Humble ➡️ amzn.to/2WxRYmx
📖 Extreme Programming Explained: Embrace Change, Kent Beck: ➡️ amzn.to/2GpQRjE
NOTE: If you click on one of the Amazon Affiliate links and buy the book, Continuous Delivery Ltd. will get a small fee for the recommendation with NO increase in cost to you.
-------------------------------------------------------------------------------------
CHANNEL SPONSORS:
Equal Experts is a product software development consultancy with a network of over 1,000 experienced technology consultants globally. They increase the pace of innovation by using modern software engineering practices that embrace Continuous Delivery, Security, and Operability from the outset ➡️ bit.ly/3ASy8n0
Octopus are the makers of Octopus Deploy the single place for your team to manage releases, automate deployments, and automate the runbooks that keep your software operating. ➡️ oc.to/Dave-Farley
SpecFlow Behavior Driven Development for .NET SpecFlow helps teams bind automation to feature files and share the resulting examples as Living Documentation across the team and stakeholders. ➡️ go.specflow.org/dave_farley
TransFICC provides low-latency connectivity, automated trading workflows and e-trading systems for Fixed Income and Derivatives. TransFICC resolves the issue of market fragmentation by providing banks and asset managers with a unified low-latency, robust and scalable API, which provides connectivity to multiple trading venues while supporting numerous complex workflows across asset classes such as Rates and Credit Bonds, Repos, Mortgage-Backed Securities and Interest Rate Swaps ➡️ transficc.com - Наука та технологія
“Knuckles” 😂
I agree. Problem is you have to lock managers( or customers!) in room and play your videos in an endless loop until they understand. Otherwise you get technical debts by command.
I have taken some heat before when joining projects that were in bad shape and the prevailing opinion was that they wanted to start over. I say, "you'll just make the same mistakes over again". I read a great story about a professor who took a position in a college for the electrical engineering section. The lab for it was in terrible shape, the instruments were all broken. The administrator asked the new professor to make a list of needed equipment and he would see if he could find the money for it.
The new professor replied "no, not a problem. We will use what we have". The administrator left, stunned. The new professor started his classes and took the new students out to the lab. Over a course of months, they took apart the broken equipment, got schematics for them, and went over what was wrong with each instrument as a group project. Slowly but surely, they got most of it working again. The students that did this because some of the best engineers the school had seen.
The moral of the story is applicable to software rewrites. The team that abandons the software and starts over does not learn anything from the existing software, even if they didn't write it. They create a new big mess to replace the old big mess. Contrast that with a team that is forced to refactor the code. They learn the mistakes of the code, how to fix it, and, perhaps more importantly of all, become experts at refactoring code.
In the last 2 years, I instituted a goal for myself that I would track down even "insignificant" problems in my code, and go after the hardest problems first. In that time I have been amazed at how often a "trivial" problem turned out to illustrate a deep and serious error in the code. Similarly, I have been amazed at how solving hard problems first makes the rest of the code go that much easier.
I have always been a fan of continuous integration without calling it that. I simply always suspected that the longer it took to remerge a branch in the code, the longer it would take to reintegrate it, vs. small changes and improvements taking a day or so. I can't take credit for this realization. Too many times I have been assigned to merge projects that were complete messes because of the long span of branch development. As the old saw goes, the better you perform such tasks the more of it you will get, especially if others show no competence in it.
Great story, thanks. Absolutely, I am a natural "improver rather than re-writer". The one caveat I have on this is my friend Martin Thompson, who I worked with for a few years doing some cool things. This is much better than me at knowing when to re-write and when to evolve. I nearly always evolve, and think it the less risky strategy.
I've worked on more than one project where the unmaintainable ball of mud actually WAS the rewrite of an even worse system before.
i partially agree, for one must have a mind of steel to not be biased on the wrong or ugly code, or become lazy and let it just be there, it is working after all. i think the more you interact with this kind of situation, the more you start thinking like that. sometimes fixes are good, but creating better solutions is often better. also, rewriting ugly, smelly and bad code is actually writing it anew, so it may be conceptually the same lol
I tend to agree with you, I've started a personal virtual pet project back in 2020 and have gotten to the point of it being a big ball of mud, and though it's been keeping me from working on it for months on end, I have zero intention of starting over. My intuition told me that to really improve as a programmer I needed to sit down and interact and confront the big ball of mud, until it becomes a vast garden of well-written features. I wonder how you feel about stories such as Satoru Iwata's rewriting of Earthbound from scratch though. (Basically the game's code was in a pitiful state and when he arrived he said "We can try to fix it and it'll take an entire year, or we can rewrite it and it'll take a month").
Sometimes there is not enough time to rewrite the code and there is no approval for it. So, this will require you to be able to squeeze out the appropriate code out of you without crashing the project and adding new technical debts...
My favorite concept is “code viscosity”. That means how easy it is to make changes and implement new features the correct way and it’s by far the most important thing when designing an architecture.
Knuckles is a great name for a testing framework.
As an Architect in my company, I've had arguments about how Technical Debt effects our productivity. Sadly since there are some that don't listen to the advice I and others give they tend to prove time and again they are not borrowing from a friend as Knuckles keeps showing up.
I'm currently working with a client whose head of engineering proclaims it's not worth fixing technical debt because everything turns in to technical debt sooner or later anyway. Absolutely jaw dropping.
Uncle Bob mentioned a project he worked on for a while where his team was developing the same program twice - one was the old, broken mess, and the other one was the refactored, clean version. Feature-wise, the refactored, clean version was 1-2 years behind the main version. They were simply cleaning behind the mess of the previous devs, not adding anything.
They finally catched up _20 years later,_ though this is a lie, and they never really catched up. They told the client that it "sucks to be you", bricked the new/dirty version, and just forced them to use the outdated, refactored version. Nobody was pleased.
Technical debt is my life. Paying it down is an art that not many developers can do. Heck, merely not accumulating too much debt is an art that not many developers can do.
I think the biggest problem is that most devs paint technical debt with the same brush, when there's so many potential tech debt indicators that you can easily define the levels of debt vs repayment.
Understanding where Technic debt is fine, and where it's absolutely 100% disallowed is key to using it effectively
@@daniellarson383 yep
I think there's also the reverse of technical debt, technical credit or something like that. It's putting a system in place that you know is going to pay dividends later and during the livespan of the solution.
I've heard people deem that practice as "not agile" - spending time working on something that's not needed now as things can change very quickly. However, in real life I know it can be a good thing to do this.
i would call it technical investment then.
Amazing, now I can finally put a name to my struggles of the last weeks! Thanks for bringing this up!
I'm glad that this video clarifies my subliminal experience as Software Reliability Engineer. Sometimes you just don't have enough time to think about such things.
I remember one product which have had bad design with a lot of bottlenecks with long maintenance time to fix these technical debts... This was pure nightmare.
I was not fired as previous guy just because I consistently noted every bottleneck and reported them to the responsible team (as the result the list was huge and we have migrated to the other service), when the previous guy didn't do that and I believe this was the reason that he stuck in the "swamp" of technical debt of the external service team and was fired.
What we need to do is teach accountants how to write technical debt in the books. That way the shareholders would see for themselves what a terrible state the company is in, and the C-suite can make better prioritisation.
I think that dev teams need to own this, because Tech debt is contextual, but yes it needs to be budgeted for. The measures to avoid tech debt are already known - Stability & Throughput, the measures often referred to as the DORA metrics and described in the book Accelerate. amzn.to/2YYf5Z8
I agree with Salvatore.
The finance team lacks a mechanism to charachterize the debt, so the result is that "expenses" under represent the actual liabilities.
Best explanation of technical debt on UA-cam. Thank you very much
As usual ! Fantastic video ! I love your content! And your description of tech debt !!
I didn't know this was a thing or that it even had a name! Thank you! As an illustrator whenever people tell me to take the simpler/easier approach that seems easier to them I tell them that later on I wouldn't be able to implement changes or would need to spend more time fixing things if I do that, most don't get it but I know not to fall for their advice. I've had similar types of advice be given to me the few times I programmed some small games for myself and I am glad I did not take their advice, the one time I did that I regretted it big time.
This is his best video yet. I'm going to pirate it for my next Teams meeting.
This was an awesome episode. A few things "clicked". Thank you
Fantastic tutorial... I absolutely agree... I worked with so many teams accumulating tech debt and never taking the time to repay it that the systems became such a garbage that putting out fires became the no 1 problem and priority and release of new features almost guaranteed having new visible bugs thus having to put out fires again... 😊
Thank you Dave, great explanation 😊
This is a superb career advice and orientation video for various experience levels: recognize and discover the potential deficiencies in a prospective employer, and decide for yourself whether you want to be there, or set the stage by proposing to help them move away from harmful ideas.
Another great video Dave, thanks.
Two things you didn't mention that I think are overlooked in technical debt discussions
- Technical debt associated with switching developers (ie replacing the development team)
- Technical debt associated with switching frameworks. (ie, let's rewrite in framework X, it's the latest thing...)
There's a ton of technical debt, when a new developer and a new framework looks at a change in a system they are unfamiliar with, no matter how the original system was written.
Great summary of the topic! I liked your comments about the blend of maintenance effort and new feature effort - that's one of the topics where you have to walk the walk in order to really get the importance of it (especially important when working in an org whose accounting team manages capitalization of the costs).
I think there's a part of the story that you might consider following up on which is how important learning is in the debt dynamic. Most engineering teams will try to do their best to deliver high quality code. Sometimes that team doesn't realize the level of debt until they learn something. If you watch Ward Cunningham talk about technical debt, he underscores that effective engineering means building to your best understanding of the requirement. But the reality check doesn't happen until after you deliver the code. The sad part is that many orgs aren't really built to drive effective (timely) learning, therefore increasing the debt.
great explaination, thank you
The techdebt that I can remember seeing falls in two broad categories:
- The conscious corner cutting, with a record of plan to make things better. The bank loan type. This is quite uncommon.
- The un-conscious poor design that only gets worse as people's fear of refactoring grows exponentially. Loan shark type. This is by far the most common type of technical debt, and it requires strong leadership to be addressed if you want to avoid the alternative, which is the product going to the rocks.
When I started watching this I expected banalities along the lines of “managers suck” and “they don’t get the beautiful code”. But it was much more deep than that. Thanks!
Thanks for another very enjoyable video! I prefer to think in terms of code base stability. The term technical debt reminds me of the IBM mainframe term for program crash: abend. As in abnormal end. Your program has abended they would say. It is not wrong but also not very descriptive of the situation: you have confused the processor, it has given up, dumped its memory, and continued on its way. Technical debt just does not describe the escalation of problems that can happen in an unstable system. I have made a career of refactoring code bases into stability. That is most of the task on the 'system' side: programs that run other programs. Most of the effort is in coming to understand what is there and how and where it can be improved.
Just went through the first day of a 3 day PI planning binge for 4 dev teams on single product… talk about tech bankruptcy.
Absolutely brilliant video! Thank you for speaking on this topic. I find your wisdom extremely appreciated
Glad it was helpful!
great video. technical debt is a great trigger to being pissed later. revisiting code too often is really boring
13:40 "our code is just the current snapshot of the continuing evolution of our system" *WORDS OF WISDOM*
I've worked with more technically bankrupt companies than not. It's frustrating, it's always a situation where I don't want to be there. I had to hop jobs a few times to get into short-term contracts.
Every company gives you a half-truth sales pitch about themselves in interviews. Same as we all do when dating. I'll be honest, I don't know how to find companies that take code seriously.
I had a go at trying to suggest some ways of detecting competence: "20 Questions you should ask you next employer" ua-cam.com/video/2Afk9KVEgpE/v-deo.html
Absolutely important term, unterstanding which you can also easily draw parallels with "waste" in the lean production.
Want to make you system healthy and move faster? Removing the bloat may be the easier measure, that trying to add complexity to make the system more powerful to digest all that bloat.
It's like air balloon, if you want go higher, you can increase the heat, or drop of the ballasts.
I've seen it for over 30 years, but I never cease to be amazed how many companies will allow the same developers to make the same mistakes on every system they touch. Worse, when they teach those bad ways to newer developers.
I think that a more direct analogy to technical debt is skimping on your health. You drink cola to "keep it going", rather than take a nap. You eat shit food because it's faster to order out. You sit and stare at the monitor because you have to "git 'er done" rather than take a walk to clear your head. Eventually you are still late, feel like crap, perform worse, and now can't recover your health.
I think we should tackle technical debt every time we see it, NOW. Tomorrow never comes, because there is always an excuse. Maybe a complete rewrite isn't practical NOW, but a crufty loop should be tackled immediately. If you don't have regression tests, (probably because someone said there were more important matters), do the extensive manual tests. Document the change and reason, and don't apologize.
The timing of this video is impeccable. (Released a hour ago) I've been in a meeting an hour ago. Team is afraid of taking on new solutions. Have been prioritizing feature delivery.
Very little consideration for design of how the system works. Instead it's just user story on user story.
This is because of the business pressure imposed on your team?
The idea of not having to repay yourself on a deadline is just an ilussion, you don't have to worry about new features that much but altering old conflicting code is the real issue, making changes is hard on quick and dirty code. For this reason, every day someone asks ou to change something, that's mr. loanshark knoking on your door.
I will someday create another account just to rewatch your videos Dave, and to give additional thumbs up. Those are words of sanity in my unwilling to change company.
👍 Thank you for your endorsement!
Have being listening to this name lately at my job
I've previously worked on a project where we were explicitly told to take shortcuts to meet deadlines that have been set before we devs were even involved.
Every time we said that this would produce technical debt that we'd then need to fix asap, and every time we got the answer "Sure. You'll get the time for that after this final deadline!".
This went on for a couple of cycles with us developers resignating at some point, until the system finally broke because of a cascade of previously known issues resulting in the production system being rendered completely unusable.
After that we were put into a" task force" to fix at least the worse issues to get it running again (day and night, including weekends). We were then told that of course it wasn't our fault since we warned repeatedly that this would happen, but several people overheard the management internally speaking about the "stupid developers programming crap" and also forwarding that information to the customers that used the software.
It was strange to see their surprised faces when we lead developers quit the company together.
I guess at least we learned something there.
@@gppsoftware Oh, the incompetence I've seen in management positions so far is unbelievable.
Problem is, that most of these people are at least rhetorically talented and they are very often able to talk themselves out of it.
Most of the management I talked about a year ago is still in place. They even got rid of some of the more competent people, putting less competent, but more obedient people in place that make them in turn look more competent and important.
But since they could not find proper replacement for any of the lead developers, they are paying me a very good price to help them outside of the business hours of my current job, allowing me to almost double my income.
But my new job is not really better. The management there overruled the developers' decision regarding the main framework to use.
Now we have to use something that only looks good on paper, is totally over engineered, and extremely inflexible plis hard to maintain.
It is of course also very hard to find anyone having experience with it and to learn it properly you have to take a lot of courses which we are not allowed to take, since as senior developers it is expected from us to be able to use it right away.
A colleague and me will quit this month and start out own company.
Since we can't find a properly managed company, we'll try it on our own.
Maybe we will also be bad at it, but at least then we will hopefully have some more insight on how hard this really is ¯\_(ツ)_/¯
Startups still have the concepts of firing the technical team once their ship is ready to sail.
Lots of good thoughts. One I would add is ego. It adds tech debt and most don’t even realize it. Sometimes is obvious by a decision made that one or two people think so high of themselves that they ignore warnings. Or sometimes is less obvious like with tenure engineers that must be right because they been around the block. Huge slippery slopes and all too common.
In a sense, all design is continuous and debt is built into all design. In addition to technical debt, a software application will also have product, UX, and UI debts. We may find a better way to do things when designing Screen 5, and this may require a revisit of the designs of screens 1-4. There are so many topics to explore further.
- Who grooms the backlog of all debt? (the balance sheet in accounting terms)
- How does the product owner add the business and user-facing valuation of this debt?
- How is the debt paid down within (or outside of) the story point calculus?
Fundamentally you have to surface the "cost" of the debt. SRE can help with this, identifying service level indicators, and setting service level objectives, and error budgets. If you start to move outside of your "error budget", then everyone has already agreed that this is important, so it is obvious that you must work to get back on track.
@@ContinuousDelivery I think unpacking the cost of debt could be a great future topic. Reliability is certainly part of it. Part of the cost is also borne by the product backlog and the speed at which we deliver future business value. Part of it could also be the emotional health of the team. Clean architecture, like a clean garage, could be its own reward even if there are no other economic gains.
@@doosrajawad Thanks for the suggestion, I will add it to my list.
The part of our code that is holds a high proportion of technical debt is a module that was put together by a contractor, using a coding framework. It goes without saying that this one reason why code frameworks should be avoided; they allow the quick development of a new application of major feature, but code maintenance becomes far more difficult; unless the framework designers envisioned any desired extension or behavior when developing the framework, there is a good chance that the extension cannot be added at all, and a high chance that the time it will take will be greater than if the framework had not been used. Frameworks only increase the number of things that have to be learned in order to get any real work done.
A lot of software shops inflict unnecessary technical debt by establishing deadlines for the completion of a change. All such deadlines are based on wishful thinking. Every program or code module, while it is being written, is being written for the first time. Any guess about how long it will take is a guess, which can be wrong.
In many instances the deadline is entirely unnecessary. If the new feature is being developed to make the product more appealing to prospective customers, and not to address the needs of existing customers, there is no need to inform anybody that it will be ready by a given time, and no reason to inform the customers that the feature is forthcoming. Wait until the new feature works as advertised, has survived all efforts to break it, and its place in the application is fully documented; then announce the feature to the world. The only technical debt consists of the bugs that got past QA (and there will be fewer of those).
Re-factoring can be done by baby steps, but it requires vision and perseverance. Instead of making a small change, testing it to make sure that it doesn't break anything, and then pushing it to the production branch, managers often want a developer to keep everything for a new feature in a separate branch until it is all ready to go, and then merge it into production. The risk and cost of each change is not linear with the size of the change, and so the huge merge is very costly it time. I've suggested doing major changes by small steps, but my supervisor shot this down because if the project is cancelled then it will take time to revert all of the changes; but should cancelling projects be common enough to warrant such concern?
Also, managers are often too afraid that a change will break things. If the fear is real, why hasn't the head of QA been sacked?
I want to preface my comment by saying I don't disagree what what you've said. I do find it interesting how far the original metaphor has been stretched to other concepts outside of what it was originally use for. (See ua-cam.com/video/pqeJFYwnkjE/v-deo.html)
The context of the original metaphor was to explain to a financial institution why developers would need to refactor (do work, that doesn't result in more functionality). It wasn't to write horrible code and then change it later (not implying that is what Dave is suggesting), but more about continuously changing the software to reflect our understanding of the domain. If you don't that would continuously trip you up which is like needing to pay interest on a loan.
The idea is you get software out there to gain experience in the domain even when you don't know enough (loan) and then refactor to incorporate what you learned into the program (pay back the loan); otherwise, it would be like you never paid back the loan and you wouldn't get the benefits, but only the downsides. That means your code needs to be "clean" enough to be able to refactor the things you learned back into the code. So dirty/horrible code would be against the original idea of the debt metaphor.
That means the motivation of the metaphor isn't an excuse for bad code or short cuts per say (again not implying Dave is suggesting), but about continuous improvement and feedback, which I find more helpful.
I see that Dave addresses tech debt from the perspective of SERVICING the debt (borrowing and repaying), I'm curious if anyone has looked at it from the ACCOUNTING perspective. If I'm borrowing time, then I'm carrying a liability; and if I'm saving time I'm creating an asset....
If you do a root cause analysis on technical debt, you will almost always find that the root cause is obsolete or mis-applied management practices. In other words, management debt.
Most companies avoid doing the analysis. If the analysis is done, they avoid dealing with the root causes, and focus exclusively on the surface technical causes. That means, sooner or later, often sooner, they are back in the same situation again, with high technical debt.
Ego, "not invented here," job skill building for the next job, fear of effort, resistance to leaving comfort zones, and real fear of being misled by unrealistic outside vendor promises are all big factors in technical debt. Technical debt is more an emotional lifestyle problem than a budget or time and resource problem. Many developers, I was certainly one, are unwilling to fairly evaluate other options. Sometimes we are afraid to look like we don't know everything. Our price is hurt if some "off the shelf" solution is bought instead of going with a build from scratch approach (I'll build my OWN javascript libraries. I'll build my OWN network discovery tool. I'll build my OWN search engine, etc). Half my career was wasted building my/our own versions of well established, well tested, well adopted, cheap or even free tools.
A better analogy maybe that code is a deprecating asset. There’s a cost to build or purchase, a cost to maintain, and a cost to replace. Like a car or house, software ages, becomes more expensive to maintain and gains greater lack of modern features, and probably also experiences a decline in expertise to support it. The quality of the code is only part of the story.
I disagree, code is not really an asset. For example imagine two systems, that do exactly the same thing. Do you want the one that is 100 lines of code, or the one that is 100,000? More code is not better, so code isn't an asset.
Software is a complicated idea, in some ways it is not really the code that matters, it is more about what you can do with it. Obviously you need some, and you need some code, and the quality of that code matters a lot, because it is quality that makes it work in the first place, makes it easy to change and improve, and makes it resilient in production and in the face of change. All these things matter in a real sense, and in some weird ways, I'd say that the quality of the code matters more than the code itself.
@@ContinuousDelivery nah the analogy works. Let’s say the asset is a machine to do some task in a factory. Is the machine fit for the task? Does it fit within the budget? How much resources does it take to maintain? Now, let’s consider those same questions over time. As new machines come out with more advanced, how does it compare? Can the existing machine be modified and upgraded? Can you still get parts? Can you still find people that know how to maintain the machine? Can it still do tasks in a market competitive way? It’s the perfect analogy because a computer program is basically a digital machine. The best part is that companies know how to budget for the maintenance and upgrade of machines, but they often don’t know how to do that for software. More people understand physical machines because they can see and touch it. This analogy is the perfect parallel to allow ordinary people to understand that software needs to be maintained too.
The good thing with just using the more generic term asset, is that it can be adjusted to what people understand. Like maintaining a car or a house or a road. And all these things can be over or under engineered, fit or not fit for purpose, too expensive or not expensive enough, just like your 100 vs 100k LOC example.
I just left a company where all the code I'd qualify as tech dept. And no one even admitted that. 5000 line classes, hour long builds and "unit" test runs. Multiple teams working on the same codebase, making refactoring difficult and, in fact, prohibited.
Yes, I worked somewhere once that actively discouraged refactoring, I confess I was shocked, it's such a terrible idea.
100% true
This is like therapy
Any idea how to measure debt? To see progress over time.
If you have data on the build test an so on then you have a chance to collect it over time and anaylse it.
Low code / no code platforms cumulates or reduces technical + financial dept?
I think it makes the chances of code debt worse, but the consequences are probably lower, because the low-code solutions are usually a simplified version of what you could do, so the ways that you screw up are a bit simpler too. So "Bank Loans" rather than "Loan Sharks" on the whole.
One time in a meeting i said we need to remove the technical debt, the project manager said: don't you dare to say the word "debt" to our customer, he could easily misunderstand it, and we will get financial problems :)
The analogy breaks when you think all companies work on debt: they take loans and build factories and pay that debt; had they worked with saved cash nothing much would get done
Technical debt is not cruft or bad code or bad designs: everything is technical debt, unless you can anticipate everything in advance. When you need to update your services to HTTP2 all that becomes technical debt. When the market moves on and your clients needs to do something else with their shop suddenly a lot of technical debt appears. You could not write software without adding "technical debt".
That sounds like you define technical debt as "anything that needs to be changed". By that definition you are right. Everything becomes technical debt as soon as some change is needed. But then the term is useless. The definition is more along the lines of "anything that is harder to change than it needs to be". This definition is admittedly very blurry, because how hard does something need to be? But at least you can draw the line somewhere and then the definition becomes meaningful.
@@bernhardkrickl5197 "But then the term is useless." exactly
we should say it is "bad code" and be done with it ...
Perhaps "Knuckles" should be on the board of every corporate?
The problem is that "Knuckles" has very limited tools at his disposal, and he often is on the board of most corporates. I was once in a meeting on a big project, hundreds of people, 10s of millions of dollars, it was a project that was going disastrously badly. We have been working 80+ hour weeks for many months, and after 18 months of development, the code didn't even all compile together. In this meeting some idiot senior manager said "It is time to 'bring the hammer down' and work harder". I confess I was rather unprofessional at this point, and called him a rude word.
Do you have any thoughts on how to help teams recover from technical bankruptcy? I've seen a couple of codebases where the technical debt has gotten so bad that getting the code back to a workable state seems completely impossible, although I feel that it should be possible somehow
I wrote a long reply on how I did exactly this, but UA-cam threw it away instead of warning me it was too long for them to keep. I'd call that some technical debt. Anyway, I'll try a too-short-to-be-accurate guide, in separate comments to follow.
My approach is summed up by Thoreau, the movie "What About Bob?", and Londo Mollari from Babylon 5 -- "Simplify!", "Baby Steps!", and "No, take care of it NOW!" Debt often starts with innocent little bits of cruft that obscure and add pointless complexity. The author then needs to accommodate that cruft, adding further unnecessary complexity that further obscures. So just as that junk built up a bit at a time, we'll tear it away a bit at a time. Instead of doing a massive rewrite on day 1, we'll fix easy, trivial stuff first, test it, comment it, commit it, and repeat. Each simplification will expose more things to simplify. I used this approach to fix 40k lines of uncommented, byzantine Lisp code in a production system that by law had to be 100% correct. That's not to brag, but to say that it works, even in tough circumstances.
1. Set up a regression test mechanism. Use plain text to formulate the tests' input and output (where you can). Put each test in a separate file and make it so you can run them one at a time, or a directory at a time, or recursively. That way you can use git to version these tests, and use Jira names in commit comments to synchronize everything. That way, when you make a change, everyone can see what it is and why you did it. Format the data so that you can see changes easily -- visually, with simple tools like diff, and by parsing it into data structures to analyze with code. Key-value pairs and trees are very helpful, as they format nicely and don't lose logical connections. Lisp s-expressions are super great for this, but if you must, p-lists. The faster and more easily this runs, the cheaper every code change becomes, and that is vital.
2. Pick a file and reformat it so it's pretty looking, and the comments are all accurate. Test, comment, and commit. You can do these changes in any order, because the regression test should show no changes in output. Repeat this for at least one directory of source. It's a cheap victory, and helps you understand what's being done currently.
3. Pick a function and all the code that uses it. Try to start with one that is only used by a few other functions. If it's not used, remove it. Now look at every aspect of that function and fix whatever you don't like. Is it named well? Are the args named well? How about the loops inside? Write the best version of that function without changing how it's used. Test, comment, commit. Repeat as needed -- the short-term goal is to remove the 'dust' that obscures what is being done inside each function.
4. After doing Step 3 enough, you'll uncover where variables and functions are being defined too far away from where they are used. You'll see where functions used by only one other thing do or don't need to be separate. You'll see state variables that are being maintained but serve no purpose -- someone just kept doing it because they couldn't see through the dust and realize that state was irrelevant. As you find these things, fix them, test, comment, and commit. Move definitions and client code closer together -- this minimizes compiler ordering issues, and lets you see more things in one editing buffer. That alone save lots of brain cycles.
I am in a team now were we have 4 products that look 95% a like function wise "classes". I mention should we not analyse which of these things that we build 4 times, unittest 4 times and system test 4 times and present it for the managers so they can see how much time we waist every day ?
I am not a popular person 😉😁🙃
I am going to use data analytics on the builds and test so that I can show cases them the time waites in the pipeline and implementing the same new feature 4 times.
Where does he find these beautiful shirts 👕🤔
I tend to compare Technical Debt to treating illness.
You can take a quick pill and get ride of the symptoms but won't be able to fulfil your potential, and if you stop the remedy you'll be in bad shape again...
or you can take the time to go to a Doctor and do what it takes to be cured, after which you'll be ready for any challenge!
Seems to me that Bit rot is just a cutsie word for entropy. Entropy exists, even in software, and (on average) only moves in one direction. No amount of up front design can stop it.
I am not sure that entropy is a good analogy. Entropy just happens, it is not an act of will. Bit rot doesn't just happen, we do it AS and act of will. It happens when we are careless. Digital assets don't get worse on their own.
@@ContinuousDelivery I’m not really using the wood as an analogy. Entropy is the tendency of a system (any system) to reach its lowest energy state. To become more chaotic. This happens in software even if the code isn’t changed. People join the team, people leave the team. Operating systems and tool chains change. Companies change there priorities, brains forget, etc.
That said we agree on how to manage this: systems need to be flexible enough to change at the code, design, or architectural level over time.
@@davidvernon3119 Sorry, but it doesn't happen at all if the software isn't changed. The bits that represented the system, still represent the system. So it is the act of people changing the software that causes the rot. So if we change how people undertake that change, so that it doesn't introduce the "rot" we can cure the effect of systems getting worse and worse. In fact, I was recently told by someone that still works on a system that I helped to create, that now, 15 years later, it is still the best system to work on that they have ever seen. Not saying that we did everything perfectly at the start, we didn't. But what we did get right, was that we established a dev culture that was good, and durable. So all of the people since have kept the code in a good state.
This is all about dev culture. There is nothing inherent in software that makes it rot, so we need better approaches to development and development teams to help them to avoid introducing the rot.
@@ContinuousDelivery buy in large i agree. I’m trying to make only a rather small point. All software (except perhaps firmware) has externalities that change even if it doesn’t: os, frameworks, web services, etc. so as time passes, software rots.
Another form of externality is the team. As time passes people come and go, and this makes modifying code harder. You can document code, but you can’t document expertise.
Another form of externality is the company. It doesn’t matter how “good” a system is if the business changes fundamentally. It’s not possible to develop code that is infinitely flexible to any changes that might ever be thrown at it.
But… do we need better methodologies and tools? Hell yeah!
I don't like the wikipedia definition. Too complex a code can contribute to tech debt just as much, if not more so. I know we learned that lesson where I work at.
So technical debt seems to be fine in some cases?
Yes, it is really all about the "interest" if you debt costs you, then it is a problem, if it is in some code that works, and is not changing, and so there is no cost to the debt, then it is probably not worth fixing.
@@ContinuousDelivery Thanks for the reply! Makes sense!
@@jozefwoo8079 maybe it’s just me, but I put great value on pride of craftsmanship. If a regression test is in place, then fixing is cheap, I sleep better, and the habit of getting it right is bolstered. The people who let small things slip give others the excuse to let more slip.
I'm not sure the terms manual and automated testing are very helpful as the definitions are very loose and that's being generous.
For example, if I use tools in Excel to investigate the contents of a file, could that be defined as automated testing to some extent as I am using a tool to help me? I think giving a solid definition of what automated testing means that stands up in the real world is not as easy as you might think.
@@timmartin325 Well that is a very unusual take I think. In general I think that what most people mean when we talk about manual vs automated, , and certainly what I mean, is that manual tests are interactively directed by a human, and automated tests execute without human intervention. So automated tests are accurately repeatable, and manual tests aren't.
@Continuous Delivery I think that "automated tests" are also interactively directed by humans as they need to be designed, written, debugged, maintained, results interpreted etc.
I like the drug addict analogy😂 will tell my po features == crack next time😂
@3:47
U ok man? :'(
Nailed it! "are they a drug addict borrowing money from Knuckles to feed their habit of Features""
This guy has never lent money to a friend 😂
This is all very cool and dandy... but... if thats is true then why there is so much crap code over there? Reality doesnt confirm those statements.
Bad code is not technical debt. It is simply bad code!