"Agile Practices are 268% More Likely To Fail"... WHAT A LOAD OF...
Вставка
- Опубліковано 2 лип 2024
- A recent widely reported article claimed that the failure rate of agile projects was 268% higher than for projects that applied the opposite of agile. Agile practice is by no means perfect, but the so called research that backed these claims was full of holes and disinformation. So what does the agile development define as an approach, and what does “the opposite” mean, and what are these extravagant claims trying to achieve?
In this episode Author, and long time agile proponent and practitioner Dave Farley attempts to debunk the nonsense and better understand what really works.
-
⭐ PATREON:
Join the Continuous Delivery community and access extra perks & content! ➡️ bit.ly/ContinuousDeliveryPatreon
🎥 Join Us On TikTok ➡️ / modern.s.engineering
-
👕 T-SHIRTS:
A fan of the T-shirts I wear in my videos? Grab your own, at reduced prices EXCLUSIVE TO CONTINUOUS DELIVERY FOLLOWERS! Get money off the already reasonably priced t-shirts!
🔗 Check out their collection HERE: ➡️ bit.ly/3Uby9iA
🚨 DON'T FORGET TO USE THIS DISCOUNT CODE: ContinuousDelivery
-
BOOKS:
📖 Dave’s NEW BOOK "Modern Software Engineering" is available as paperback, or kindle here ➡️ amzn.to/3DwdwT3
and NOW as an AUDIOBOOK available on iTunes, Amazon and Audible.
📖 The original, award-winning "Continuous Delivery" book by Dave Farley and Jez Humble ➡️ amzn.to/2WxRYmx
📖 "Continuous Delivery Pipelines" by Dave Farley
Paperback ➡️ amzn.to/3gIULlA
ebook version ➡️ leanpub.com/cd-pipelines
NOTE: If you click on one of the Amazon Affiliate links and buy the book, Continuous Delivery Ltd. will get a small fee for the recommendation with NO increase in cost to you.
-
🔗 LINKS:
The Register: 268% higher failure rates for Agile ➡️ www.theregister.com/2024/06/0...
ENGPRAX: 268% Higher Failure Rates for Agile ➡️ www.engprax.com/post/268-high...
DevOps Research and Assessment ➡️ en.wikipedia.org/w/index.php?...
LinkedIn Post By Hillel Wayne ➡️ / urn:li:activity:720448...
The Agile Manifesto ➡️ agilemanifesto.org
The Chaos Report ➡️ www.csus.edu/indiv/v/velianit...
-
CHANNEL SPONSORS:
Equal Experts is a product software development consultancy with a network of over 1,000 experienced technology consultants globally. They increase the pace of innovation by using modern software engineering practices that embrace Continuous Delivery, Security, and Operability from the outset ➡️ bit.ly/3ASy8n0
TransFICC provides low-latency connectivity, automated trading workflows and e-trading systems for Fixed Income and Derivatives. TransFICC resolves the issue of market fragmentation by providing banks and asset managers with a unified low-latency, robust and scalable API, which provides connectivity to multiple trading venues while supporting numerous complex workflows across asset classes such as Rates and Credit Bonds, Repos, Mortgage-Backed Securities and Interest Rate Swaps ➡️ transficc.com
Semaphore is a CI/CD platform that allows you to confidently and quickly ship quality code. Trusted by leading global engineering teams at Confluent, BetterUp, and Indeed, Semaphore sets new benchmarks in technological productivity and excellence. Find out more ➡️ bit.ly/CDSemaphore
#softwareengineer #developer #agile - Наука та технологія
I've been a developer for over 30 years. I've done numerous projects where they called the methodology "Agile". Only one of them actually was.
@@kendickinson8307 I was once in a team that was very very agile purely "by chance"... they then brought in an "agile consultant" to make them "more agile"... the consultant introduced lots of rigid processes which eliminated all the agility they had.
This is very true. Most projects I have been in have been traditional projects with standups. In all fairness, communication has been pretty great, but the key has always been good documentation and discussions about the requirements (what and why) so the developers can meet that with how to realize those needs.
Same and same, maybe two projects qualified
Aint that the truth about pretty much everything
I have exactly the same observation...
I’ve only worked on one successful “Agile” team. Issue is enterprise “Agile” is anything but agile.
Very well put, liked that comment a lot!
only a minority of the companies out there are primarily IT companies. far over 90% have their primary value chain with other stuff, not it. "computer stuff" is just a cost center for them. that fundamentally changes how they approach things. you will never get the mindsets that dave promotes on his channel there.
When management calls it "Agile", it usually means:
- The software engineers must be as agile, as a Russian gymnast...
- Management themselves are as agile, as an upside down turtle...
That's why that failure percentage is so incredibly high!
The problem isn't Agile itself, but managements ego that get's in the way of practicing the methodology correctly!
Software engineering these days, is the equivalent of building a house without any prior architectural drawings! Then the customer is allowed to constantly change requirements on the go, without any expected changes in deadlines or price! There is only one direction this can end up in: total failure!
"Corporate Agile" has nothing in common with Agility. After a few years seeing the "corporate agile" in practice I am not surprise that projects fail... Being agile and the agility is the key to success. Sadly, most of the "Agile" was taken over by consultants to monetise the "knowledge" got during 2-3 day trainings.
The problem, as I see it, is that Agile is based on trust between people and the culture of the team as a whole (this is well illustrated in the book "Toyota Dao"), which many teams do not have. Consequently many teams call Agile a Jira app and 10 minutes in the morning :). In addition unscrupulous people use Agile as an excuse for not doing anything: yes I didn't have time, and delayed it for 2 weeks, but "we are agile". Which of course does not suit the business, because instead of getting results that are more predictable and flexible, they get excuses. Agile = Trust & Team Culture.
Agile has caused many business analysts to think that requirements and acceptance criteria does not matter anymore. You literally have to extract that info from them using violence. At least in the waterfall days they attempted to give you some sort of spec document. I have been in this business for 34 years, and in my experience Agile has been a step back in the whole software engineering process.
@@henryvaneyk3769 that's exactly it - 'agile' became an excuse. "Oh you mean we don't have to do the work of figuring it what we need? You'll figure out what we need and make it for us? Agile is great!"... Later.. "wait agile doesn't work"
So much this.
That's not Agile's fault, it's the fault of people willingly and purposefully misusing it. If you read the manifesto, it says "over comprehensive documentation", It doesn't say "no documentation at all". And guess what, requirements are the FUNDAMENTAL documentation, not the extensive over detailed one. Much like Constitutions, Bills, contracts, law in general and many other things, the manifesto was put together by well meant people that didn't go into details because they (naively) thought other people were similarly well meant and would understand where they coming from and what they meant. And we all know how well that goes
No requirements, no thought about what you are actually doing!
Should be obvious.
I think the problem is that business analysists are writing requirements. There used to be software experts in the field of requirements.
After i heard about this book Impact Engineering I went and "read" the audio version while on the night coach last week. It represents many of the reactionary notions that get floated around, and occasionally get injected into our scrum arteries by anxious middle managers during sprint planning. Its not a book i'd completely ignore if i were a agile evangelist and if your starting out with agile development you'r gonna hear these arguments put forward in the book all the time while being told to get in line with some sort of fake-agile linearized sprint programme. Take heart from what Dave says!
I also read his book. I felt an overwhelming need to bang my head on a wall reading it. Literally everything he described as anti-agile was literally agile practices. Everything he described as agile was willful misunderstanding of what agile is.
For example, he misread “Working software over comprehensive documentation” as advocating for no documentation at all. Which takes some doing given the last line of the manifesto.
The reality is that this is some young guy who read Goldratt and now thinks he has all the answers. Which he clearly doesn’t as he has failed to study primary sources before trying to attack him. Unfortunately for him, the poor quality work done here will follow him for the rest of his career.
Fortunately for the rest of us, the industry isn’t really taken with this. It’s on the pile of general anti-agile sentiment, but anyone who looks at it with any criticality immediately dismisses it. Primagen, for example, called out how terrible the study was. Even though he is very anti-Scrum and wanted to believe the study.
So I’m not too worried about this. I’m more worried about the poor state of agile in the industry as a whole.
The problem is that "Agile" doesnt mean anything anymore. It was too vague for people that didn't understand the problems that 'agile' was meant to solve. So it was co-opted by whoever wanted X to justify X.
I have a feeling that they meant Scrum in this rather than Agile. Measuring the successful deliveries of projects saying they use scrum is very easy to do, but as @kendickinson8307 said here the vast majority of those projects are not agile, but ad-hoc with standups...
It’s just more evidence that agility is hard for a lot of people to understand.
You can be Agile without calling yourself agile.
I've been saying for a while I think it overestimated your typical manager. It assumes they understand their product, their developers, etc. And that they are capable of identifying and solving their organization's problems. Basically, it assumes they are competent. Many of them just want a formula to apply so they can mindlessly do their jobs. The Agile Manifesto didn't come with a formula so they were easily sold the nonsense that is rampant in the industry today.
Even many parts of Scrum are targeted at real problems. But when followed mindlessly you don't often see benefits.
@@Spoonbringer Managers, especially middle managers, have the worst position in any organization. They are basically proxies for information and if the information is wrong (like if the estimates developers give are wrong), then their behind is what is being flogged. Many managers, especially in old companies, inherit their roles due to long service rather than competence, so you are very right that some managers could not hit a wall from inside a barn even if their livelihood depended on it, so they blame everyone else and hope to live another day.
It is not just managers that are looking for logic and structure. We all do. Scrum provide a process that is neither good nor bad, it is just good intention colored in. It always fall on the people to make Scrum works and to be honest we already talked about managers not being the brightest always, but I am sorry to say that some developers are not very good at making up work processes either.
This is due to ego, and it does not apply to all developers. When ego hits and people keep telling developers they are superstars, and they are empowered and all that nonsense, they start to think they are better than they really are. This is where the extremely simple development process (not the coding, that is complex) suddenly will have absurd amounts of steps that only make things more complicated. Often this is from young developers that want to show off how good they are, while older developers don't have that need, and they instead want to simplify and reduce the clutter that prevent them from focusing on the code.
Most managers don't understand code or the process of writing code. Most developers have no clue on how budgets, finance or even marketing and sales works. Sadly, most developers have little to no understanding of psychology of UI and UX design and even less about CRO. The ones that do are amazing assets, and they work very well with the UX teams that lead exploration and R&D.
Unless we find a way to stop the segregation between the different disciplines and bring them closer, we will not be agile. Most scrum teams are still doing Waterfall in two week iterations, where they self-isolate and see managers as enemies. It is one of my 3 anti-patters I see a lot:
ua-cam.com/video/HSk914tnJBA/v-deo.html
The only legitimate software engineering study to me is the mythical man month
👍
Thanks for this video. I had looked at this "study" and we had just decided it didn't apply to us. All of my work is scientific software and we don't understand the problem until we start working on it. Sometimes we don't even know exactly what problem we need to solve before we seriously sit down and start working on it. Agile works quite well for us but this idea of knowing everything in advance and having some kind of a timetable is just utterly unrealistic.
This study was for projects and not exploration/ideation/R&D so what you do is different. If it works for you, then it is the right fit. Regardless of what anyone else says :)
@@jimiwikmanofficial are you implying that when you work on "projects" that at least one team member has perfect prescience?
I can't really imagine setting out to solve a problem which I am not even aware of. I can honestly understand how painful it sometimes is to have to make estimations or plans,... but the point isn't to then fanatically stick to them. The point is to set yourself and the management on the same track, helping you actually get anywhere. If you realize halfway through, that actually this isn't the way, and that a completely different solution needs to be developed, you go back and redo the plans. In that sense it's not really different from Agile.
I guess you have the luxury of not having any oversight, so you can "just sit down and start working" on nothing specific.
@@edgeeffect If you are asking if I have witnessed projects that have been delivered on time and on budget because we did the discovery and defined the requirements, then yes. I have done that multiple times as a consultant. It is not very difficult when you know what you are building and why.
@@JanVerny I agree 100%. Requirements, or any definition of what to build and how, is a living document throughout the project.
This is why close connection and good communication with the stakeholders as things are changing is key.
That and to make sure there is a process for change management so you don't get scope creep, but controlled changes to the requirements.
I agree but honestly, you can't blame anyone else but our industry for the misuse of the words agile and devops. Like you said, subversion to marketing. When you sell agile and devops then you know things are wrong. With that understanding, for most people there is little distinction between agile and bad agile. Lets not forget that most managers don't understand software and are irrelevant to the process. Software engineering cultivates ignorance and irrelance like not other industry does. So, another misinformation to add.
What I loved was the following part
"Our research has shown that what matters when it comes to delivering high-quality software on time and withing budget is a robust RE-process and having the psychological safety to discuss and solve problems when they emerge, whilst taking steps to prevent developer burnout"
--> Sounds to me more agile than what most companies do :D
So the hypothesis is: Agile is more likely to fail, try to be more agile 🙈
Most companies that think they are agile are not at all. I've seen companies where agile meant upper management just rushes everything.
Agreed.
@@12q8 I have the same experience. It is more ad-hoc than anything else.
One of the reasons people doesn't like agile is because what is taught to them as Agile. I am seeing, in different environments I've been working, people thinking that Agile is:
- having a meeting in the early day on what they did the day before.
- reporting how many hours they spent on some feature
- trying to implement some process
- having a scrum master at the end of the week asking how things are going
- having a sprint window of 2 weeks
- having estimations meetings
- having a retrospective meeting at the end of each sprint
- etc.
This can be Scrum, but it is not definitely the definition of Agile, as Agile is a more generic and abstract concept. We can even say that there are other ways to do Agile which are not Scrum. The main reason people is in such a state is because the Scrum process is commercially more sellable as training courses, than Agile.
Come on people, the Agile manifesto is only 12 principles and so much more quick and easy to read and understand!
I cannot help but love bs being called out.
My team used to be agile back when there were only two of us and the business basically left us alone to do what we wanted to get our work done.
Those were some of the most productive years of my life. My two man team go so much done. Now my team has swelled to about 10 and our "standup" we're forced into takes almost an hour every single day.
This is not a failure of agile. It sounds like a failure to be agile. You will need to thoroughly analyse WHY you need to spend that hour each day. Introspection is a vital element in effective agile. 10 people will never work as quickly as 2, because there is a coordination tax with larger numbers of people in a project. Two people can also never build what 10 people can, which is why we accept that. But losing an hour to your daily each day is way too much. Our team of 7 takes 15 minutes, which is still too long I should add.
@@MaximilienDanton It's exactly a failure to be agile. I also know exactly why we spend an hour a day in a standup meeting.
We are a cargo cult now. The meetings we have are there because that's what everyone does. I actually got the meetings cancelled once which was nice; but then we got a new manager who reinstated them.
It's worth noting that a few teams were combined into one due to bureaucracy which is why there are so many of us now. There wasn't an increase in total developers. We just go slower now.
This is 100% a culture mandated by management issue.
@@MrC0MPUT3R I can't say I haven't been there. The question will be whether you can stay and push or whether it isn't worth it. It's taken me a long time but eventually some things have started sinking in. I started by predicting failures and when they happened I would offer an alternative. It's very painful. It can take years.
The sad part is many "data driven" managers will get sold by the headline and the few numbers they show. Secondly there are a lot of devs out there repeating this junk because that are suffering under cookie cutter scrum, and now hate agile.
If your goals are defined as the code (artifact) that needs delivered, then sure, your chances of success with project management is better than such with agile. But if your goals are defined as an outcome - as solving or significant improvement of the problem that is measured by the data and user feedback, then agile works better.
The truth is that delivering artifacts is a cost center, where solving of the problems is what generates revenue.
I've seen quite a few companies that managed to successfully deliver artifacts on time and gone bankrupt.
I worked agile prior to the agile manifesto before 2001. We called it "Vaseline".
"Agile" died 10 years ago.
What is it that Google, Microsoft, Amazon, Netflix, SpaceX, Capital Bank, ING Bank, Tesla and SpaceX do then?
For Agile to work everyone must be involved and actually willing to identify, name and solve problems by thinking about them and coming up with solutions. From the management level down to the individual contributor. What I've found is that most people don't want to work that way. They want to be given clear instructions and just do their tasks. Which is ok if the higher ups actually know what they are doing but I've seen the same mentality at the middle management level, just follow some prescribed process by the book and then get completely lost when it doesn't work.
As said by Jurgen Appelo in the book Management 3.0: "One of the things I learned this past decade is that Agile software development is the best way to develop software. But I've also learned that OLD-STYLE MANAGEMENT is the biggest obstacle to the adoption of Agile software development around the world."
We created the Agile Manifesto, but forgot to agree with company managers and executives!
The problem is the word "project". There is no such thing as an Agile Project. As soon as you call something a project, people will ask "how long will it take?", "What will it cost?" and "What will I get?". To answer those questions you need to spend months in analysis, attempting to write huge set of requirements. I have seen this recently in big organisations with failed Agile Implementations that have decided the problem was they didn't "start" well enough. Those organisations and projects never were Agile in any way. The headline should read "Projects that try to mix in Agile practices are even worse than projects that don't"
Anything that clashes with our current best knowledge should be treated with heavy skepticism, especially when stakes are not low.
Thanks for calling out garbage and bringing the receipts to prove it!
I have participated in several successful Agile projects, but they often led to burnout. Many developers quit because Agile can feel like a relentless, never-ending hamster wheel. It involves constant story points, status updates, velocity projections, and demo days, all under the pressure of meeting the two-week Sprint deadline, leaving little time for research or experimentation. I don't think developers can sustain long-term careers in Agile methodology when their work is constantly measured in two-week Sprints, scrutinized by the SCRUM master and the product owner based on how many story points they have cleared. It's an awful system.
Once, while explaining Agile to a client in a meeting, I likened our process to delivering a bike when they had asked for a car. They stared blankly and asked, "What would I do with a bike when I specifically asked for a car? I dont want a bike, Just build me the f*cking car."
My last job involved an EXTREME deployment of Agile... The most intense I've ever had... With constant meetings around the clock!!! All the while, as the team I was parachuted into, kept on fighting one bug after another, in a microservices architecture that would make NASA proud!!!
I quit 2 years ago... And haven't had the energy to get myself back into the industry since. Preferring instead to work on my OWN stuff with an ex colleague!
But, with half my savings gone, I will soon have no choice but to dive back in. This time however, I'm just going to do freelance / contractual work! I am DONE working full time in the Corporate World!!!
I feel like you haven't correctly implemented lying / holding back. If you are constantly under pressure assess why. Did you underestimate? pad your estimates. Did you estimate more work than you had? don't agree to start the sprint, and pad your estimates. Did your requirements change during the sprint? reject the changes and make them in a future sprint / revise the estimate and take something else out, and pad your estimates. This is an industry where work is hard to estimate (probably +-50%), so also make sure to always give 80% so you can pull out some hidden efficiency when everything is on fire and you're at risk of having to work overtime. Alternatively you could negotiate for paid overtime, but for some reason that is quite rare in my area for software.
It would be a reasonable argument to suggest what you experienced was SAFe, not agile. Overlapping terminology is a killer.
if you're so stuck to the process, its no longer agile... its in the bloody definition of the word!
Cut away processes you dont need. Properly timebox your devs weekly efforts, if things cant fit in, too bad. Prioritise and lower priority stuff goes inti the next cycle
2 week sprints feels rush? make it 3 weeks. Ive been in an org that had 6 week sprints.
status updates and velocity projections do not sound very agile to me. more like waterfalls 'are we still on time and budget'
one reason why it got so much attention is that many devs hate agile because of the tons of meetings each day that could just be emails.
many lose 2-3 hours per day to pointless meetings in the name of some form of agile that lost it's meaning long ago.
One thing that struck me when I first saw reports of this study is that I've failed to even find the study. Not even a "this is behind a paywall", just some articles about this supposed study. I was wondering what exactly these numbers meant. How did we establish a project was a failure? Was it failure to make money? Was it people who worked on the project said it was a failure? I couldn't find it, nor anything that stated anything precise enough to be useful.
Project's success is measured in delivery on time and budget. It always has and always will be the success criteria because the outcome of the project ROI is part of the product or service goals and not the project itself. There is a difference between project delivery goals and product value generation goals.
@jimiwikmanofficial That's a ridiculous requirement, though. Most projects are either not delivered on time, or not on budget, or both, full stop. Full stop. And yet companies still do projects. Why would they do that, if almost all projects fail? The answer, of course, is the THIRD criteria, which you failed to mention, is FAR MORE IMPORTANT than on budget or on time: Did the project meet the needs of the business! This is the most important criteria! To the point I would call any project that meets the needs of the business a success. If it comes in on time and under budget I'd call it a nearly perfect success.
Obviously, if the product is so late it no longer meets the needs of the business that's a failure, and a project so over budget the business can't afford it is also obviously a failure. But notice that these two cases can also be framed as the project not meeting the needs of the business.
It's rarely the case that a project deadline is so firm that the project might as well not be delivered in the deadline is missed. Likewise the project budget is rarely so tightly defined that going over budget buried the project must be abandoned.
However, a project that fails to meet the needs of the business is always a failure.
@@jeffwells641 That is a silly response and of course it is not true, :) Many projects are delivered on time and on budget. I have done so on many occasions myself.
The value proposition for a project is the determine factor for deciding if the project should be funded or not. That is the same as for a feature in a project backlog, but it just requires more time to estimate and define properly. The decision is the same where the cost of measured against the proposed value and the likelihood of the ROI to be true.
This value is obviously measured, but for a project manager and the project team(s) that is not their measurement of success. They are responsible for getting the project delivered on time and on budget.
The measurement of the ROI falls on the product/system owners and the stakeholders to measure after delivery. This is because a project have a fixed start and a fixed end. Since evaluating ROI from a project is not something you can measure quickly, no organization will keep the project team(s) and project manager sitting on their butts after delivery.
If you are building a new e-commerce solution or rolling out a new e-commerce platform in 35+ countries for example, then measuring the value creation takes 6 months to a year...or more.
@@jeffwells641 It sounds to me like they were given a task with perfectly known requirements - for which waterfall is optimal.
@@jimiwikmanofficial I wasn't really asking what random other commenters thought succes meant, I was asking how the study decided something counted as succes and quantified that. As near as Dave and I can tell this study was done by questionaire to software engineers. Software engineers typically aren't the people who set deadlines and they usually don't know the budget either. So I don't think the study could even measure what you are proposing to measure.
In addition, you haven't told me how we know what the budget or time constraints were. What the project manager said it was? What the company can afford? I do expect some precision of a study, not just in its execution but also in its reporting.
Lastly, your notion of project succes divorces it from being useful for the organisation. If your commercial company ships a product six months late at 50% more costs than expected, but makes their costs back in triple that would be a succes in most people's eyes. A product ready on time and within budget that customers refuse to buy would count as a failure to most sane people, but not by your standards. Even if your definition of succes could be made precise enough to study and someone did actually study whether agile projects brought these things closer or not, I don't see why anyone would care.
TBH, traditional, waterfall approach might actually have a lower 'failure' rate in a sense that they don't check if particular project is worth the cost, instead they (as Agile Manifesto puts) follow the plan instead of responding to change. The 'fail fast' approach in Agile will give more failures as a result... which is a good thing!
I found it really interesting that there was an almost gleefulness to a lot of the reporting and commenting on that 'report'.
The biggest problem with "Agile" these days is encouraging non-tech people to be directly involved in the development process. Having a Scrum Master or Product Owner without a tech background is a disaster 95% of the time. Replace them with a good Lead Developer or Team Lead who has both a tech background and people skills, and I think even Scrum-like setups can work fine. We, the developers, understand the Agile way of working naturally. We don’t need some dhead treating us like children. I would just add that I blame developers as well, because a lot of them put up with that BS and are scared of standing their ground.
Yes... because technical people have a perfect understanding of every single domain that software can cover.
In my experience, having POs with tech background is a recipe for disaster, as they tend to focus on the "how-to" and ignore the "what" and "why" parts. Then you end up with stories which are really tasks. Developers like technical POs, as they talk the same language, but the risk is they develop things nobody needs or different than what the customer wants. As for the Scrum Master, I don't really see the value of this role.
@@edgeeffect You don't understand my point. I am not saying that there is no need for someone like a product/project manager, but leading the development team should be left to someone with a tech background. He should collaborate with the product/project manager who has more domain knowledge. The problem arises when you place a product owner, who acts like a product manager, directly into the development team.
@@Jedimaster36091 At least we agree on scrum master :) I understand your point and it is valid. You can't take just any developer and place him in that role. You have to have someone who understands business side as well. I worked with that kind of lead dev once and believe me...one of the best working experiences ever. Customer was happy as well as developers and managment
I am a strong advocate of always making sure you have a requirement analyst in every project, and I don't mean a business analyst that work only on the business side. A Requirement analyst is someone that work like a translator between business and tech to make sure they understand each other.
If you have a tech lead that understand budgets and portfolio management and you have a business owner that also understand tech, then you might be able to get by without one, but those situations are rare (but amazing).
I often find my frustration with requirements is that they aren’t clearly defined where they ARE understood. New products in a known domain come with some level of requirements otherwise we aren’t building anything specific. Where THESE are not clearly defined, waste dev/test/qa iterations that could have been used for genuine discovery.
I feel the OG iPhone not having GPS is a slightly disingenuous example as at the time GPS wasn’t a known requirement, where as the SMS protocol would have been a known requirement and should have been clearly defined.
Love the channel Dave, you’ve helped me and my career a great deal.
Is it agile can be rephrased as "is it well documented?"
Pretty much every enterprise project is called agile, but when management doesn't have comprehensive, live documentation, it's not agile.
Great video - now where can I buy that shirt? Love Douglas Adams!
There’s a link in the description with a discount code 😎
I dont know about failure or about such a high percentage, but Agile is all about the ceremonies... and every single time I have attended an Agile ceremony (for around a decade now) it has been a time-waster. I have never seen a single productive outcome of using Agile. Could it be the workplaces? Possibly, but even something as core as a daily standup is far too often and wastes at least 30 minutes of a team's time (more, if you count people ensuring in advance they have their spiel in order).
I prefer a more hybrid approach. It's fine to take some ideas from Agile, some from Waterfall, and find your own ways to combine/improve. A continuous feedback loop during testing and and delivering is fine, but doesn't require formal requirements of every single worksay standups... it just requires competent dev leadership.
Honestly, it should be a no-brainer to people in 2024, but there really is no one-size-fits-all framework. Every project differs, and every workspace has a different dynamic.
Agile is NOT "all about ceremonies" -- "Individuals & Interaction OVER Processes & Tools" is the first line of the agile manifesto. If I play cricket and call it football, that doesn't make it football!
1. Lack of business domain knowledge
2. Lack of system wide design that is visible to everyone, rudimentary at first, evolves as the project progresses
The thing that drives me nuts about this channel, because it has a lot of good advice and well thought out ideas, is that it has a very narrow understanding of the types of software projects that are out there. For 8 years I ran a business where our primary focus was on people with very small but limited budgets and they had to get a product out that they knew exactly what they wanted. We used waterfall on every single project and it worked great. Nowadays I work with gigantic billion-dollar businesses that do a mix of waterfall because they know exactly what they want and it takes just one shot to get it done and then they have other projects where they have to experiment and agile is more useful. I really hate it when people get up and say there's only one way to do it and while you're not exactly saying that basically that's what you're saying here.
Great video. The author of the book/article is supposed to be a PhD and well versed in scientific method - too often we just accept opinions and views from people with a bunch of letters after their name. We need to get back to not being afraid to challenge stuff because academics aren't always right.....in fact, I remember somewhere saying that the vast majority of all published academic papers are wrong.......we can add another one to the list....
Agile™ is whats really used widespread 😂
Thanks for sharing this perspective. I've been reading these breathless articles and thinking the exact same thing. One nit - I think your comment at 15:30 about the 7% improvement of avoiding late changes - I think you may be reading that backward vs how the factor is printed in the table.
The reason software needs to be perfectly predictable and business does not is because the people who build businesses have access to the sources of funding and people who do software do not.
Very well said. Thank you for the honest and thorough review of that nonsense report
Agile is one thing... But the implementation of Agile is something else entirely!!! And that is where the problem lies...
Great video: it’s so rare to watch authors being able to share a well founded criticism of a book on such a complex issue.
I fully concur with each and every argument. Agile processes are not the manifesto, they are just processes, which can be changed at will.
Another great video!
People offering quick fixes or methodologies for making the outcome of SW projects more predictable, and are not truly Agile, are no better than snake-oil salesmen. They offer false security.
The problem is that in SW, all the repetitive work is automated, and thus doesn't cost anything. Whereas in the other engineering disciplines the repetitive work costs an order of magnitude (often several orders of magnitude) more than the non-repetitive, creative, design work. So any methodology that borrows from other engineering disciplines simply doesn't work in SW development. SW development is inherently more complex and difficult than most other engineering work. As someone said, _there is no silver bullet._
(Systems Engineering is also inherently complex. Thus system engineering methodologies borrow heavily from SW engineering and vice-versa.)
It seems that management gravitates towards planning. Budgets and business plans. This results in people having careers that are dedicated to these meta activities. So when they get introduced to software development they Intuit that more planning and estimation will lead them to a better outcome. I'm amazed at the number of times I've sat in meetings where we priroitised what was fastest to do rather than most valuable for the customer. That drives me bonkers. We avoid doing highly valuable things because they take too long. I also worked at a company where design and product worked waterfall but expected development to be agile. That was a real impedance mismatch.
The "Agile Practices _ more likely to Fail", but "the project" was successful?
( a tongue -n- cheek take on the title ... practices vs. project. ; )
A well researched, fact based response.
I'm a big fanof swear words, especially implied swear words... nice title Dave. :)
Impact Engineering seems to follow the idiom "you never fail things you never try."
No it's the bag of meetings and focus on communication that causes this, people will spend an hour debating percentage of time needed to dedicate to backlog, The time they could have spent to actually solve on.
These managers, who failed their coding career are the root cause
Hi Dave,
Regarding project requirements being clear and well understood before beginning (8:50).
I agree predicting consumer needs upfront is silly crystal ball work, but what if you have existing consumers that do know what they would like to see changed.
Wouldn't it be valuable to spend some time eliciting & "cementing" ~90% of these requirements before starting the programming work? It would build alignment.
Imagine you're writing software targeted at businesses and you accept medium/big change requests that take the form of projects. So you discuss with the customer what they'd like to see changed and together you setup a list of requirements and discuss them using some preliminary design work (e.g. wireframing). You organize workshops to discuss how the system would behave after the change.
After this is done, you start the code work (in sprints together with the customer), only agreeing to small changes to the cemented scope in this period.
Wouldn't that be a scenario where one mostly achieves "clear requirements upfront" (the alleged 97% improvement) mentioned in this questionable research?
I'd say the above is a hybrid agile/waterfall type process for RFCs (with requirements gathering done iteratively and only allowing small changes in sprints afterwards).
I wonder if for defined RFCs on existing systems this type of process can be efficient: where you spend quite some time gathering requirements upfront.
Simpler put: I wonder if/suspect processes for new product development != processes for change requests on existing software
Maybe the above scenario is indeed rare and I'm just in a niche, or maybe I'm missing some nuance from you on "software where project requirements being clear and well [..] are corner cases"
These were my thoughts while watching this video.
The author seems to be infatuated with the Waterfall methodology: start out with a perfect description of the requirements, follow up with precise implementations of those requirements. At least: that's what he considers a "success".
Agile sucks. Endless meetings, contrived roles and terminology and developer crunch aka "sprint". Lacks creativity. Designed only for grind.
In most organisations you can't really do agile because of contractual restrictions and defined processes and roles. Trying to do agile in such an environment only results in changing the process and roles but still not doing agile.
Maybe the video that was recommended somewhere in the middle will answer this but when I saw the key points of agile methodology and one of them was "working software over comprehensive documentation" and it surprised me a little bit. Obviously documentation is important but so is working software. Is "comprehensive" doing the heavy lifting here? That is, you should document things but don't work so much on documentation you fail to deliver the product?
The agile manifesto closes with "That is, while there is value in the items on
the right, we value the items on the left more." It doesn't outright forbid documentation, obviously.
What, no definition of 'failed'?
If a project using Agile failed, how do we know that if a different methodology had been used instead, the project would not have failed?
Even with perfect requirements, software development is hard.
There is universal hatred of Agile in the software world because people just want to code in boxes and reinvent the wheel over and over in a desperate attempt to be "I am very smart"
Well, I'd argue that agile has a 0% failure rate. After the first sprint you're likely to have a "hello world" and celebrate it as succes.
Absolutely disgusting what that author have done! Falsifying information just to promote is book, hope nobody buys it.
engineering is predicting the outcome before the building start, you need detailed plans for the non changing variables .
9:40 putting architecture as example of "asuming perfect requirements is naive" sound wrong because they have clauses about what can be changed in the original desing and i feel that issomething that we most copy into computer enginering, agile try to move away from enginering to the realm of bullshittery that marketing ppl use to sell their products.
There is no such thing as "Pseudo" Scrum, there is only Scrum, which is not, nor can ever be, agile.
This comment section in a nutshell:
- We tried "hammer" to drive in nails but the damn thing is way too light and the plastic handle keeps breaking.
- Um, what you are describing doesn't sound like a hammer at all but more like a screw driver.
- No true scotsman!!!
I'm not a big fan of the Agile ecosystem, but this study looks like complete nonsense. It's pretty obviously an attempt at viral marketing, and it seems pretty successful for it.
Since all companies claim to be agile, where do you even find devs that claim not to? Hrrm. That said I have a few horror stories from very large “agile” companies (agile at dev level, waterfall on top layer, in the extreme), such as inability to understand how the plan/releases plans maps to different parts of the industry specification … but let’s say these companies would claim to be waterfall instead of agile, almost nothing would change. It’s not that individual dev teams claim to do scrum/Kanban/whatever that is the primary cause of top level plan being opaque… communication problems on the top are what they are. That customer delivery points don’t directly align with completely specific industry specifications, that’s a business vs industry spec priority. I could blame agile devs for big companies being hard, but that’s the wrong take. Failure to communicate / visualize information in very large organizations is one of the core problems with very large orgs.
Personally,l I don't care about "agile" practices. I just want to have tight feedback loops. But don't use that as an excuse to sacrifice planning and strategizing as it is easy to get stuck in a greedy algorithm or spinning one's wheels. Also, I aim to involve developers heavily in requirements gathering.
Try asking Philips!
"Agile" is implement like shit, excuse my French, by 99% or more of the companies that toot their horn for being "Agile".
CONsultancies are 268% more likely to fail and consultancies claim to practice agile.
I’m going to be first in line for Impact Engineering certification, and then watch the dumb money roll in.
Agile is a bit like socialism. Sounds totally reasonable and cool in theory and never works in practice.
👏well said, Dave!
In terms of my empirical experience agile is terrible . We delivered faster and better with waterfalls
nobody knows how to do agile LOL
Well, doing Agile helps you finding out how to do Agile well ;).
@@krumbergify in 30 years. it's actually not a critic, just a hot take
@@krumbergify Oh no. Not the recursion 😨
"managers" try to do agile but all they do is have an inspiration on it. actually waterfall and the other methodologies are interlinked there and people are blind to the notions, and ignore signs that the project is gonna fail, blinded by the ego and not adapting. that's why projects fail. because they're not agile, actually
@@ThaitopYT Well agile is at least circular. If you work in short cycles in a cross functional team then you can incrementally learn, improve and change direction as you go along and learn what works for your team.
Agile is not about having a bunch of stupid meetings that don’t provide value. Skip them if your team doesn’t find them worthwhile. If you organisation requires you to have them then you are not doing Agile.
I think that the report is probably accurate. The "Agile" methodology has an inbuild fault:
- If the project is unsuccessful, the blame is that the team was not Agile enough
- if the project is successful, then is the Agile merit.
Practically, you cannot evaluate if any Agile methodology is good for you. But because of the buzzword, any money/people burning project is hiding behind the Agile label.
@Dave Farley: Please be honest, how many failed projects with the Agile label do you know?
First and foremost, the math is all wrong. It boggles the mind how something can have an over 100% failure rate. What, it causes non-agile projects to fail as well?
If you see a dev claiming agile is bad, it just means they are a bad dev. Agile means the devs determine their own process. So either these devs accepted a process they disagree with or they claim things without knowing what they're talking about. Both are the hallmarks of a shitty dev.
YES, a total load of S**T!
Great video.
Agile is nothing but agile nowadays. Far too many talking only people in the industry.
The Wikipedia entry about Junade Ali has been removed - user banned.
Is this the same Junade Ali who has published books like "Mastering PHP" and "Object Orientated PHP"? Are those topics even worth a book?
In my experience this is what happens when engineers do software. If the calculations of predictability don't work, it's the formula thats wrong. Just add on to the polynomial equation, do some fft and voodoo to make the curve fit the points.
Reminds me of the fact that 43% of statistics are made up on the spot 😂
I heard it was 82.7% 😉
P-values don't mean "chance of being wrong". In fact, p-values should generally be avoided because they have a niche technical meaning that doesn't often align with anything (non-mathematician) researchers give 2 figs about. See, for example, American Statistical Society's 2016 special issue on p-values for a proper rundown.
Yes, this leaves us plebs in the lurch when researchers keep mis-using statistics (and failing to publish more useful metrics) but at least if we know an argument hinges on p-values then it's probably bunk, and our critical brains can engage. Apologies for the mini-rant...just doin' my bit in the battle against crappy statistical practice
As usual there tool isn't the problem. Agile and waterfall can both work great, if used correctly. I think a good manager can work with both and adapt it to the needs of the company and the team. The problems start of they are not allowed to adapt or the manager puts his own view over the needs of the team, then all methods will fail.
Waterfall can't work great, because it says that we have to forsee and have a solution to all of the problems we will encounter before we begin - that doesn't work!
@@ContinuousDelivery waterfall works if your Software has high legal requirements and you need to get it approved by the government.
@@ContinuousDeliveryRead Royce article "Managing the Development of Large Software Systems". Why should I do it twice, if I had a solution to all problems?
If you should write a program to compute the wage/income tax, all problems are known. The difficulty is to read the legal texts.
@@ContinuousDelivery Doesn't it depend on the problem? If my work is churning out relatively simple and well defined products then it'll be fine, not so if it's complex?
However pure and sacrosanct the original principles of Agile may be, in my experience it's ended up with a crushing and demoralising waste of time, energy, morale, and of course, productivity. Agile, whatever it's supposed to be, in the real work, clearly never really ends up working (or very rarely). I honestly thing the whole domain of team workflow philosophy has just been spun up into a world of bs so people with bs jobs can feel important. "oh, so you're saying waterfall is better lol.." no, I'm saying I don't care. Let's just act like humans, speak when we need to, and get it done. No religion needs to be invoked.
Read the manifesto and tells us where it is wrong!
Agile blows chunks, and I refuse to participate in Agile projects. Everyone pretends it is the equivalent of an empathetic guy with the Golden Rule rather than a bunch of witch burners demanding obedience and the approved incantations. They then defend it by saying "oh, that wasn't Agile, then". But religions are what the adherents practice, not what some apologist claims them to be. I watched a high morale team able to crank out essentially bug-free functionality at a high pace get turned into a process-obsessive bunch of fraidy-cats by certified Agile coaches. Quality plummeted, productivity more than cratered, morale went negative, most of the people left, and the managers gave themselves awards for it. So... you go ahead and say "well, that wasn't Agile" but that's what the cult really resulted in, so that's what it really is.
If you are a successful SW engineer (successful as in making successful solutions), I can guarantee that you have incorporated most if not all of the Agile Manifesto in you way of work.
Do you listen to your users and incorporate their feedback, or do you doggedly continue building what was agreed years ago even when users are dissatisfied? Do you write reams of documentation no-one will read and even when they do, they discover it is out of date? When a colleague comes with a question, or you see him struggling, do you refer to the documentation system or the Design, or do you sit down to work with him? Does your Design change when you discover that it must? Did you ever talk with a User or other non-SW person to discover you misunderstood something about the solution, and then changed the SW accordingly?
If you answer these with Yes, you are Agile.
Agile is an undisciplined "practice".
Software has completely lost the ball while going for the goal.
The best teams that I have worked on, or seen, we agile AND also the most disciplined teams that I have ever seen.
"what utter nonsense" sums up that article really well.
Too much paid ad
18 Years of of software development has though me that both Agile and "waterfall" is garbage, but at least the waterfall model has the common decadency not to treat me like a child that needs to be emotionally manipulated to do my work.
You can use "whatever" development model if you have good people and a good culture.
And not ONCE in 18years has Agile been an improvement of the culture!
Amem
Odd to hear that Agile is what leads to treating the team members like children. In fact, with Agile it is the team that has the responsibility for delivering the backlog, to devise their own internal way of working. And this require mature, confident and inquisitive people.
@@Jedimaster36091You’ve clearly never worked in enterprise “Agile.”
Man, you did this again. When you try to argue against a point by semantics or absurd interpretations of sentences(ie, what is the opposite of Agile, when OBVIOUSLY that was not the point), you dont have an argument. Just sayin...
@@spectr__ Hard disagree. Words are important and have meaning, especially in the context of something presenting itself as a scientific study. He's right to call out how "success" has been defined for the purposes of the questions, and yes, if the author is suggesting that one of the tenets of Agile is fundamentally wrong, to show what the opposite looks like.
@@stevenfewster Words have more or different meanings in context of sentences.
Sorry to pop your bubble, but your arguments are very poor, and it’s hard not to interpret this as anything but confirmation bias response, based on your previous beliefs.
You want us to dismiss the whole thing because it’s “marketing for a book”, but have no issue shilling “Agile Software tools” not 5min later.
All branches of engineering started as some sort of craftsmanship, and those craftsman decried standardization, planning and processes and quality control to be things of the past, not applicable to this new “science” that required sharp minds, flexibility and adapting to the circumstances. Yet all of them have gone that way and are all the better for it.
It‘s amazing how out of touch software engineers can be, as if the rest of us don’t spend all day with computers or something. My CAD can also be infinitely replicated at no cost, but I don’t use it as an excuse to avoid proper controls.
Agile is great for developers, because if requirements can be changed at any point, then success is pretty much always guaranteed, even though it’s obvious that software quality keeps falling. This by itself already makes any study pretty much impossible.
You are seeing in your *own* backyard people switching back to vim and it’s relatives, because old software is just *that much better*.
You can’t predict the future? *me neither*, in fact that’s the point of good engineering. But the solution is not to just throw stuff and see what sticks, it’s to model and prototype, to answer your questions without wasting resources and move on.
Take your "model and prototype", add "show them to the (surrogate) user ASAP" and you have... Agile.
I think you misunderstand something about agile. It's certainly not about doing away with planning, processes and quality control, quite the oposite actually. It just acknowledges, that you can not plan years ahead in detail and it's better to take small steps in a complex environment and check regularly, if you're on the right track, as was hinted at in this video.
@@Reashu I guess I could also add a bagpipe and make it a Scotsman...
I don't understand your argument. What do you mean "I don't use it as an excuse to avoid proper controls".
Side note: Vim is by no means old software, it still in active development, with seemingly automatic many-times per day releases. As recommended by this guy.
@@Adnidi are you really going to tell me that “people over processes and tools” means Agile is really into quality control?
Software quality control is implemented in most places *despite* the Agile crowd, not because of them. Scrum for instance has lots of processes, but literally no concept of quality control. With people rushing code every 2-3 weeks to have something to show off on “demos”, is it really surprising that codebases turn into spaghetti in a matter of months?
"If you're not successful with agile it's because you're not doing _true_ agile" -- perfect No True Scotsman fallacy there
No it isn’t. Agile is a tool, if you aren’t actually using the tool you can’t say that the tool doesn’t work. It would be no more inane for a carpenter punching nails into timber using his bare fists claiming that hammers don’t work.
@@OnlyForF1It absolutely is because he’s saying Agile always succeeds if done correctly. That’s an assertion without a proper backing.
"Agile" is garbage. You waist 60% of your time in meetings instead of getting anything done. Its total BS.
That's not Agile at all. Too many meetings is a symptom of a process heavy organization.
@@Jedimaster36091 lol that's "agile" all over the place.
What the heck has agile got to do with the amount of time in meetings? Just goes to show that most people’s interpretation of agile is way off the mark.
@@leerothman2715 Omg you're statement makes absolutely no sense whatsoever. Have you ever worked anywhere?
@@eyesopen6110 What don’t you understand? You equate agile with meetings, why?
Better title: "17 Minute Cope Stream." This is a pretty disingenuous video, honestly. Saying "agile fails" doesn't mean anyone is advocating for the direct opposite approach. The report speaks about reality meeting agile's practices. The industry still refuses to acknowledge that 60-80% of projects fail to meet deadlines/expectations. How on earth would the development framework you work in be shielded from that? More and more, you speak from a bit of an ivory tower, that's hardly relatable to most of your viewers. That is the real naive nonsense.
PhD: can 8u115h17 at a post-graduate level
oh, a bit of a fail,...bud arrogance is your failure
Agile sucks
Depends on what you mean by Agile.
Real Agile has never been tried 🤡
The most used definition of agile, SAFe Agile. We know that true agile is not like that but that is not what is commonly used.
@@spectr__ Well it has, but only for small teams and in organisationens that value progress over following their process.
@@krumbergify That is not Agile, they are Just Doing The Thing.
No True Scotsman fallacy
I would have to agree with the study. Noone does agile properly and nor do they want even know how to do it right. I have NEVER seen an agile. project succeed. They are all massively delayed and the product goes to market BROKEN. I consider that a failure even though the product was delivered.
As far as I'm concerned, agile is like communism, works in theory..