Study Reveals 268% Higher Failure Rates for Agile Software Projects
- A study of 600 software engineers from the UK and US reveals that projects implementing Agile Manifesto practices are 268% more likely to fail compared to those that do not.
- Research shows that adopting a new Impact Engineering methodology can reduce Agile software project failure rates by 6.5 times.
- Adopting Impact Engineering could save the USA $115 billion annually in wasted R&D spending, while British taxpayers could save an estimated £7 billion annually on failed government digital change projects.
Edinburg, Scotland – 4th June 2024; The Agile Manifesto has been around for over 21 years, yet there is still a lack of empirical research on its true impact on the industry. This is despite recent findings showing that 81% of business decision-makers in the UK and 89% in the USA are concerned about the on-time delivery of software projects in their organizations.
Today, new research conducted for a new book, Impact Engineering, has shown that 65% software projects adopting Agile requirements engineering practices fail to be delivered on time and within budget, to a high standard of quality. By contrast, projects adopting a new Impact Engineering approach detailed in a new book released today only failed 10% of the time.
The statistical significance of the study data showing that projects using Impact Engineering practices performed better than all others is so strong that the probability of the finding being incorrect is the equivalent of rolling a number six consecutively six times on a fair six-sided dice, on the first attempt.
The study, conducted by Junade Ali PhD CEng FIET and J.L. Partners, saw participation from 600 software engineers (250 in the UK and 350 in the USA). Fieldwork was conducted from 3rd to 7th May 2024. J.L. Partners is a member of the British Polling Council and abides by its rules.
Three of the four practices listed in the Agile Manifesto are “Working software over comprehensive documentation”, “Customer collaboration over contract negotiation” and “Responding to change over following a plan”. However, the new research has found that projects which had a specification or requirements documented before development started were 50% more likely to succeed than those which didn’t, projects which had clear requirements before starting development were 97% more likely to succeed and projects which did not require making significant requirements changes late into the development process were 7% more likely to succeed.
Other practices also increased success. Projects in which the software engineer reported feeling psychologically safe to discuss and address problems quickly when they emerged were 87% more likely to succeed those which didn’t. Projects where the requirements were accurately based on a real-world problem were 54% more likely to succeed than ones which didn’t.
Interestingly, the study found no statistically significant difference between project success for those working on one project versus those working on multiple, despite reducing work-in-progress being a key tenent of the Lean software development methodology. However, previous research Dr Ali conducted has shown that 83% of software engineers report feeling burnout with “high workload” reported as the main reason.
The research comes in the wake of catastrophic software failures increasingly being present in the public consciousness. Dr Ali’s previous book “How to Protect Yourself from Killer Computers” investigated numerous cases of fatal software where the technical causes have been attributed to software design problems, including aircraft entering ‘death dives’, fatal car crashes and killer radiation overdoses in hospitals.
Indeed, the Horizon IT system was one of the earliest large-scale projects to use an Agile methodology, namely Rapid Application Development, which has been condemned by Fujitsu engineering witnesses in the public inquiry (Terence Austin and the whistleblower David McDonnell) as a cause of the technical problems due to the absence of a robust requirements engineering process. Charles Cipione, the technical expert witness to the inquiry, summarised simply, saying that “if you don’t have a good design, it’s not going to work properly.” The failure to address these issues and cover-up attempts led to the Post Office scandal which has been described as the largest miscarriage of justice in British history, linked to multiple suicides with those wrongly imprisoned including a pregnant woman.
The research also found, disturbingly, that software engineers in the UK were 13% less likely to feel they were able to discuss and address problems than those in the US; the largest difference of all engineering practices between the two countries. The finding comes after November 2023 research by Engprax found that 75% of software engineers in the UK faced retaliation the last time they reported wrongdoing.
Author of Impact Engineering, Dr Junade Ali said: “With 65% of projects adopting Agile practices failing to be delivered on time, it’s time to question Agile’s cult following. Our research has shown that what matters when it comes to delivering high-quality software on time and within budget is a robust requirements engineering process and having the psychological safety to discuss and solve problems when they emerge, whilst taking steps to prevent developer burnout. This is fundamental to the philosophy of Impact Engineering.”
“Impact Engineering” is now available on Amazon in Kindle eBook and paperback formats. The business novel is based on real-world case studies of personal and organisational transformations using the Impact Engineering methodology and a newly developed psychological framework for achieving successful transformations, alongside having a chapter describing the underlying scientific basis of the methodology.