The problem with Agile, SCRUM and DevOps – and all abstract solutions – is that expectations are always set too high toward unachievable goals.  Requirements management and applications development are just parts of a continuous journey, not the destination, because there is no final destination. Agile is a strategic attitude, not a CPA exam.

It Always Sounds Better Than It Is

Whether it’s Agile, DevOps, SCRUM, enterprise architecture, digital transformation or even cloud computing, we always wax poetic about how they will save a fortune, generate new revenue and, OMG, change everything.  Technologists write business cases, sell them to non-technology executives and then proceed with unjustified optimism.  This time it’s Agile methodology and its cousins SCRUM and DevOps. 

Agile projects fail almost as much as all the others.  Yet we still sell “Agile” as an elixir:  “If we only had an Agile environment, an Agile team and executives who supported Agile transformation, everything would be great.”  Listen to what Andy Hunt says about Agile:

“I am proud to be one of the 17 founders/authors of the The Agile Manifesto back in 2001.  I think it provided a jolt of energy, hope of a better way of doing things, of creating software and making the world work better … but in the 14 years since then, we‘ve lost our way.  The word ‘agile’ has become sloganized; meaningless at best, jingoist at worst.  We have large swaths of people doing ‘flaccid agile,’ a half-hearted attempt at following a few select software development practices, poorly.  We have scads of vocal agile zealots – as per the definition that a zealot is one who redoubles their effort after they’ve forgotten their aim.  And worst of all, agile methods themselves have not been agile.”

There are lots of failures and something of a cottage industry dedicated to explaining why Agile fails so often.  Then there’s SCRUM and DevOps.  They also fail more often than they succeed.

The Problem With Abstract 

Everything’s an abstraction these days.  How many “existential threats” are there?  We need “universal” this and that, but let’s not forget that relativism – one of abstraction’s enforcers – is hovering around all the time making things better or worse, depending on the objective of the solution du jour.  Take COVID-19, for example.  Based upon the assumption that the US knows how to solve “enterprise” problems – the abstract principle at work – the US has done a great job.  But relativism kills the abstraction:  the US has roughly 4% of the world’s population and 25% of the world’s deaths.  How many technology solutions sound good in the abstract, but are relatively ineffective?  

The Agile family is an abstract solution to an age-old problem:  requirements management and timely cost-effective software applications design and development.  But the relative context is way too frequent failure.  We’ve been wrestling with requirements validation for decades, which is why the field constantly invented methods, tools and techniques to manage requirements and develop applications, like rapid application development (RAD), rapid prototyping, the Unified Process (UP) and extreme programming (XP), to name a few.  All of the “new and improved” requirements/development methods, tools and techniques were absorbed by another new and improved requirements management and applications development methodology in what was called the Manifesto for Agile Software Development.  SCRUM was developed in the mid-1990s as yet another approach to software development which is now often subsumed under Agile.  SCRUM has its own methods – small teams, goals, sprints, daily meetings, reviews, etc. – that developers have tried to practice for years (or sometimes only months), but with mixed-to-poor results.  DevOps – Agile’s second cousin – which links development to operations – also has a record of failure especially in the current age of managed services and the proliferation of “software as-a-service” delivery models and low code/no code delivery platforms. 

So where Does that Leave Agile? 

Agile is a really useful state of mind, an important tilt to discipline.  It assumes requirements and development volatility.  It acknowledges development challenges.  But what it should do is recognize the impossibility of applications design and development because no one completely understands the problems software applications are designed solve, or the best ways to develop such applications.  Can we just admit that we don’t know how to build perfect or even near-perfect applications?  That applications – like vaccines – improve over (sometimes long periods of) time?  That rapid anything is just next another step toward an illusionary final destination?  All of the methods, tools and techniques we’ve developed over the decades seek perfection or near-perfection, which are both unachievable.  But even worse is the aspiration of perfection, because constantly striving for an impossible result is expensive and demoralizing, among other unpleasant realities.  This is how methods, tools, techniques and frameworks get misunderstood and why they will continue to be misunderstood until we contextualize them as just steps toward “better” – without recriminations about why “better” just isn’t good enough, when “better” is just fine.

Agile should not be a CPA exam – a set of precise steps that must be followed to completion.  The problem with Agile, SCRUM and DevOps – and all abstract solutions – is that expectations are always set way too high and the goals are therefore always impossible to achieve.  Requirements management and applications development are just parts of a continuous journey, not the destination, because there is no final destination.  Each step is just a step in the journey to understand ever-changing requirements, and develop software applications that satisfy additional requirements that over time – sometimes long periods of time – might approximate improvement.  If Agile can reinvent itself this way, then it might help the always-chaotic, always imprecise and always too expensive requirements management and software development process.  If it remains a complicated formula unto itself it will remain misunderstood and ineffective. Agile should be a strategic commitment to incremental discipline, not a tactical playbook. Agile should be top down, not bottom up. Companies that adopt “strategic Agile” can improve development, but companies that insist upon “tactical Agile” will fail more often than they succeed. Maybe “masters” should be architects, not engineers.