I spent the best part of a decade as an analyst, writing specs that were wrong before the ink dried. Then the next decade in Agile teams, having conversations that evaporated the moment someone left the room.
For a long time I told myself it was a methodology problem. Pick the right framework, train people properly, get the ceremonies right - and the work would flow.
Earlier this year I began to consider that perhaps I had been the type of person I often berated. A methodology obsessive when, ironically, the methodology was never the problem.
Waterfall didn't fail because we had too much documentation. Agile didn't fail because we had too little. They both failed for the same reason: humans couldn't keep up with the expectations of the people paying for the product.
Recently a CTO made the case that AI enables a methodological pivot back toward Waterfall. His logic: Agile succeeded by reducing documentation overhead and promoting smaller batches of work with faster verbal feedback. Now that AI makes documentation cheap and fast, why not embrace comprehensive specs again?
AI does make documentation cheap and fast. But that conclusion misses why Waterfall failed in the first place - and, more importantly, why Agile teams often got ridiculed.
Start with Waterfall. We couldn't keep documentation synchronised with reality.
There's a telling truth buried in every piece of software: the only documentation that stays accurate is the code itself. Code does exactly what it does. It doesn't lie. Architecture diagrams, requirement documents, product backlogs, test plans, training materials - all drift. They were rarely executable, and reviewing them meant a human sitting down and trying to stay awake through reams of pages that might already be wrong.
We treated code as a byproduct of documentation, when it was always the other way around. The code was the truth; everything else was a hypothesis about it. And the moment the first line was written, the documentation started lying.
Take a classic Waterfall project: three months gathering requirements into a 200-page spec - illustrative numbers, with some attractive UML diagrams to match. Stakeholder sign-offs across months of review cycles. Then development. By the time you ship, the market has moved - or, most crushingly for me, you discover that everything you had committed to paper had been interpreted very differently from what you knew was needed, leaving you with a colossal turnaround effort to salvage some form of usable software, as well as your reputation.
We tried to solve this with change review boards and formal amendment procedures, which just made things slower and widened the gap further.
We treated it as a discipline problem. But that was like asking sprinters to outrun a car: the discipline was never the issue. The speed differential was.
So Agile said: talk instead. Only document when the need is immediate - and always remember, working software over comprehensive documentation. And it was better. At least I was happier, and the teams I led seemed to be too.
But we swapped one speed problem for another.
User stories were supposed to be placeholders for conversation - then became an Issue Type in Jira. Conversations happened, and then what? Someone probably should have written something down, but more often they didn't, and knowledge got lost. Or they did, and the write-up captured most of the nuance, and three months later no one could remember what was really meant.
Specification by Example tried to fix this by making tests the documentation. It was a revelation for me and my teams, but as soon as deadlines began to nip at our heels, writing tests up front was perceived as wasteful because it took developers away from writing code. You can guess what happened next.
Backlogs became graveyards of half-understood tickets. Stand-ups became status rituals. Neither gave us a mechanism to share knowledge fast enough.
What I've noticed, across two decades of this, is that verbal communication is brilliant in a room of five people. But what happens when someone joins three months later? Or a decision needs revisiting six months on? Humans don't consistently share detailed knowledge at scale except verbally - and beautifully, we are designed to forget. Verbal knowledge doesn't persist. If you've ever spent half a day hunting for "why we decided to do it this way," you've hit that wall.
Neither methodology failed because people were lazy or incompetent. They failed because both hit the ceiling of human processing speed against everyone else's expectations.
A commercial team gets a customer question. Someone pulls data from multiple systems, can't find everything they need, and starts building a spreadsheet to make sense of what they have. By the time they've assembled something presentable, the quarterly business review they should have been preparing has been quietly neglected because the data hunt ate the day.
Or internal knowledge: how many times have you needed to know why something was built a certain way, and had to track down the one person who remembered - or just made your best guess because the context was never captured?
It's a trigger for me when people call this a process failure. It's a bandwidth constraint.
AI doesn't force a choice between Waterfall's comprehensiveness and Agile's adaptability. It removes the speed ceiling that made both fail.
Rather than trying to force all organisational knowledge into one maintained system, agents can navigate the scattered islands of knowledge that already exist - CRM, support tools, data platforms, past decisions captured in Slack threads or old documents. That connective layer is nascent, still rough in places, and won't run without human judgment keeping it honest. But it's directionally right in a way that neither methodology ever managed: knowledge that flows at something closer to verbal speed, without demanding that someone manually keep it current.
That CTO asked: "Should we pivot back to Waterfall?" It's the wrong question.
The right question is: what becomes possible when human speed stops being the bottleneck?
For data-intensive businesses, the answer matters. You can turn data into decisions faster, more comprehensively, and more defensibly - if you pair AI's throughput with the domain expertise to know which questions matter, which data to trust, and what the customer actually needs. That judgment becomes more valuable, not less, because it finally has the throughput to match it.
AI doesn't favour Waterfall or Agile. It removes the constraint that made both fail. What we build next won't look like either methodology - it'll look like the thing we've been trying to build for decades: knowledge work where the bottleneck is the quality of thinking, not the speed of the humans doing it.
The question is whether you start building that now, or spend another decade debating which framework to use.
The Builders Blog is a practitioner-first publication on AI, aviation data, and building things that actually work. Subscribe to The Builders Blog LinkedIn newsletter here.