NHS England’s Darwinian Trap: Why Evolution Doomed It to Fail & What Lessons Can We Learn

This week it was announced that NHS England is going to be abolished, well, at least in two years’ time, which, in bureaucratic terms, is lightning fast. But why did it fail? I argue that it was evolution. NHSE did not change the system, the system changed NHS England. The concept of a Darwinian Trap provides an excellent insight to how and why NHSE failed, and how it could have been avoided and what lessons can we learn.

The Announcement of the Abolition of NHS England

Keir Starmer took to the stage this week to talk about cutting bureaucracy and how this fancy new AI whathmacallit was going to magically transform the civil service. In a speech very light on real detail came the announcement of the abolition of NHS England.

Keir Starmer said the reason for getting rid of NHS England was to:

“Free the NHS to focus on patients, less bureaucracy, more money for nurses, refocusing on cutting waiting times at your hospital.”

Why Was NHS England Established

To those with long memories, this might sound familiar to comments made when NHS England was set up in 2013 as a politically independent body:

• “That decisions are driven by the needs of patients and the public.” (NHS England Putting Patients First 2013)
• Reduce bureaucracy, increase efficiency, and streamline services.
• “Turn taxpayers’ money into good clinical outcomes.” – Sir David Nicholson
• “To ensure that patients are not left waiting unnecessarily for treatment.” (Health and Social Care Act 2012)

Et Tu Streeting?

When making the announcement to parliament, the Health Secretary, who until recently had been very complimentary about NHS England, turned on them in a venomous political assassination. Even Brutus would have been shocked by the sudden change in the once Mr Nice’s tone:

Wes Streeting described NHS England to parliament as an “inefficient bureaucracy,” adding: “The budget has soared to £2billion. Taxpayers are paying more and getting less”. Demonstrating the inefficiency by pointing that “There are more than twice as many staff working in NHSE and DHSC  combined today than they were in 2010.” and that “Frontline NHS staff are drowning in the micromanagement they are subjected to by the various and vast layers of bureaucracy”. Finally, far from the two very different organisations envisaged when NHS England was set up they were “Two organisations doing the same roles with an enormous amount of duplication.”

NHS England and the Paradox of Failure

There is a significant paradox here: How did NHS England become the very thing it was set up NOT to be? How did it become a duplication of the organisation that it was supposed to be different to. The very problems it was created to solve became the reasons for its destruction. Why did politicians ultimately find it indistinguishable from the civil service? How did it become yet another ‘inefficient bureaucracy’? This is a lesson we should all pay attention to.

What is a Darwinian Trap

In his book The Darwinian Trap Kristian Ronn describes how organisms and organisations can fall into Darwinian traps were they optimise for short term survival. But then end up being maladapted to a changing environment. These traps occur when organisms, and organisations, continue to rely on outdated strategies that no longer serve their best interests in a new situation.

Traits or behaviours advantageous for survival and maximising success in the short term and reproduction become unfitted when the environment changes, leaving them as easy prey.

The giant panda is an example of this. Perfectly adapted to bamboo forests. It can only live off bamboo, it has to eat all day and is a pretty bumbly clumsy creature. When there’s no bamboo it would starve. Without human help, there would most likely have died out.

In the private sector, we can point to Kodak focusing on film cameras long into the digital camera era until it simply failed to exist.

NHS England’s Darwinian Trap

In the case of NHS England, in order to court political funding and support rather than focus on clinical improvement and innovation as had been explicitly intended, it ignored the NHS constitution which clearly focuses on how healthy and well people live:

“The NHS is there to improve our health and wellbeing, supporting us to keep mentally and physically well, to get better when we are ill, and when we cannot fully recover, to stay as well as we can to the end of our lives.”

NHS Constitution.

It became focused on power and optimising activity through funding. Indeed, efficiency became not doing things better, but simply the requirement of everyone conforming to NHS England’s rules and targets. Assurance was seen as a precious mineral of great value to be mined out of the healthcare system. Whilst the Orwellian ‘Unwarranted variation’ was seen as something to be attacked and eliminated. Unintentionally, meaning that variation and therefore innovation became less and less tolerated throughout anything and everything NHS England touched..

NHS England in a Bureaucratic Environment

Far from combating bureaucracy, NHS England found itself in a bureaucratic environment. As Marshall McLuhan says, “the message is the medium”, the environment dictates what the organisation is and does. Many people (ok very nearly everyone) recruited into NHS England therefore were bureaucrats from the wider system. 

A Clinicially Non-Clinical Environment.

Whilst NHS England itself was always an alien place to clinicians, it treated no patients (I’m reminded of the Yes minister episode where the most efficient hospital in the NHS is full of bureaucrats but hasn’t opened yet! Made 30 years before NHS England was established, this is still a very good metaphor for what NHSE would become, and was located in an immaculate shiny glass office in London. (Not quite an ivory tower, but basically the modern equivalent) If the staff had been hosted at NHS Hospitals and GP practices throughout the UK it’s relationship to clinicians and the work would be very different.

Why NHS England Did Not Become Clinically Focused

Without being firmly anchored in the clinical environment, NHS England could never truly be clinician-focused. There were undoubtedly some great clinicians who worked there and led initiatives, but they rarely influenced policy. They were often the acceptable product of the system than genuine implementers of policy. 

Noticeably, every NHS England Chief Executive was a healthcare manager, not one was a clinician. For an organisation intended to focus on clinically-led decisions, this was a major, even fatal mistake. Though maybe this was unsurprising given executives were always picked by bureaucrats. Which bureaucrat would choose a clinician vs a bureaucrat? And therein lies the fundamental problem of NHS England. It was constantly usurped by bureaucracy from the shadows.

Optimising for Bureaucracy

AS time goes on NHSE system has become increasingly optimsed for bureaucracy. Anything not focused on funding for targets was optimised out of the system and deprived of funding. Such as many of the CQUINN schemes, which were often genuinely good bonus schemes promoting clinical initiatives, funded initially from extra cash then recycled imaginary cash (That’s a tale in itself) then disappearing entirely. 

Evolutionary Selection Bias

Evolutionary selection bias over the years has worked against long-term investments, clinical innovation, standards improvement, public health and prevention, community based work, and large scale digitisation schemes. Anything not fitting predictable targets and fixed funding programmes was aggressively selected against, closing the Darwinian trap and preventing innovation and new value creation. Leaving NHSE stuck and unable to change. Meaning that a 10 year shift was suddenly seen as somehow being ‘radical’.

Funding: the Lure Into The Trap

The funding mechanisms of the civil service and NHS perceive everything as costs. People training, technology are all costs…not benefits or assets. There is no ‘rewards’ to be won from investments, especially in the tight one year funding cycles. So the focus becomes to maximise budget predictability and minimise risk for fixed outputs (not those tiresome outcomes!). Slow, accurate decisions are favoured to minimise risk, slow spending and maximise fund control. 

So naturally the funding administrators found their perfect partner in ponderous and control orientated bureaucratic decision-making. This created a self-reinforcing relationship where one supported the other to grow. (Read more about how NHS funding approaches stifle growth in this post)

Funding as a Pathogenic Path for Bureaucracy

Arguably, the funding mechanism itself can be seen as the main pathogenic path for spreading bureaucracy within the health service. With any funding requiring following strict requirements. To win funding, organisations must demonstrate a bureaucratic structure for reporting, monitoring, and controlling fund usage. Spreading bureaucracy both inside and out of organisations and across the NHS. Actively growing bureaucracy in other organisations and starving those that did not comply.

The Power of the Evolutionary Drivers Have Swallowed Others 

The power of this evolutionary trap has not just swallowed NHS England. In recent years NHS Improvement, NHS Digital and Public Health England have been sucked into the evolutionary pit of bureaucracy. First being amalgamated into NHS England, then having their functions paired back to the bare minimum to fit into the bureaucratic fold. 

Indeed, the largest damage of this particular trap is almost certainly not NHS England, but the damaging evisceration of Public Health England and its functions. Few would argue that public health is not in crisis in the UK and there is a desperate lack of ownership of the problem. Scandalously public health in England has been left in the invisible no man’s land of “somebody else’s problem”. Someone needs to get the message to the government that this is one of the main reasons that hospital waiting lists are so big.

The Value of NHS England?

NHSE Focused on Value Creation

Was the death of NHS England inevitable? If it had good and courageous leadership, standing up for clinicians patients and communities, it could have led the way. To justify its role, NHS England needed to demonstrate the unique value it created clearly, highlighting its distinct contribution to the healthcare ecosystem. Showing a real benefit from its existence. The lack of anyone at all demanding for NHS England to be saved is a sad indictment of its lack of benefits.

How Much Value Did NHSE Need to Create?

The current cost of NHSE is £2 billion. Due to loss aversion (where losses are felt more acutely than gains), NHSE would have needed to demonstrate clear functional gains within the NHS of 2-5x it’s costs so £4–10 billion annually. This could have been achieved through innovation, improved outcomes, efficiency, and preventing illness, improved management of long term conditions. So how much value did actually NHSE create? Was it £2billion a year well spent?

Lack of Value Demonstration

Sadly noone knows how much value it created, despite repeatedly demanding evidence based practice and thorough evaluations from everyone else. NHSE appeared shamelessly averse to demonstrating the value of its efforts, never evaluating ANY of its massive plans at all! Without demonstrating learning, progress, or clear value delivery or learning from failed projects and plans, how could it ever improve? It’s an incredibly hard job running healthcare. But not looking at the results of your actions turns into guesswork

By default the Darzi Report became the only way that the NHSE could demonstrate its value. It was no surprise to anyone that the Darzi report  was damning. “It is inevitable that public trust and confidence will have been damaged by the inability of the NHS to meet the promises of the NHS Constitution.”

NHSE Stuck in a Niche

Rather than value creation, NHSE instead focused on defining its role ever more narrowly, relying on assurance oversight and financial monitoring as its primary roles. Because of it’s lack of value creation in evolutionary terms it became a sad parasite on the healthcare system, sucking out resources before it could get to anyone else. Rather than an active contributor to the system. This meant that NHSE ended up competing directly with DHSC in the same niche. It became inevitable one would eat the other.

Bureaucracy Doesn’t Work

All of this really happened because bureaucracy doesn’t work. NHSE were a busy organsiation, they have produced several plans (big and small), countless targets, monitored 1000s of KPIs and untold number of action plans for those around them. Constantly monitoring and assuring the rest of the system. Yet this business did not result in improved healthcare. It focused on measurement rather than understanding the problem. Almost every failure listed in the Darzi report had previously been a declared objective by NHS England. Whilst providers have been beset by scandals, declining performance and financial losses. If bureaucracy was effective none of this would have been a problem. 

Was Turning NHSE Around Possible?

Had NHS England focused genuinely on clinical innovation and value creation, instead of reinforcing bureaucratic systems, it could have created a virtuous circle sustaining itself. However, trapped in a bureaucratic environment dependent on DHSC resources, it could not escape this dependence.

Far too often it was too timid to develop a strong voice representing the needs of staff and patients. In typical fashion, it talked long and hard about empowering patients and staff, recognising diversity, inclusion, and cocreation in making decisions. Whilst not actually DOING it. If it had listened and given power to others, its story may have been very different. 

The Bureaucracy is Dead Long Live the Bureaucracy.

Just because NHSE has been abolished in a war on bureaucracy does not really change anything. Whether the bureaucrats running around wear a ‘NHSE lanyard’ or a DHSC one does not really make a difference. It will be basically the same people saying the same stuff. As Keir Starmer says NHSE has been brought into the heart of government. That is into the warm embrace of bureaucracy. The bureaucrats are very much still in charge. It is simply putting the bureaucratic emperor in some fresh new clothes.

(If you want to learn more about why cutting bureaucrats doesn’t get rid of bureaucracy read here.)

So the NHS is Doomed Forever to be a Bureaucracy?

Is the NHS doomed forever to be entrapped in a dysfunctional bureucracy? The answer is no, it doesn’t have to be.  The New NHS 10 year plan is still an opportunity for escape. If the NHS can move its locus of control from bureaucrats in offices to local clinical led services, rooted in the needs of communities, then the environment that decisions are made in can transform the effectiveness and function of the NHS. There are several examples of excellent community schemes and clinical initiatives. This is the promise that politicians have alluded to. Whether that is actually allowed to happen under the ever manipulative clutches of their own bureucrats is another story. But yes there is hope.

Bureaucracy in a Technological Society

Bureaucracy is very much a disease of human decision-making, existing for human power and control. But AI has no need for bureaucracy, gives knowledge and power in the hands of everyone. The pace of change is immense. The power will shift from our institutions to organisation groups and people who can adapt and change with technology and move faster than bureaucracies can understand what is going on. 

It seems highly likely that the slow plodding obstructive decision making of our bureaucracies will find themselves in their own Darwinian trap. Unable to adapt and compete in societies that are technologically moving in ways that bureaucracies can not make sense of let alone control. The evolutionary battle for survival of NHSE is lost. But the battle for survival of the bureaucracies may just be beginning.

Key Learnings About the Death of NHS England

Learning 1: Bureaucracy has to be Contained Controlled and Replaced

The amazing power of bureaucracy to seduce, subvert and undermine and reinvent itself can’t be overstated. At heart it is no more than a bad decision making process, but one that suffocates everything we do with pointless work. It engages in endless meetings, reports and data collection and discussions, all to inevitably lead us to the obvious “Minimum Acceptable Solution” that was probably known from the start.

 It dresses in fine clothes pretending it is something else, a rational, logical way of making decisions. It creeps its way into everything we do and pretends it is ‘how things should be done around here’. It substitutes speed, purpose and effectiveness for stagnation, power and control.

If we are to fix our organisations we have to recognise it as the enemy of excellence, learning and progress. Using it only when stability and control are necessary, and so it is only used in the right place at the right time by those who understand its toxic infectious nature. 

Learning 2: Hoarding Power and Control is a Road To Failure.

At every stage NHSE prioritised control and power over other people. So many of the processes and plans, monitors and controls are deeply paternalistic. Instead of defining problems and solutions to be imposed on everyone. It has forgotten the core goal of its charter of improving the health and wellbeing of society. Creating narrow thinking where care only matters inside the hospital door or the doctor’s office. 

NHS England should have been the voice and champion of OUR NHS. Instead of responding to the needs of a diverse society. It created tickboxes to put everyone into. Then pass the buck on addressing society’s needs. This was perhaps it’s fatal failure. As NHS (or any system) that does not respond to the society it is in ends up failing (Ashby’s Law of Requisite Variety) So the NHS has ended up increasingly swamped with failure demand, trying to treat society better rather than responding to it’s needs. Resulting in a physical and healthcare crisis and a broken NHS.

Learning 3: Our Purpose Defines Our Future

Purpose is not some convenient ideal. The purpose of our teams organisation and ourselves has to focus on the value we create for others. It must define our niche and our function, who we are and what to do. Focusing instead on opimtising for short term goals and short term survival may help you thrive…in the short term. But ultimately could lead you falling into a Darwinian trap, getting stuck down an evolutionary blind alley, leaving you as vulnerable prey for others.

No-one is really to blame for the failure of NHS England; and whilst there were some isolated successes, NHS England did fail in almost every way possible. There are some brilliant people who have worked and continue to work at NHS England but it is the system itself that failed rather than any individual.

Ultimately NHS England failed because it became the system rather than changed the system.

Do you have any lessons and observations that we can learn?

If you would like to maximise your evolutionary potential and not fall into someone else’s trap. Come join me at the Edge of Possible. (EdgeofPossible.com)

Conclusion

The abolition of NHS England provides a stark yet essential lesson about the dangers of bureaucratic entrapment and short term thinking. NHS England’s fate was sealed not by its initial ambition but by its submission and spread a system that prioritised bureaucratic self preservation over genuine patient care and innovation. Escaping this Darwinian Trap requires shifting power away from institutional control and placing it firmly into the hands of communities and clinical leaders. By embracing adaptability and aligning decisions closely with real, local needs, the NHS still has a chance to thrive. The real test for the future of healthcare is whether it can harness change rather than fear it, turning evolution into a tool for growth, not a trap.