For decades, political leaders have repeated that a government's foremost responsibility is protecting its people. On 11 September 2001, the United States plainly failed in that obligation. Relevant intelligence existed, but it was not connected quickly enough to prevent catastrophe.

The attacks later known simply as 9/11, and the response that followed, shaped foreign policy, security policy and intelligence work for much of the first two decades of this century.

On 11 September 2001, when al-Qaeda hijackers flew passenger aircraft into the World Trade Center, I was nearing the end of an assignment in the Middle East. A Palestinian taxi driver was taking me through the wooded hills west of Jerusalem towards Tel Aviv's Ben Gurion airport when he raised the volume on the radio. “Something major has happened in America,” he said. He was right. That day, 2,977 people were killed.

The assault was not entirely unforeseen. The CIA had monitored al-Qaeda and its Saudi-born leader in exile, Osama bin Laden, for years before 2001. It even maintained a dedicated unit tasked with tracking him.

Nine days after the attacks, President George W Bush announced what he called the “War on Terror”.

Across the following quarter-century, intelligence agencies on both sides of the Atlantic have recorded important achievements as well as grave mistakes. Plots have been disrupted, suspects detained, prosecuted, convicted and imprisoned.

Yet the larger question remains: how have intelligence collection, counter-terrorism and strategic decision-making developed since the deadliest terrorist attack in American history?

There is little doubt that 9/11 represented an immense intelligence breakdown, more precisely a failure of imagination within the US intelligence system. Both the CIA and FBI held information that, had it been properly exchanged and assessed, might have changed the outcome.

The subsequent 9/11 Commission Report pointed to failures in imagination, policy, capability and management. It found that the CIA and FBI did not adequately evaluate intelligence, share it effectively or take enough action to disrupt the conspiracy.

That fundamental lesson has largely, though not entirely, been absorbed. Information-sharing among Western intelligence services is now vastly stronger than it was in 2001.

Former institutional rivalries, including those between MI5 and police counter-terrorism teams, have been reduced. Regional fusion centres were established across the UK, followed more recently by the high-security Counter Terrorism Operations Centre in London. There, MI5 officers can work immediately beside police staff responsible for operational intervention. They may also sit alongside FBI personnel or language experts on attachment from GCHQ.

Since 9/11, the US intelligence structure has also been extensively reorganised. A National Counterterrorism Center was created near Washington, DC, alongside a Director of National Intelligence intended to improve coordination among the country’s 18 intelligence bodies. The CIA’s relationship with Britain’s Secret Intelligence Service, commonly called MI6, is among the closest intelligence partnerships anywhere. It is reinforced by the Five Eyes arrangement linking the US, UK, Canada, Australia and New Zealand.

Even so, serious failures in intelligence, judgement and strategic planning have continued since the destruction of the Twin Towers. Before the 2003 US-led invasion of Iraq, MI6 allowed intelligence to become entangled with political pressure, contributing to its most damaging reputational crisis since the betrayals involving Kim Philby and other Soviet spies during the 1950s and 1960s.

Under intense pressure from Washington to demonstrate that Saddam Hussein’s Iraq retained Weapons of Mass Destruction, MI6 supplied Tony Blair’s government with intelligence that proved unreliable. Iraq had possessed such weapons in earlier years, but they had been eliminated after the 1991 Gulf War.

After the 2003 invasion, MI6 overhauled its method for assessing raw intelligence, known internally as CX, received from field agents. There is now a clearer division between officers handling sources, who gather information from places such as Iran and North Korea or from banned terrorist networks including al-Qaeda, and reporting teams responsible for testing, challenging and seeking further corroboration for that material.

“The Iraq war in 2003 rested on profoundly defective intelligence,” says General Sir Richard Shirreff, who served in Iraq as a divisional commander. “Have we learned from it? No, I think we are poor at learning... As a commander, it was painfully clear that we were moving towards strategic failure.”

The Iraq invasion prompted another official investigation, eventually leading to the Butler Review. It concluded that British intelligence on Iraq’s alleged weapons programme had been seriously flawed and depended too heavily on sources that had not been verified.

MI5, the Security Service, has likewise experienced failures despite many demonstrable successes in preventing terrorist attacks. “After 9/11,” says Lord Jonathan Evans, who led MI5 between 2007 and 2013, “the view was that the most urgent threat to Britain by far came from al-Qaeda and associated people and groups.”

But even then, MI5 held records on more than 20,000 Subjects of Interest while conducting many investigations at the same time. Deciding which leads require immediate attention was, and remains, a daily challenge. The service has not always judged correctly.

In 2005, it did not stop the 7/7 suicide bombings in London, which killed 52 people. The Security Service knew of Mohammed Sidique Khan, later identified as the group’s leader, but did not know what he was preparing to do. The 2017 Manchester Arena bombing was another more recent failure.

Terrorism has not disappeared, but Lord Evans argues that the balance of threats has shifted. “Non-state terrorism remains a major concern, but state threats have become increasingly serious and are at least as important to national security as non-state terrorism. They arise in different ways from Russia, Iran, their proxies and China.”

Some would also describe President Donald Trump’s decision to join Israel in strikes on Iran this year as a failure of intelligence imagination. America’s Gulf allies understood that Iran might respond by attempting to close the Strait of Hormuz. The warnings were visible, yet Trump appeared to have no adequate response when Iran acted on its threats, severely disrupting global energy markets.

Immediately after 9/11, many feared another wave of attacks was imminent. Al-Qaeda itself promised a “swarm of planes” targeting further buildings. In the haste to prevent another strike, the US and its allies committed numerous human rights violations. Suspects were detained in Afghanistan and Pakistan, often on the basis of weak or questionable information. Hundreds were transported, restrained, blindfolded and diapered, to the improvised prison at the US naval base in Guantanamo Bay, Cuba, where they were held without trial.

Human rights organisations condemned the system as a lasting blemish on America’s moral standing.

Abuse of detainees in American custody in Iraq and Afghanistan later emerged through a series of scandals. Some individuals were punished, although critics argued that senior officials avoided accountability. The US still uses detention without trial at Guantanamo Bay, while its 2025 deportation of more than 200 men to El Salvador’s controversial Terrorism Confinement Centre drew further condemnation.

Britain’s record in the years after 9/11 was also marked by human rights failures. In 2004, MI6 was involved in the abduction and rendition of Libyan Islamist Abdelhakim Belhaj and his wife from Thailand to Libya. Belhaj spent years imprisoned and was tortured under Colonel Muammar Gaddafi’s regime. When the case became public, the Attorney General delivered a full apology in Parliament in 2018. Today, lawyers are reportedly among the fastest-growing parts of the intelligence agencies.

“We devoted the first 20 years of this century almost completely to counter-terrorism and counter-insurgency,” said Sir Alex Younger, who headed the Secret Intelligence Service from 2014 to 2020. Speaking after leaving the role, he added: “We failed to recognise China’s rise as a military power and strategic challenge.”

General Shirreff made a similar warning in his 2016 political thriller, War With Russia, which imagined a full-scale Russian invasion of Ukraine.

In the years after 9/11 and throughout the War on Terror, Western governments concentrated on fighting the Taliban and Islamic State, or managing the troubled occupation of Iraq. Meanwhile, Russia and China pursued huge rearmament programmes, including advances in hypersonic missile technology that have narrowed the military edge once held by Nato.

“The West’s error was not only becoming distracted by counter-terrorism after 9/11,” says Sam Olsen, chief analyst at the security intelligence company Sibylline. “It also misunderstood China’s rise, assuming that economic integration would eventually bring political convergence.”

China, like Iran, Russia and North Korea, is regarded as a difficult intelligence target. In 2001, biometric systems were still relatively undeveloped. Since then, facial recognition, iris scanning and gait analysis have made it far harder for agents and their handlers to cross borders without detection.

So how deep is Western understanding of China?

Alongside Russia, China is now one of MI6’s principal intelligence priorities.

“Western intelligence knows a great deal about China,” says Sibylline’s Sam Olsen. “The greater risk is seeing separate facts without understanding the whole they create. Chinese strength is no longer only measured in ships, missiles and soldiers. It increasingly rests on semiconductors, critical minerals, batteries, ports, telecommunications and industrial capacity. The issue is often not a lack of collection, but weak strategic interpretation.”

Twenty-five years ago, on the day of the 9/11 attacks, US intelligence officers held parts of the puzzle, a comparison often used by spies. But collectively they failed to assemble those pieces and recognise what was approaching.

Excellent intelligence gathering has limited value if it cannot be used to achieve strategic advantage.

The intelligence and counter-terrorism landscape has clearly changed almost beyond recognition since that clear September morning in New York. Artificial intelligence and quantum computing will accelerate that transformation further, in some areas at extraordinary speed.

But in a world of competing interests and conflicting narratives, Western countries do not have exclusive ownership of intelligence failures, missed warnings or strategic misjudgements. Other major examples include Russia’s disastrous full-scale invasion of Ukraine in 2022 and what some Israelis have called their “own 9/11” — the Hamas-led attack from Gaza on 7 October 2023.

In the end, the most difficult questions concern major national choices. Was Britain right to support President Bush’s invasion of Iraq? Was it right to join the US-led campaign in Afghanistan, then leave the country to the Taliban in a hurried withdrawal two decades later? Those questions, along with many others, continue to echo.