Sir Olly Robbins, the UK Foreign Office’s top civil servant, has been forced out of his post after the decision to fail Peter Mandelson during his security vetting was overruled by his department.

Robbins was the Foreign Office’s most senior official in late January 2025 when the decision was made, paving the way for Mandelson to become the US ambassador.

The Guardian understands that Robbins was told he had to resign after prime minister Keir Starmer and foreign secretary Yvette Cooper lost confidence in the civil servant.

The Guardian revealed on Thursday that Mandelson was initially denied clearance in late January 2025 after a developed vetting process, a highly confidential background check by security officials.

Keir Starmer had by then announced he would be making Mandelson ambassador, posing a dilemma for officials at the Foreign Office, who decided to employ a rarely used authority to override the recommendation from security officials.

Mandelson’s failure to secure vetting approval had not previously been publicly revealed, despite intense scrutiny over his appointment and the release by the government of documents supposed to shed light on the case.

The Guardian understands that Yvette Cooper and her office only became aware that Peter Mandelson had failed security vetting when the newspaper broke the story on Thursday.

The foreign secretary spent the evening in the department and Downing Street with the prime minister where she spoke to Robbins. They concluded that he could no longer continue in post.

Amid calls from opposition parties for the prime minister himself to resign, Downing Street released a statement late on Thursday stressing that Starmer and David Lammy, who was foreign secretary at the time, had no knowledge that security officials advised Mandelson should not be given clearance, and said responsibility lay with the Foreign Office.

“The security vetting process for Peter Mandelson was sponsored by the FCDO. The decision to grant Developed Vetting to Peter Mandelson against the recommendation of UK Security Vetting was taken by officials in the FCDO,” a spokesperson said.

The Guardian understands that Starmer – who insiders said was furious – first learned that Mandelson had failed security vetting on Tuesday this week, while Lammy learned about it when the Guardian broke the story two days later.

Robbins, the UK government’s former Brexit negotiator, took over as permanent secretary from Sir Philip Barton in early January 2025. He had been in the job just three weeks when the decision regarding Mandelson was made.

Although he left the civil service in 2019, he had years of experience in international affairs and national security, which helped him navigate a complex global landscape, with the return of Donald Trump to the White House and conflicts in Ukraine and the Middle East.

Speaking on BBC Newsnight, Labour MP Chris Curtis called Robbins’ departure “the right decision”, but maintained that the news around Mandelson should not lead to speculation over “who should lose their roles”.

“There have been a series of processes and decisions that have been made right at the top of government that have been the wrong ones,” he said. “And we as a government, as political leaders, need to learn from those and ensure that things like that do not happen again. It is a good thing that someone has been held responsible for this, but it’s far bigger than that, far bigger questions need to be asked.”

Other figures are calling for Keir Starmer to resign over the recent revelations over Mandelson’s appointment. One anonymous government minister reportedly told The Times: “This is a tipping point. There can be no more excuses; we’re past apologies and there cannot be another fall person in the form of Olly Robbins.

“The PM must go.”