Nancy had peeled off to the lavatory as the Camerons and friends got into their cars at the Plough Inn, near Chequers, the prime minister’s official residence near London, media reports said.
Mrs Cameron assumed that Nancy was travelling in her father’s car with his bodyguards, while the prime minister believed her to be with her mother.
The parents realised that she was missing after 15 minutes, when they got to Chequers, and were “distraught,” said the spokesman.
“Thankfully when they phoned the pub she was there safe and well. The Prime Minister went down straight away to get her.”
When he arrived to collect Nancy he found her helping staff.
The Camerons were at the pub with Nancy and their other children Arthur, six, and Florence, 22 months, as well as two other families.
The incident happened “a couple of months ago”, a spokesman said, but was only confirmed on Monday.
The couple’s oldest son, Ivan, who had cerebral palsy, died in 2009.