Bentiu, the capital of oil-rich Unity state, is now controlled by a military commander loyal to former Vice-President Riek Machar, said Col. Philip Aguer, the South Sudanese military spokesman.
"Bentiu is in the hands of a commander who has declared support for Machar," he said. "Bentiu is not in our hands."
The armed rebels were said to be in control days earlier of some of South Sudan's oil fields, which have historically been a target for rebel movements, endangering the country's economic lifeblood.
South Sudan gets nearly 99 per cent of its government budget from oil revenues, and the country reportedly earned $1.3 billion in oil sales in just five months this year, according to the London-based watchdog group Global Witness.
Although the country's capital, Juba, is mostly peaceful a week after a dispute among members of the presidential guard triggered violent clashes between military factions, fighting continues as the central government tries to assert authority in the states of Unity and Jonglei.
Bor, the capital of Jonglei, is said to be the scene of some of the fiercest clashes between government troops and rebels.
Michael Makuei Lueth, South Sudan's information minister, said Machar was believed to be hiding somewhere in Unity state.
"He is a rebel, he's a renegade and we are looking for him. He's moving in the bushes of South Sudan," Lueth said of Machar.
The U.N. Mission in South Sudan said in a statement Sunday that all non-critical staff members in Juba are being evacuated to Uganda. The mission said the move was "a precautionary measure to reduce pressures on its limited resources" as it continues to provide assistance and shelter to more than 20,000 civilians gathered inside its compounds in Juba, the mission said in a statement.
Hilde Johnson, the U.N. secretary-general's envoy in South Sudan, said the evacuation doesn't mean the U.N. is "abandoning" South Sudan.
"We are here to stay, and will carry on in our collective resolve to work with and for the people of South Sudan," she said. "To anyone who wants to threaten us, attack us or put obstacles in our way, our message remains loud and clear: we will not be intimidated."
Hundreds have been killed in the fighting and world leaders are concerned about civil war in a country with a history of ethnic violence and divided military loyalties.
The U.S. and other countries have been evacuating their citizens from South Sudan. The U.S. has evacuated about 680 Americans and other foreign nationals so far, State Department spokeswoman Jen Psaki said in a statement.
President Barack Obama told U.S. Congress he may take further military action to protect Americans in South Sudan. In a letter to Congress, Obama said that about 46 U.S. troops were deployed Saturday to help evacuate Americans. That's in addition to another 45 troops deployed to reinforce the U.S. Embassy in Juba.
Obama is on his annual vacation in Hawaii, but he said in the letter to congressional leaders that he's monitoring the situation.
"I may take further action to support the security of U.S. citizens, personnel, and property, including our Embassy, in South Sudan," Obama wrote.
On Saturday, gunfire hit three U.S. military aircraft trying to evacuate American citizens in Bor, wounding four U.S. service members in the same region gunfire downed a U.N. helicopter on Friday. The wounded troops are in stable condition, the White House said.
It remains unclear how many Americans are still stranded in Bor and other rural towns.
Earlier this week, the top military general in Bor defected with his troops, starting a rebellion that appears to be spreading to other parts of the country.
Aguer said Bor is still under the control of pro-Machar forces, disputing reports the rebels had fled as government troops advanced on Bor.
South Sudan's President Salva Kiir, an ethnic Dinka, said on Monday that an attempted military coup had triggered the violence, and the blame was placed on Machar, an ethnic Nuer. But officials have since said a fight between Dinka and Nuer members of the presidential guard triggered the fighting that later spread across the East African country.
Machar's ouster from the country's No. 2 political position earlier this year had stoked ethnic tensions. Machar, who has criticized Kiir as a dictator, later said he would contest presidential elections in 2015.
U.N. Secretary-General Ban Ki-moon on Sunday urged South Sudan's leaders "to do everything in their power" to stop the violence.
Foreign ministers from neighbouring countries Kenya, Ethiopia, Uganda and Djibouti were in South Sudan earlier in the week to try and diffuse the crisis.
South Sudan, which became independent in 2011 after decades of a brutal war with Sudan, has been plagued by ethnic discord, corruption and conflict with Sudan over oil revenues.
Although the south inherited three-quarters of Sudan's oil production when it declared independence in 2012, its oil exports are pumped through pipelines running north, raising concern a rebel takeover of southern oil fields could invite Sudan into the conflict.