Torrential rains Friday night caused three rivers surrounding the southern city of Mocoa to overflow -- sending a torrent of mud and debris surging through the city.
President Juan Manuel Santos said 170 victims had been identified.
He said 43 children were among the dead and 22 more had been hospitalized. A further 203 people were injured, many severely.
"To all [victims], we send our prayers. We send our condolences to their families, condolences from the entire country," Santos said in an address earlier.
Concerned by the increasingly heavy rainstorms Friday, Simon Uribe and his family shifted from their first-floor riverside apartment to the level above.
Trapped by the river
The river broke its banks 50 meters up the street, splitting and leaving their home effectively surrounded by two strong currents. The electricity cut out.
"At some point, most of the city was (covered) with water. No power but you can see rivers in every street," Uribe told CNN.
"I watched people -- some girls coming down the river -- some bodies coming by the water. It was terrible. It was pretty shocking," he said.
Uribe and his family -- including his 10-month-old son -- escaped to higher ground. The next morning, Uribe found several feet of muddy water where his home had been.
"There was basically no house left," Uribe said. "It was shocking."
Racing against time
More than 1,000 soldiers and national police officers are involved in the rescue effort, and they are facing enormous challenges.
"The difficulties we are facing are that it is still raining in the region and the (mudslide) turned up a considerable amount of land. There are mobility issues on almost 80% of the roads, and where the road ends, it is three hours to the place where the (mudslide) took place," a police spokesman told reporters.
Photos released by Colombia's military showed rescuers carrying old women and children over downed, mud-caked trees and homes.
Santos said Sunday that 10 mobile water tanks were in place and 16 more were on the way to Mocoa, as well as water purification systems. He added that a local hospital was back in operation and medical supplies had been flown in.
He pledged to rehouse those displaced.
"To those who lost their homes, we will build new ones in a safer place. In the meantime, we will give them a monthly rent stipend of 87 dollars. And I have asked the housing minister to begin this initiative today," the President said.
Save the Children said at least 70 children had been separated from their parents.
The organization said it would be working to support children and families in the area and was helping to coordinate hygiene kits and temporary friendly spaces for affected children to attend.
"This is obviously a highly shocking and chaotic situation for the population in the most vulnerable situation, especially pregnant women, children and adolescents," executive director Maria Paula Martinez said in a statement calling for support.
What caused the landslides?
Heavy rains, high levels of deforestation, informal housing and dense human populations are some factors that can leave communities vulnerable to landslides, scientists say.
CNN meteorologist Michael Guy says it appears that the surrounding region had saturated ground from above-average rainfall this year.
During March and April, the rainy season ramps up with moisture moving in from the Amazon River Valley. Estimates suggest some 127 to 200 mm (5 to 8 inches) of rain fell in the past week.
"Taking into account geography for the region, topography, deforestation due to agriculture, etc ... this creates the perfect environment for mudslides/landslides on the leeward side of the Andes mountains where this town is located," Guy said.
"The floodwaters got stuck up in the mountain and when it came down many people didn't have time to react and they were washed away," a survivor told CNN.
Running for their lives
Aerial footage of the site showed some rooftops poking above the muddy deluge that flattened other homes, bridges and highways.
Power and water supplies to Mocoa have been cut by the disaster, and the hospital system has shut down, firefighters say.
Gabriel Umaña, a spokesman for the Colombian Red Cross, told CNN that 300 families had been displaced and more than two dozen homes had been flattened.
Santos personally comforted Marcelo Garreta, who said he could see dead bodies being carried away by floodwaters but was powerless to help.
"We couldn't help anybody, because if we tried we would've been washed away as well. I saw light poles washed away by the floodwaters. This is a great tragedy," Garreta said.