At least nine people died and more than 860 were injured, officials said, as shaking toppled buildings.
The magnitude 6.2 quake struck at 21:26 on Thursday (12:26 GMT), near Kumamoto city, on the island of Kyushu.
Japan's seismology office recorded the shaking at some places to be as intense as the huge earthquake that hit the country in 2011.
That quake sparked a huge tsunami and nuclear meltdown in Fukushima.
Most of those who died in Thursday's quake were in the town of Mashiki, near Kumamoto, where an apartment building collapsed and many houses were damaged.
More than 40,000 people had initially fled their homes, with many of those closest to the epicentre spending the night outside. Some are now returning to their homes despite the frequent aftershocks.
The quake was initially registered as magnitude 6.5 by the US Geological Survey, but this was later revised down to 6.2.
By Friday afternoon, more than 130 aftershocks had hit the area, which is where Japan's only online nuclear reactor is located, according to public broadcaster NHK.
No tsunami warning was issued after the quake struck, and nuclear power stations in Kyushu were apparently unaffected.
The two nuclear reactors in Sendai, in the south of the island, were operating as normal, while the three normally operational reactors at the island's Genkai plant were already closed for routine inspections.
Japanese Prime Minister Shinzo Abe said the government had mobilised thousands of police, fire fighters and troops for rescue operations. His cabinet secretary, Yoshihide Suga, will also visit the area.
"I am praying for the souls of those who perished in the earthquake and offering my sympathy to the families of the deceased, the injured, and the evacuated, and those affected by the disaster," Mr Abe said.
The first deliveries of emergency aid have now arrived.
The quake struck at a depth of 10km (six miles) and was followed by aftershocks.
"The shaking was so violent I couldn't stand still," Hironobu Kosaki, a local police official, told the Associated Press news agency.
Many factories, including Sony and Toyota plants, halted operations after the quake, and have been conducting safety checks to assess possible damage.
One high-speed bullet train, which was not carrying passengers at the time, was derailed and some other rail services were suspended while lines were checked for damage.
Part of the walls and roof of Kumamoto castle were also damaged.
An official in the nearby city of Uki said houses there had also collapsed, as well as part of the city hall's ceiling.
Japan is regularly struck by earthquakes but stringent building codes mean that they rarely cause significant damage.