The revered 82-year-old emperor's comments came in only his second-ever televised address to the public.
Emperor Akihito did not explicitly say he wanted to abdicate as he is barred from making political statements.
PM Shinzo Abe said the government would take the remarks "seriously" and discuss what could be done.
"Upon reflecting how he handles his official duty and so on, his age and the current situation of how he works, I do respect the heavy responsibility the emperor must be feeling and I believe we need to think hard about what we can do," he said.
Akihito, who has had heart surgery and was treated for prostate cancer, has been on the throne in Japan since the death of his father, Hirohito, in 1989.
In his 10-minute pre-recorded message, he said he had "started to reflect" on his years as as emperor, and contemplate his position in the years to come.
If he were to abdicate, it would be the first time a Japanese emperor has stepped down since Emperor Kokaku in 1817.
The BBC's Rupert Wingfield-Hayes in Tokyo says right wing nationalists who support Mr Abe's government do not want any change to the current law, which insists emperors must serve until they die.
Emperor Akihito said he hoped the duties of the emperor as a symbol of the state could continue steadily without any breaks.
He said one possibility when an emperor could not fulfil his duties because of age or illness was that a regency could be established.
But he suggested this was not the ideal outcome, saying: "I think it is not possible to continue reducing perpetually the emperor's acts in matters of state and his duties as the symbol of the state."
Akihito's eldest son, 56-year-old Crown Prince Naruhito is first in line to the Chrysanthemum throne, followed by his younger brother Prince Akishino.
Women are not allowed to inherit the throne and so Princess Aiko, the daughter of Crown Prince Naruhito, cannot succeed her father.
Why can't the emperor abdicate? Abdication is not mentioned under Japan's existing laws, so they would need to be changed for the emperor to be able to stand down. The changes would also have to be approved by parliament.
What do the public think? Most support the emperor's desire to step down - a recent survey by the Kyodo news agency found more than 85% saying abdication should be legalised. But the move is opposed by some more conservative sections of Japanese society.
Is this the first time a revision of the law has been discussed? A debate about whether or not a woman would be able to ascend the throne was triggered in 2006 when the emperor had no grandsons, but was postponed after a boy was born to the imperial family.
What does the emperor do? The emperor has no political powers but has several official duties, such as greeting foreign dignitaries. Japan's monarchy is entwined in the Shinto religion and the emperor still performs religious ceremonies. He also plants and harvests a small rice paddy inside the palace while the empress raises silkworms.