A ceremony, attended by PM Shinzo Abe, was held at Hiroshima's memorial park before thousands of lanterns are released on the city's Motoyasu river.
The bombing - and a second one on Nagasaki three days later - is credited with bringing to an end World War Two.
But it claimed the lives of at least 140,000 people in the city.
A US B-29 bomber called the Enola Gay dropped the uranium bomb, exploding some 600m (1,800ft) above the city, at around 08:10 on 6 August 1945.
On that day alone, at least 70,000 people are believed to have been killed. Many more died of horrific injuries caused by radiation poisoning in the days, weeks and months that followed.
People across Japan have observed a minute's silence to mark the anniversary. In Hiroshima a bell tolled at 08:15 local time - when the US aircraft dropped the bomb that flattened the city centre.
Addressing 40,000 people who attended the commemoration ceremony at Hiroshima's peace park near the epicentre of the 1945 attack, Mr Abe called for worldwide nuclear disarmament.
He said that that atomic bomb not only killed thousands of people in Hiroshima but also caused unspeakable suffering to survivors.
"Today Hiroshima has been revived," the prime minister said, "and has become a city of culture and prosperity.
"Seventy years on I want to reemphasise the necessity of world peace."
Mr Abe and Hiroshima Mayor Kazumi Matusi were joined by US Ambassador to Japan Caroline Kennedy for the official ceremony of remembrance on Thursday, which included silent prayers, the release of doves and a declaration of peace.
Mr Matsui described nuclear weapons as an "absolute evil" while urging the world to put an end to them forever.
"To coexist we must abolish the... ultimate inhumanity that is nuclear weapons. Now is the time to start taking action," he said in his annual speech.
Later in the day, thousands of paper lanterns will be released on the city's Motoyasu River - symbolising the journey to the afterlife of those who died.
The anniversary comes as divisions in Japan rise over Mr Abe's bid to pass unpopular legislation to expand the country's military role worldwide.
The bomb that changed the world
- The bomb was nicknamed "Little Boy" and was thought to have the explosive force of 20,000 tonnes of TNT.
- Colonel Paul Tibbets, a 30-year-old colonel from Illinois, led the mission to drop the atomic bomb on Japan.
- The Enola Gay, the plane which dropped the bomb, was named in tribute to Col Tibbets' mother.
- The final target was decided less than an hour before the bomb was dropped. The good weather conditions over Hiroshima sealed the city's fate.
- On detonation, the temperature in Hiroshima reached 60 million degrees, killing and injuring thousands of people instantly.