The demonstrations come after Mr Zuma's sacking of a respected finance minister, which led to the country's credit rating being cut to junk status.
The move added pressure to South Africa's already embattled economy.
But supporters of Mr Zuma have also turned out to defend the president.
This week Mr Zuma survived calls by powerful groups allied to the governing ANC for him to go. He also got the backing of a major decision-making body within the party.
One of the immediate sources of controversy has been last week's cabinet reshuffle which saw the removal of, among others, Finance Minister Pravin Gordhan.
In Johannesburg, there was a heavy police presence as protesters planned to converge near the ANC headquarters.
Hundreds of veterans of the ANC's armed anti-apartheid struggle and members of the ruling party's youth wing also gathered outside its headquarters at Luthuli House to protect the building.
Supporters of Mr Zuma told the BBC's Milton Nkosi that they came out to promote the president's policies. They said that Mr Zuma was promising radical socio-economic change.
One supporter said that it would be wrong to force regime change in South Africa, adding that such a move would be "going back to the dark ages of apartheid".
Police fired tear gas and rubber bullets at protesters in one part of Johannesburg, local media are reporting.
Crowds later gathered outside a compound belonging to the wealthy Gupta family, which has ties to the president, in Johannesburg's affluent Saxonwold suburb.
Pictures shared on social media on Friday showed large crowds assembling in Church Square in Pretoria, despite confusion over whether demonstrations in the capital are permitted by law.
Police said on Thursday that marches in Pretoria were illegal because they had not been granted permission by city authorities. But this was later overturned by a magistrate. Protesters in the capital later marched to the seat of government, the Union Buildings, waving placards that read: "South Africa is not for sale."
The BBC's Karen Allen, in Pretoria, said that despite a heavy police presence, the protest there was going ahead peacefully.
In Durban, banners stating "Jacob Zuma must step down" were draped from buildings as part of nationwide demonstrations. Protesters marched through the city with their faces painted, carrying messages of "downgrade Zuma, not South Africa" in reference to the country's amended credit rating.
The government, which has appealed for peace during the countrywide protests, tweeted that the laws in South Africa are also there to "protect the right of those who would not like to participate in protest action".
Earlier this week, a major decision-making body within the ruling ANC party discarded a complaint against Mr Zuma that he had failed to consult executives before reshuffling his cabinet.
The move to drop the finance minister angered both Mr Zuma's opponents and allies, causing a rift in the ruling ANC party, which has governed South Africa since 1994.
It left some in the ANC leadership questioning whether Mr Zuma should remain as president.
Key ANC allies, the South African Communist Party (SACP) and the main trades union federation Cosatu, joined in the calls for him to go.
But the party's National Working Committee (NWC), discussing the cabinet reshuffle, later gave the president its backing.
Mr Zuma has been described in local media as the Teflon Don because of his survival skills following the acceptance of his explanation for his controversial sacking of the finance minister.
Opposition parties, together with some in his own governing alliance, planned Friday's mass action through public protests to demand his resignation.
The Economic Freedom Fighters went to court asking permission for his impeachment.