The demonstrators blocked a major highway with rubble, burned tyres and pelted stones at motorists to show their anger.
Many South Africans are feeling disillusioned and frustrated with the ANC.
The party that saw Nelson Mandela become the country’s first democratically elected president 30 years ago, with nearly 63% of the vote, could lose its parliamentary majority, according to opinion polls.
Mr Mbeki, who served as president from 1999 until 2008, is outspoken on matters of governance – and has not been afraid to openly criticise the ANC and its current leader, President Cyril Ramaphosa.
But dressed in full ANC regalia, he said he felt it was still the party for people, it just needed to be “fixed”.
Addressing the crowds in Xhosa and English, he acknowledged the issues facing the ANC: “Inside the ANC there are rotten apples and thieves.
“Thieves that wear a T-shirt like this,” he said pointing to his own top. “We need to get rid of them so that this party can be one that you know.
“I promise that when I say vote ANC, I promise that we will fix the ANC.”
The ANC’s majority is under threat from the increasing popularity of smaller parties like the Economic Freedom Fighters (EFF) – popular with young people – and the new uMkhonto weSizwe (MK) party, led by Mr Mbeki’s successor Jacob Zuma.
Mr Zuma was forced to step down as president in 2018 because of corruption allegations. Three years ago, he was sentenced to 15 months in jail for failing to testify at an inquiry into the allegations, though he only served three months on medical grounds.
It has led to divisions and infighting within the ANC.
Mr Zuma was suspended from the party in January after announcing that he would campaign for MK.