The Southern Transitional Council (STC) accused the government of failing to perform its duties and of "conspiring" against the southern cause, and said self-governance had begun at midnight.
The government condemned the move and said the separatists -- who have long agitated for independence in the south -- would be responsible for the "catastrophic and dangerous" outcome.
The breakdown between the one-time allies comes as a Saudi-led coalition, which backs the internationally recognised government in a battle against the Iran-backed Huthi rebels, has extended a unilateral ceasefire aimed at fending off the coronavirus pandemic -- a move rejected by the Huthis.
Yemen's separatists signed a power-sharing deal in Riyadh last November that quelled a battle -- dubbed a "civil war within a civil war" -- for the south that had in August seen them seize control of the second city of Aden.
The Riyadh pact quickly became defunct, failing to meet deadlines for key measures including the formation of a new cabinet with equal representation for southerners, and the reorganisation of military forces.
The STC announced in its statement that it was declaring "self-governance in the south starting midnight on Saturday April 25th, 2020.
"A self governing committee will start its work according to a list of tasks assigned by the council's presidency," it said.
Aden residents reported heavy deployments of STC forces in the city and a separatist source told AFP they had set up checkpoints "at all government facilities, including the central bank and port of Aden".
Military vehicles drove through the city with STC flags flying aloft.
The political landscape in the south is complex, and despite the STC's declaration some southern cities said they did not recognise the call to self-rule and would remain aligned with the central government.
Yemen's Foreign Minister Mohammad al-Hadhrami said the STC move was "a continuation of the armed rebellion last August and a declaration of rejection" of the Riyadh agreement.
Tens of thousands of civilians have been killed over the past five years in the war between the government and the Huthi rebels.
Earlier this month, Yemen reported its first case of coronavirus in Hadramawt, a southern government-controlled province, raising fears of an outbreak.
Compounding the country's troubles, at least 21 people were killed in flash flooding this month, with Aden's streets submerged and homes destroyed.
The UAE, like the STC, has a zero tolerance policy towards the Muslim Brotherhood and Yemen's Brotherhood-influenced Al-Islah party, which has representatives in the internationally recognised government.
Last August, deadly clashes broke out between the government and STC forces who seized control of Aden, ousting unionist forces who had set up base there when President Abedrabbo Mansour Hadi fled the Huthi-held capital Sanaa in February 2015.
The tussle for control of the south exposed divisions between the coalition partners -- Saudi Arabia, which backs the government, and the United Arab Emirates, a backer and funder of the STC.
The Riyadh agreement had been welcomed as preventing the complete break-up of Yemen, and hailed as a possible stepping stone towards ending the wider conflict in Yemen.
But the cracks soon emerged, with complaints over food shortages in the south, a sharp depreciation of the currency, and a lack of funds to pay public sector employees.
The STC's statement Sunday said there had been a marked deterioration of public services, which "was clearly reflected in the latest torrential rains that caused the people in Aden deep suffering".
The government was using its powers as "a weapon to bring the southerners to kneel," it said.
While the government and the STC are technically allies in the long war against the Huthis, the secessionists believe the south should be an independent state -- as it was before unification in 1990.