Abdul Hai Khateby, the provincial governor spokesman, said fighting raged for two days before Afghan soldiers drove the militants from the Taywara District center on Friday.
Khateby said 13 Taliban militants had been killed in the clashes. There had been no casualties on the side of the Afghan forces.
The residents who had fled their homes also began returning to the district capital, Khateby added.
Taliban took control of the center of Taywara — the main town of the rural area — on Sunday as part of a broad push that saw heavy fighting across much of the country.
Since then, government forces had been fighting to push the militants back.
Provincial Police Chief Mohammad Mustafa Moseni said the Taliban militants left behind mines, which demining teams had already begun to clear.
Ghor Province is located in a remote mountainous region, where the Taliban have long had a strong presence, and more recently, Afghan security officials say, militants loyal to Daesh Takfiri group, have established themselves in the province.
Taliban militants have stepped up their attacks over the past week.
Earlier this week, around 30 Afghan soldiers were killed when the militants attacked an outpost in the southern province of Kandahar.
There has also been heavy fighting in the province of Helmand as well as the northern areas of Faryab, Baghlan, and Badakshan, among other places.
Fighting and insecurity prevails in Afghanistan despite the presence of thousands of foreign forces in the country, who have been there since a US-led invasion in 2001.
An additional 4,000 US troops are expected to be sent to Afghanistan to reinforce the US-led foreign troops in the country.