The 35-year-old took off from Fujian province on Wednesday morning, aiming to land on one of the Tokyo-controlled islands, known as the Senkakus in Japanese and Diaoyus in Chinese, the coastguard official said.
But he sent a rescue request when he got into trouble near the islands in the East China Sea hours later, the official said.
A Japanese rescue helicopter found the balloon 22 kilometres (14 miles) south of the islands and rescued the man, who was unhurt, the official said. He was handed over to a Chinese patrol ship outside territorial waters.
Chinese and Taiwanese activists have tried to land on the islands by ship several times but been blocked by the Japanese coastguard.
Regional tensions rose in late November when China abruptly declared a new Air Defence Identification Zone over the East China Sea, including the disputed islands.
Chinese state-owned ships and aircraft have approached the Senkakus on and off to demonstrate Beijing's territorial claims, especially since Japan nationalised some of the islands in September 2012.
Tensions between the countries have risen further after Japanese Prime Minister Shinzo Abe visited Tokyo's controversial Yasukuni war shrine last month.
The shrine honours several high-level officials executed for war crimes after World War II, and serves as a reminder of Japan's 20th century aggression against China and other Asian nations.