The original building of the mosque was built in 1949 by the Nationalist Muslims whom mostly came with the Nationalist Army who fled from Mainland China after their defeat in the Chinese Civil War by the Communist Army. This wave of migration is considered to be the second wave of Muslim migration to Taiwan in the 1950s. The construction of the mosque was initially suggested by the Muslim public officers working for the Nationalist government.
The first mosque building was built at Wufu 4th Road in Kaohsiung City. After two years, they moved the mosque to a new bigger place at Linsen 1st Road in 1951 to accommodate larger crowds. In 1990, they moved the mosque again to its current location at the Jianjun Road. The mosque was opened on April 1992.
- "Kaohsiung Mosque, Kaohsiung, pingtung hsien 912, Taiwan - Islamic Centers, Masjids Mosques, Muslim Owned Businesses, Islamic Schools and Colleges". Archived from the original on 2013-10-23. Retrieved 2013-11-28.