He was elected to the Federal Council of Switzerland on 10 December 1986. He was member of the Christian Democratic People's Party of Switzerland from the canton of Appenzell Innerrhoden. He handed over office on 30 April 1999.
Koller was Chairman of the Board of the Forum of Federations from 2006 to 2010.
During his time in office he held the following departments:
- Federal Military Department (1987–1988)
- Federal Department of Justice and Police (1989)
- Federal Military Department (1989)
- Federal Department of Justice and Police (1990–1999)
He was President of the Confederation twice in 1990 and 1997.