Eiermann was born in Neuendorf bei Potsdam (now part of Babelsberg, Potsdam), and studied at the Technical University of Berlin. He worked for the Karstadt building department for a time, and before World War II had an office with fellow architect Fritz Jaenecke. He joined the faculty of the university in Karlsruhe in 1947, working there on developing steel frame construction methods. He died in Baden-Baden, aged 65.
A functionalist, his major works include: the textile mill at Blumberg (1951); the West German pavilion at the Brussels World's Fair (with Sep Ruf, 1958); the West German embassy in Washington, D.C. (1958–1964); the highrise Langer Eugen for the German Parliament in Bonn (1965–1969); the IBM-Germany Headquarters in Stuttgart (1967–1972); and, the Olivetti building in Frankfurt (1968–1972). By far his most famous work is the new church on the site of the Kaiser Wilhelm Memorial Church in Berlin (1959–1963).