From 1868 he was a student at the University of Göttingen, and following military duty in the Franco-Prussian War, he continued his education at the University of Bonn, where he had as instructors Franz Bücheler and Hermann Usener. At Bonn his fellow students included Georg Kaibel, Friedrich von Duhn, Georg Dehio and Hans Delbrück. After graduation in 1873 he toured countries of the Mediterranean extensively.
In 1881 he became an associate professor at the University of Kiel, followed by professorships at the Universities of Rostock (1883), Strasbourg (1888) and Göttingen (1889). At the latter institution he was university rector (1903–4), and was a colleague of Ulrich von Wilamowitz-Moellendorff. In 1889 he became a member of the Göttingen Academy of Sciences.
Much of Leo's earlier work concerned research of Seneca's tragedies and the writings of Venantius Fortunatus. Later his focus dealt largely with works of Roman playwright Plautus and early Roman literature in general. A few of his better known publications are the following:
- Die griechisch-römische Biographie nach ihrer literarischen Form, 1901
- Plautinische Forschungen (Plautine research), 1912
- Geschichte der römischen Literatur (History of Roman literature), 1913
In 1883 Leo married Cécile Hensel (1858–1928), daughter of the landowner and entrepreneur Sebastian Hensel, granddaughter of the composer Fanny Mendelssohn and the painter Wilhelm Hensel and descendant of the entrepreneur and philosopher Moses Mendelssohn and sister of the philosopher Paul Hensel and the mathematician Kurt Hensel. Both Leo and his wife came from families who were assimilated German Jews, having converted to Lutheranism in the early 19th century. The couple had three children: Erika Brecht (1887–1949), the writer Ulrich Leo (1890–1964) and the theologian Paul Leo (1893–1958).