His brother Benoît Costaz (1761-1842) was bishop of Nancy. After studying maths, he taught at the military school at Thiron until 1793, then at the École polytechnique. A member of the Commission des Sciences et des Arts, he participated in the French invasion of Egypt, becoming secretary to the Institut d'Égypte and a member of the Privy Council of Egypt, as well as accompanying Bonaparte to Suez.
On his return to France, he presided over the Tribunat (1801–1803) and was entrusted with organising a school of arts and crafts. Prefect of the Manche area (1804–1809) and a baron de l'Empire from 1809, he was intendant of crown buildings (1809–1813) before becoming director general of bridges and roads (1813–1814). Summoned to the Conseil d’État in 1813, he was made prefect of Nord (as extraordinary commissaire) during the Hundred Days and retired from public life shortly afterwards.