Graun was born in Wahrenbrück in Brandenburg. He sang in the chorus of the Dresden opera before moving to Braunschweig, singing there and writing six operas for the company. In 1735 Graun moved to Rheinsberg, after he had written an opera for the crown prince, when marrying Elisabeth Christine in Salzdahlum. He was kapellmeister to Frederick II of Prussia (Frederick the Great) from his ascension to the throne in 1740 until Graun's death nineteen years later in Berlin.

Graun wrote a number of operas. His opera Cesare e Cleopatra inaugurated the opening of the Berlin opera house in 1742. Others, such as Montezuma (1755), with a libretto by King Frederick. None of his pieces are often performed today, though his passion Der Tod Jesu (The Death of Jesus, 1755) was frequently given in Germany for many years after his death. His other works include concertos and trio sonatas.

Carl Heinrich Graun was the brother of Johann Gottlieb Graun, also a composer.