He inherited Apley Park in 1795 on the death of his father and in 1811 extensively rebuilt Apley Hall. The following year he became a partner in the London bank of Chatteris, Whitmore & Co. He was a J.P. and Deputy Lieutenant for Shropshire and was appointed High Sheriff of Shropshire for 1805–06.
Whitmore died in 1846 at the age of 63. He had married Catherine Thomasson, daughter of Thomas Thomasson of York and had 3 sons and 3 daughters. Their son Thomas Charlton Whitmore was also MP for Bridgnorth.