The house was built around 1480. Henry VIII is said to have stayed at Bolebroke when he went hunting for wild boar and deer in nearby Ashdown Forest. It lies only 5 miles (8.0 km) from Hever Castle and was the base from which King Henry conducted his courtship of Anne Boleyn.
The house was used as a location in the 2008 film The Other Boleyn Girl.
The Bolebroke Castle Miniature Railway was located within the castle grounds.