Baskerville was the son of Sir Thomas Baskerville, a knight and commander of the English army in France, by Mary, daughter of Sir Thomas Throgmorton. He was born at Saint-Valery, in Picardy in northern France, on 5 April 1597. He himself states: ‘I was christened by one Mr. Man, the preacher, and I had all the captains, about thirty-two, to be my godfathers, it being the custome so of the wars, when the generall hath a son (they say); but two only stood at the font or great bason: one was Sir Arthur Savage, the other I cannot remember his name.’ His father died when he was only nine weeks old. He was instructed under the care of Henry Peacham, author of The Compleat Gentleman, and afterwards became a student of Brasenose College, Oxford.
He travelled a good deal on the continent, and spent the latter part of his life on his estate at Sunningwell, Berkshire. Anthony à Wood, who visited him there in February 1658–9, found him to be a melancholy and retired man, and was told that he gave the third or fourth part of his estate to the poor. Baskerville was so great a cherisher of wandering beggars that he built a large, barn-like place to receive them, and hung up a little bell at his back door for them to ring when they wanted anything. Indeed, he had been several times indicted at Abingdon sessions for harbouring beggars.
Hannibal Baskerville had sixteen sons and two daughters by his wife, Mary, daughter of Captain Nicholas Baskerville, second brother of Sir Thomas Baskerville. He was buried at Sunningwell on 18 March 1668.
In the Bodleian Library, among Dr. Rawlinson's manuscripts is ‘A Transcript of some writings of Hanniball Baskervile esq.; as they were found scattered here & there in his manuscripts and books of account, and first a remembrance of some monuments and reliques in the church of St. Denniss and thereabouts in France by Hanniball Baskervyle who went into that country with an English ambassador in the reigne of King James.’ This manuscript contains several curious particulars relating to Oxford and the persons educated there.