Like his contemporary and fellow music-hall artist Florrie Forde, Whelan was born in Melbourne, Australia, in 1875. Early in his career, Whelan moved to Western Australia, where he found fame as a singer and dancer, entertaining the miners working the gold fields. At the turn of the 20th century, he emigrated to Britain, making his debut in a novelty dance act at the Empire, Leicester Square. He rapidly honed his act, and settled on a style which would vary little over his career, although his ability to update the content of his act ensured his career was both long and successful, lasting well into his eighties. Whelan was acknowledged as one of the first entertainers to have a signature tune, appearing on-stage (and exiting at the end of his act) whistling Robert Vollstedt's waltz from Die Lustige Brüder (The Jolly Brothers). Immaculately dressed in bow-tie and tails, he sang, danced and played the piano. He was an excellent mimic, and adapted easily to changing vocal styles.
His recording career spanned the first half of the 20th century, from The Whistling Bowery Boy in 1905 to his final recordings made in 1960. He also had minor roles in a number of British films of the 1930s and 1940s. His son was the pianist Gordon Whelan.
Whelan was a member of the exclusive entertainers' fraternity, the Grand Order of Water Rats and served as "King Rat" for one year in 1948. He was the subject of This Is Your Life in 1957 when he was surprised by Eamonn Andrews at the BBC Television Theatre.