Some even probably thought it was him.
But renowned actor Sir Ian McKellen has revealed he turned down the role of Albus Dumbledore in the Harry Potter film series because the original actor, Sir Richard Harris, did not approve of him.
Speaking on BBC HARDtalk, the British actor who famously played the white wizard Gandalf in The Lord of the Rings franchise and villain Magneto in the X-Men films, said the late Sir Richard was not critical of his acting skills, but believed he lacked passion for the art.
"Technically brilliant, but passionless," HARDtalk host Stephen Sackur quoted Sir Richard as saying.
It was that stinging critique, Sir Ian said, that ultimately made him turn down the opportunity to play the headmaster of the Hogwarts School of Witchcraft and Wizardry when Sir Richard died in 2002.
"When they called me up and said would I be interested in being in the Harry Potter films, they didn't say what part," Sir Ian said.
"I worked out what they were thinking and I couldn't.
"I couldn't take over the part from an actor who I'd known didn't approve of me."
Sir Richard had played the role in the first and second movies, but there were still six films yet to come.
Irish actor Sir Michael Gambon, known also for his roles in Sleepy Hollow and Gosford Park, eventually took on the character.
But his likeness with Sir Ian did confuse some, including Sir Ian himself.
"Sometimes when I see the posters of Mike Gambon, the actor who gloriously plays Dumbledore, I think sometimes it's me," he joked.