Supposedly Sean Connery turned down a role in Lord of the Rings but it's unclear whether it was Gandalf, Theoden or another role, perhaps Denethor or something. Personally I think the LOTR cast was perfect and wouldn't have benefitted from Sean because everyone just thinks of him as Sean Connery!
Several top actors were asked to play Albus Dumbledore in the Harry Potter series. I don't think neither Richard Harris or Michael Gambon were right for the role, though Gambon is better because Harris felt so frail as the greatest wizard who ever lived. I believe they asked Peter O'Toole to step in initially when Harris died but he turned it down. Of course, the perfect actor for that role is Ian Mckellen, but he was hardly going to do it, was he, after playing Gandalf.