No, I would guess that you are pretty much spot on.
It is common knowledge that Anthony Hopkins has claimed he will never play Hannibal Lecter again, as he felt he was too old to play him when Red Dragon was released and refused to even do a narrative part for Hannibal Rising.
Cate Blanchett was the favourite name bandied around by Universal when Jodie Foster first said she wasn't going to reprise the role - followed closely by Gillian Anderson, Helen Hunt, Gwyneth Paltrow, Cameron Diaz, Ashley Judd, Charlize Theron, Angelina Jolie (yes, really), Hilary Swank, Calista Flockhart and even Sarah Michelle Gellar.
It was actually Ridley Scott who suggested Julianne Moore's name to Dino de Laurentiis.
Jodie Foster turned down Hannibal for several reasons - one, because the star of Hannibal is Hannibal Lecter, not Clarice Starling, whereas in Silence Of The Lambs she's definitely the main character and Lecter is just a side-story. And two, because of the (to quote) "outlandish image of unholy lovers Lecter and Starling feasting on human brains" and because Starling had been given "negative attributes" which Foster refused to perform - this was of course based on the conclusion of the novel, not on the final screenplay. She was also supposedly offered a smaller pay packet than Anthony Hopkins was, and as she had between the two movies made Sommersby, Maverick, Nell and Contact she felt that she deserved to be on a similar salary to Hopkins. She (like Jonathan Demme) felt the script was too violent also - particularly the Paul Krendler brain-eating bit at the end.