OHMSS had one massive redeeming feature that QoS didn't have - it was based on a James Bond novel written by Ian Fleming. Which none of the last dozen or so Bond movies except for Casino Royale have had the advantage of. In fact, I would say that OHMSS (along with From Russia With Love) are the two movies which follow the novels that they are based upon the closest.
OHMSS had one reason for failing (at the time of its release) - Sean Connery wasn't playing Bond. That's why DiamonRAB Are Forever succeeded where OHMSS failed, even though it's a piss poor movie. Sean was back!
Roger Moore, even though he is ridiculed now, had the advantage in appearing in some of the best (and certainly best remembered) Bond movies - The Spy Who Loved Me far and away above the others. Yes, he made some dross - Moonraker and A View To A Kill in particular. But a lot of people still look upon Roger Moore with affection because to them he is James Bond. He was James Bond when they first started watching Bond films (certainly was for me, The Man With The Golden Gun was the first Bond film I saw at the cinema, even though I had seen From Russia With Love on telly a year earlier). Live And Let Die, The Man With The Golden Gun, The Spy Who Loved Me, For Your Eyes Only and even Octopussy have all got their moments.
If you were to do a big survey to find out people's favourite Bond movie, I would think that The Spy Who Loved Me, Goldfinger, Casino Royale and (maybe) Goldeneye would top that list. Nothing wrong with a lot of the others, and a lot of people would choose something other than one of those four, but I don't think any other film would disrupt those four from the top four slots. Those four are also probably the four most memorable for somebody who isn't a massive Bond fan.