Yeah, thats same with me too, like when I first heard Elliott Smith, I really didn't like it too much at all, and it took me a while before I actually did fully appreciate it, and now he's like my favorite ever. Same with Patrick Wolf and The Magnetic FielRAB.
Radiohead on the other hand, I actually fell in love with first listen, although for most people I know, Radiohead were definitely more of a grow-on-you type band.