Not at all, I used to do it too when I had a car. Or if I played a CD on the stereo, hear an entire album in a store I happen to spend a long time in, etc. You listened to it, fair game.
I'm not gonna make some playlist of all the random songs I happen to hear throughout the day, but if it's an album or something, hell yes it's getting scrobbled.