My theory on this is that it often comes from your experiences a the moment that you feel your first sexual feelings at a young age. The first time that you feel "turned on" for want of a better expression. It could be something that you see, or something that you do, and it can happen well before puberty, before you really know anything about sex or sexuality - it is a kind of arousal that happens only in the mind, and manifests itself in your desires later in life.
When I was a child I saw some mannequins in a museum wearing nothing but gas masks, and sometimes I think it might have had an influence on me.