sometimes there is something that shames a man or a woman that keeps them from sex, personal depression, that is hidden behind work, or conflict.
He may harbor fantasies that he feels shame in discussion, ask him softly, quietly, allow him to explore his mind with you, without fear.
I have known several this way, and it turned out to be fantasy that made them withhold