You can't make sure he is with you every time you have sex, sorry. It's completely normal and natural for people in relationships to fantasize about sex with others. These fantasies may or may not also include their real-life partner.
He's just being human dear, but what you can do is talk to him about how you feel, how sex has changed with you two and why...and what you both need to do to get that old spark back. Apparently something is missing or gotten a bit "routine" or he wouldn't be looking around. Bonus is (as far as you know) he's not playing with anyone else physically.
Don't let this ruin your relationship or your sex life, but DO communicate about it. Bottom line, you have no control over what goes on in his head and vice versa...there are no thought police and he may have been fantasizing throughout your entire relationship. He's with you though, so don't dwell on something you cannot control and work (with him) on making your relationship as fantastic as you can.