Hmm toughy... does he know you cheated on him? If anything he could have cheated on you, and is to afraid to tell you he has them to because he doesn't want you to know he cheated and gave them to you. Lol i think to much into things though...
You can get it even when wearing a condom, but the chances are very very low and he would have had to be in the middle of a break out. You can also get them from mouth to genital contact, if the person has a cold sore at the time.
I guess you will really never know, but if you are having doubts on his honesty about it, you may possibly want to find someone you can trust.