Sorry but I wouldn't accept that.
I would dump him if I was you.
You know what your values are. And you know what you want in a guy. Obviously, he doesn't have it.
:)
You can do better! Don't let him try to convince you that he was having "innocent fun". You wouldn't do that to him, and...