I don't see why there would be an issue with it... colleges only have problems with it because of the potential power the teacher has over their students and their impressionable minds.
And as long as there isn't any sort of disturbing age difference I don't see why it would be anybody's business but the people involved.