Id say you would be better off wearing running/gym shorts, without a doubt....but no one should give you a hard time. In my opinion, there are a lot more cocky "climbers" that are real snobby and think they are hot s**t at the gym than there are when you actually go outdoor climbing....but if someone gives you a hard time, tell them to screw off and just keep climbing.