I personally have no problem with people wearing affliction / ufc shirts, if they just wear them and go around with their business.
who i do hate are those who wear the shirts and think it just made them Anderson Silva, and walk around trying to be tough guys who think they can fight just because they watch UFC and theoretically know how to do an arm bar or RNC.
So as long as you act like a normal person, wear the shirt and admit you are only a fan, not someone who trains in this, then people shouldnt have a problem with you.