No... A love of something does not make you weird at all.
I hear that too from some of my friends. They think it is weird & they don't understand why I watch it. I tell them all the time that anime is my life, but they don't understand.
Everybody has some form of obsession. It could be wanting to always be the best, love of a celebrity, etc. But when people hear that 'somebody loves anime' then obsession seems to become a bad thing, but that isn't true at all.
Just point out to your friends that they all love something & are obsessed with it [no exceptions] and they shouldn't be so quick to judge others.