It's defiantly not fat, my boyfriend is about 57.5'/5'8 and the same. Though he gained a couple of pounds because we've had really big meals all week.
(I actually kind of like it, he's not fat but right now it's enough that I can grab onto when we cuddle though he hates it xD. Plus mine went...