My backpack is pretty heavy and the weight of the backpack while riding may make me fall on my back so I don't think I could ride my bike with a backpack on my back.
I don't see anything awkward about it. If my backpack is empty or has very little in it, I roll it up and strap it on the rack. I don't wear it on my back if I don't need to, as it makes my back sweat a lot.