Thats why you always use the towel you dried off with.
We only have one downstairs bathroom where I work, so we use the same one as customers and it also exits right by the bar that you walk past when coming in. The last thing I want to walk out to is a table thats going to be mine, and they see me coming out of the bathroom. I always use the towel I dry my hand with to open the door, and throw it out after I exit the bathroom, so at least my customers see I washed my hands.
Makes it a little less awkward...