You don't really need to take your fish out to clean your tank, but if you feel you absolutely must, then it shouldn't bother them too much.
First, drain most of the water using a suction tube. That way they can't really evade the net. Second, remove any large rocks or other things they could...