Passive bicycles put in such a position are not stable. Any movement will cause the bicycle to tumble. A bicycle that is standing still does not have stabilizing angular momentum in the wheels.
However, a coordinated human bicyclist can keep up a motionless bicycle at least temporarily by moving his center of mass to compensate for the instability of the bicycle.
Of course, if the bicycle is moving, the angular momentum of the wheels plus the coordination of the rider provide the stability to keep the bicycle in a dynamic balance.
