I would get a bigger flash drive. Flash is Solid-State, meaning no moving parts. As long as you're not almost out of space, the access times for the flash drive will almost always be faster than an external drive. The problem you're seeing is just that...once they get low on space, access times decrease drastically.
Plus, an external drive is going to wear out a lot faster, and is more susceptible to damage if you plan on transporting it at all.
Factor that with the price of a 4gb flash drive versus a 300gb external...yeah, I'd def. go with the flash drive.