I know of two explanations. First, you are using angles to retrieve your fish. You use leverage, which is all about angles, to overcome the fish and take it from the water.
Second, the word comes from a Latin or Middle English root word that relates to "bent" or "bend," as in the bend in a hook, or the bending of a fishing rod. So, maybe when the word was forming, it really meant something more like "bending" and "benders" rather than "angling" and "anglers."
Personally, I like the second one best.
