Giving them to someone to eat would mean they would be killed and destroy the purpose of not eating them/having them killed.
I will not say anything about the fish feeling pain because some people are going to say they don't and some people are going to say they do.
Whether or not they feel pain, it is wrong, because you are using it for your own entertainment. Putting a hook through it and reeling it in, and setting it free is capturing it and overpowering it. It is in the wild and should be free. It is like capturing any other animal and sticking it in a cage until you get bored with it and letting it go. It does not necessarily hurt it, but it is not fair to the animal and is cruel.
If you caught an animal in a cage and only held it there for a short time to do some sort of research that would benefit their kind, then I say it would be okay. Because it is not animal testing in which the animal is hurt in anyway or even touched, it is for a short time [assuming you know the meaning of "short" time in that situation], and it could benefit them.