phew, took awhile, but I figured out that that the first man has 5 fish originally, and the second man has 7 fish.
let's call the first man bob, and the second man joe, for the ease of explanation.
if bob gives a fish to joe, then bob will have 4 and joe will have 8 (twice as many as bob).
however, if joe gives a fish to bob, then they will both have 6!
I hope that clarified things for you!