Yes he is testing you to see if he can get away with it. If you do let him get away with it and you become fearful he will carry on dominating you. He will sense it too and carry on doing it.
First you must be brave and positive and approach him in a confident forward matter of thinking. Walk up to him in the pasture, but for your safety's sake ALWAYS carry a whip near him and do not hesitate to use it if he becomes aggressive again.
If he does lunge at you, you stand tall and with your eyes wide, shouting a loud NO, crack the whip loudly (but do not hit him with it) and make him be the one to move away first.
In horse language, the one who drives the other is in control and you must adopt this matter of thinking aswell, otherwise you will not have the respect from him that you deserve. Good Luck.
You could try researching on horse language to help you.