He can use that against you. Is this a custody battle? You can try to use the emotionally/verbally abusive against him but that wouldn't be substantial in the case because it will be viewed as 'if it was so bad, why were the police never contacted?' And just put down to normal arguing, possibly...