Not via any VOIP hardware, but if he has a PC and admin level access to the router, he can potentially look through any cache saved in the router - which may giveaway your browsing history.
If he dosent have admin level access to the router or your PC - you should be fine.
