Does adding horsepower make a huge difference on the car's performance?

[Simply the best.]

Let's say you have a Camry with 180 horse power (just an example) and you trade for a Mustang with 300 hp will the additional hp make a huge difference when it comes to acceleration or are there other factors to take into consideration?