Are you using Internet Explorer, Firefox or some other browser? I could give you step by step if I knew what you were using. But in general, go into the internet options, and somewhere under History, put the number to 0.
yoiu don't neede a button to find history (pressing ctrl+h calls up the history) so deleting a button (even if you could ) wouldn't fix this.
yoou need to get itnot the internet settings and either tell it to delet history on close (fastest and easiest and most comprehensive) and maybe even change the storage volume to a smaller amount. all of this is in the general tab when you call up 'tools' then 'internet options'.