In short, NO. Removing things from your hard drive will only speed up your hard drive if you're using an outdated file system. If you have Windows XP or better installed on your system then removing these files will have no significant impact. (Load times of specfic files maybe increased by a maximum of 1-2 seconds, but that's really pushing it)
Defragmenting your hard drive WILL improve performance. You can do this with the tools built into Windows by opening My Computer and right-clicking on the drive you wish to defragment. In the "Tools" menu you will find the "Defragment" option. I suggest you go there now and analyze your hard drive, checking specifically the percentage of fragmentation.
It is better, however, to use a third party utility such as IOBit's Smart Defrag (that's what I use) to do the work of defragmenting. Use it to defragment your hard drive when you go to bed tonight. In the morning restart the computer and you should see SIGNIFICANT speed improvements. In the future, Smart Defrag will run in the background, keeping your hard drive from getting seriously fragmented again to prevent this in the future. The software is free for personal use. You can easily disable the software when running games but I haven't noticed it affect performance at all (it automatically stops working when your computer begins working hard.)
I should explain that the things stored on your computer are not stored in your computer's memory, they are stored on your hard drive. Memory, more accurately referred to as Random-Access Memory (RAM), is unaffected by the volume of information stored on your hard drive.