Well I already explained sort of, although it wasn't very detailed because I am lazy. But looking at the link I posted and the second one that tariq_h posted it's pretty easy to figure out.
I did not have a .shsh from 3.1.2, I was on iOS 4 and successfully downgraded. You need to edit the hosts file, this directs itunes to cydia's server instead of apple's where they see the ipsw you are trying to load isn't signed and they reject it causing the error in question. Changing the hosts file fixes this.
Now I'm working on loading a custom iOS 4 according to this: Steps To Jailbreak iPod Touch 3G iOS 4 with Sn0wBreeze 1.7 | TheTechJournal.com