Most Likely: Check your battery connections, this sounds like a connectivity issue.
Alternative: Could be bad seal in the tranny. When you put it in reverse a piston moves to change the gears. If the seal is old and worn fluid can get past it. Since it happens only sometimes and in the cold I would guess the seal is starting to fail and when it heats up it expands to its correct size
Other possibilities: IAC (Idle Air Control) Like Mediaslayer said.
If it is the IAC, Don't go buying a new one though. Remove it and clean the plunger area. Flush it with oil. Use engine oil or I've read where people use WD-40 with same success. What happens is the plunger gets hung up and won't open to allow air in for idle. It is possible your IAC motor is bad, but unlikely. Regardless, at 50-60 buck for replacement, it's worth a cleaning. I cleaned mine and am going over a year now with no issues. Cold weather usually triggers the problem of plunger sticking. If need be, free the plunger up with a gentle push of a small screw driver.