If it's a tube type TV, the magnet will slowly fade the colors. That's why all TV speakers are magnetically shielded. As for LCD & Plasma TVS, the harm isn't as bad but still does more harm than good. It's always best to keep magnets away from TVS @ all times.
Hope I answered your question