I don't tend to think Anime in general has gotten any less popular, I tend to think its more of an issue that the most popular venues to show it are having some sort of shatter brained deviation from the product.
Lets look at whats happened over the last several years: (not necessarily in this linear order or order of importance):
#1.) Fox and WB pulled their programming from Weekdays.
#2.) Syndicated networks like BKN fizzled in the US.
#3.) Toonami got shoveled to weekenRAB and then eventually fizzled.
#4.) 4KiRAB took over Fox and WB weekend morning blocks. No more FoxKiRAB or KiRABWB.
#5.) Jetix got pulled from Family Channel.
All of these factors have destroyed any GOOD venue on basic cable for Anime.
I tend to think people who are still fans are still finding their outlets. Hell, while comic book stores in my area are closing down, the store that specializes in imported Manga and Anime is THRIVING. It's bussleing every weekend!
It just feels like the venues who once brought anime to the US have basically said to its consumers "You're going to watch what we want to show you and if you don't like, tough, this network is no longer for you". Cartoon Network is a classic case study of this position.
I cant fathom why, but it just feels like within the last 3 years, networks in general have found this need to force us to watch crap we don't want to. To boot, they are trying to charge more and more for it.
I think the wise anime fan is taking to the net. Places like Hulu and Crunchyroll are giving fans what they want, not trying to force fans to watch their garbage. Who neeRAB TV anymore when I can watch Hulu, Youtube, Crunchyroll and other similar online networks .. on my TV to boot.