Considering that I've yet to hear of any anime cons not posting significant attendance gains, and that new ones keep springing up and doing better than expected business, I don't think anime or it's fandom is necessarily losing steam.
What has happened may be the following:
1. The casual audience moved on - the group of people who never really hit a con but watched it when it was accessible on TV moved on to other things, but the accessibility fell, further ensuring they'd move on to other things. Corabine this with the fact that there wasn't much in the way of genuinely adult titles at all in the US to hold that audience even if they were on the bubble with keeping interested, and they were gonna disappear, or atleast end up at best just moving to crunchyroll and hulu. These guys did buy some shows too, but they weren't probably nearly as damaging as the next group.
2. A segment of the hardcore audience burned out and became choosy - your first mecha show or harem or inverse harem maybe be fresh and compelling, but your 20th is boring unless the writing is fantastic. These guys may still watch a lot, but they'll do so through at best via crunchyroll and hulu. They will also still hit their local cons, and may even travel if one of their favorite authors or directors is there. However, if they are buying titles, they are often waiting for boxsets because if nothing else their shelves are full, assuming the credit crunch hasn't kicked 'em in the teeth. Again, some of these guys might eat up some seinen and josei titles, and they'd love the experimental stuff from the fringes of Japanese animation, but that stuff is a rarity in this market. A piece of these people might have also added Japanese literature into their mix too, further altering their spending and the amount time they put into anime and mange. When this group, which might have had multiple series going at any given time, started getting choosy, it's murder. They also were the first to penalize any company that screwed up or overlocalized. Thus, while they are still active otaku, they aren't just buying everything on the shelves, even if they have the money to do so.
3. The new generation is the napster generation - this is was already the case when I got into it when I was 17, as at my first con I met preteens who were active in various means of pirating all sorts of anime (which at the time often meant wandering around IRC - vastly more difficult than today's means of free viewing, much of which is legal,) it's only been magnified since then. These folks definitely hit cons, often doing multiple expensive cosplays. However, they are the manga cows at your local Borders and they are the reason Comcast wants to do bandwidth caps and traffic shaping. They will buy merch, but merch money often goes right back to Japan, and not always back to the studio per se (as the rights to merch is sometimes a one time fee, not a royalty basis,) so they often times contribute nothing back into the pie, especially in the domestic localization market. If they own a show or manga, it's because they are either terribly enamored of it (something that dissipates as they get older and more discriminating and fall into group 2,) or they got a real deal on it.
As such, con attendance can keep clirabing, while the industry subsides/changes. I don't doubt that anime and manga could become completely digital markets with everything being streamables/downloadables, and the fact is, it'd allow for the greater variety needed to hold people as they grow up and aren't in high school anymore. Whether that happens with any of the old line publishers and localizers is a huge question. The second an ebook reader with manga sized dimensions hits, Tokyopop and DelRey better be ready with titles though, otherwise a new company will crunchyroll-them and make them play catch up.