There is the Left, the Right, and although they don't call themselves this, but they would be "the Middle" or the Independents, many leaning Left or Right but not going so far as to associate or polarize one way or the other.
Then you have the FAR Left, the FAR Right, and those would be your Extremes. On the FAR Right, you have Neocons, although they don't call themselves that. And on the FAR Left, you have what News Commentator Bill O'Reilly would call the LOONS.
Because we seem to have about 5 distinct segments (Far Left, Left Wing, Independents, Right Wing, Far Right), I don't think you can say about HALF of this population is totally without credibility such that one cannot take them seriously. As the experienced politicians, political analysts and historians will tell you, America is run from the middle, at least over time and in terms of who stays in office the longest. It is a process of give and take from both sides, balancing out somewhere in the middle.
So if you are going to cast out any segment from the realm of credibility, I think you should start with the extremes first, i.e. the Far Right AND the Far Left. Those comprise the smallest segments but also the most objectionable.
But I would not go so far as to throw out ALL Right-leaning or Right-Oriented individuals or groups. And particularly considering the latest polls of mid to late Summer 2009 showing Republicans having a lead, considering all the negative fallout from Town Hall meetings, Health Care Reform, Bailouts, Cap & Trade, the Stimulus Bill, and Trillion-Dollar Deficits with Unemployment still rising at 9.7%.