I think it's overrated. I'll admit gangster films aren't really my cup of tea anyway but I found The Godfather to be overly long, often confusing, and with no emotional point of entry.
I'm not a massive fan of The Godfather Part II either but it's better in the sense that you get a more emotionally involving character arc; the end of the original just left me thinking "Okay, so what happened and why should I care?"
With films as highly rated as The Godfather, I always want to watch them just to see what the fuss is about. If you're like me, give it a shot for that reason.