I started motorcycling about 3 months ago.
First and foremost, take an MSF course. I speak from experience; I took my bike out before MSF and I low sided off on my very first turn (grabbed brake in a panic). I had a 23 year old bike in perfect condition until I got cocky and thought riding a motorcycle would be "like riding a bike." If I hadn't made such a poor error in judgment, my stator cover wouldn't be scuffed and left rear turn signal not being held on by tape. I got REALLY lucky and the bike slid 10-12 feet to a stop, with me falling relatively gently on the pavement. Scraped knee and damaged ego.
As for my ride, I got REALLY lucky and got a hold of a vintage 1985 Honda VF500F Interceptor. Only 14k miles on the odometer and 900 dollars for it. I got it in PRISTINE condition, absolutely immaculate for it's age (Until, as mentioned, 'I' happened to it).
I have to say I have no regrets about my bike choice. Some things I like about it you might want to consider are the seat height (can you reach the ground comfortably) and ergonomics (is the riding position tolerable?), is it something you like cosmetically *AND* mechanically? If it lacks respectable power, doesn't sound nice, looks like a booger with two wheels, etc. well you just won't be happy in the long run.
The seat height on my VF500 allows me to easily reach both feet flat on the ground (with riding boots on) and I'm about 5'8. I love the riding position too, I sit nice and upright allowing comfortable operation for as long as I choose to ride (Longest I've done is 6 hours with a 5 min break at the 3 hr mark, afterwards felt like I had just spent the day in a recliner).
Since it's a V-Four the motor runs silky smooth, no intense vibrations or jitters like you might get with other bikes. The high RPM motor provides PREDICTABLE, smooth power; if you hit a pot hole and blip the throttle, you aren't going to launch into a power wheelie. In fact, it doesn't have enough power to deliberately wheelie without the clutch (only 66 peak HP that comes late in the band and it's a hefty 450 pounds for such small displacement compared to today's bikes), very newbie friendly! You just have to remember to avoid lugging (shifting too early) the motor for the sake of mechanical stress and safety (you may need the power, my bike likes cruising at 7.5 grand, shifting at 10).
Gets about 40 Mpg. It's unique because it's red/white/blue with a quarter fairing (the engine is exposed, VERY uncommon on newer sport type bikes). It's eye catching, a piece of history and most importantly, *I* love it. If you aren't head over heels in love with a bike, you're going to get bored of it quick and want something better (depending on your finances, that may or may not be OK).
Some people buy a junker for the sake of destroying in the name of learning. Others, like me choose a respectable bike that can take the abuses of a learning curve and still be fun afterwards!
Granted, vintage bikes can offer huge savings in the initial costs, in the long run parts availability will probably put an end to its service life (A day I fear myself). That and many vintage bikes need work from the start (mine needed new carburetor boots, carb cleaning, tires etc.) I myself am mechanically inclined and have previous automotive/4 stroke engine experience; your ability/willingness to maintain/repair the bike yourself is going to affect whether or not a vintage bike is a good option for you. Fuel injected bikes are going to have less maintenance than carbureted bikes, but when something goes wrong the fix isn't usually cheap or novice user serviceable.
I'm probably missing many factors that should come into play before purchasing a bike (I bought mine rather impulsively in a sort of "love at first sight" sense). Though, just make sure the bike is something YOU like that meets YOUR needs and YOU are prepared for (in terms of riding skill, mechanical skill, power, finances, level of responsibility, etc.)
Accept the fact that you WILL fall off at some point while moving (like I did), you WILL drop the bike (first time I swung my leg over the bike when I got it home it fell on a sprinkler spigot and punched a hole in the right side panel). Be willing to have bad things like that happen, you don't WANT them to happen, but accept them as highly probable.
Oh, and enjoy the ride; make sure to wave to other motorcyclists you see on the road (UNLESS operating conditions make it unsafe to do so, such as in a turn or navigating dense traffic). Don't be like me and be a dweeb giving a full palm wave either, learn to extend your left arm in a super suave 'peace' sign.
Keep the rubber side down!