Well, it's doable. You could try using slicks to make it easier, and probably go single speed (if your town doesn't have a lot's of ups and downs.
A road bike will make you go faster, but depending on the use you want to do, it could be a better or worse choice. If you like to do some stunts (however small), probably just changing tires will make your mtb a good or better choice, if you want to go fast, a road will be a better option.