I'll only answer on the 400 as I have only ridden the 650 short distances, long before my DRZs.
With knobbies, the DRZs is great in dirt, I have ridden it through hilly, dusty, muddy terrain and trails of all sorts. It seems a bit top heavy, but take off the mirror and it feels normal for a dirt bike(way better which is weird for such a small change I know). Thinking of getting MSR plastic fold down mirrors as I ride to many trails.
Highway, as long as you are not racing I find it good. My longest day was about 500km on a windy day, with a bit of butt and neck soreness. I think the neck issue will be the same with the XR, and something I do not notice on a 300+ km trip, and who wants a fairing on a dual sport. The stock gas tank is small, I fill up about every 100-120km (60-70 miles), not using reserve. In measured terms I get high 60's, low 70's in mpg using imperial gallons, figure about 10% less for US gallons. So I know I could stretch it further between fill-ups.
Only problem with my DRZ has been battery replacement, but I just had the electrical system tested and its good, so it was just poor batteries and probably my leave-it-till-spring winter storage. I hated the stock tires (bridgestone trailwings) on and off road. The knobbies (Dunlop 606's) were wiggly on road when new but have stabilized with a bit of wear. If I were riding a lot of streets I'd be thinking of Pirelli super motostyle of tires.
I did put off road protection for radiator, crank case/transmission covers, and aluminium bars/hand protectors, but otherwise its stock and I wouldn't spend money on performance parts. 33 bhp is what everyone says stock. 4strokes.com is good for cheap and expensive performance tips if you want to go that way.
I have a full sport bike, but I often take the DRZ for the fun factor and option to take gravel roads and trails on my rides. In town, no contest the DRZ is better for the tall view and suspension (crappy roads) compared to the sport bike which has no need to get out of 1st gear. The DRZ will do about 100 mph, but why? Its comfortable around 70-75 when you want to. Never been tempted to change gearing for off road because I ride a lot of roads, and first gear is short enough on trails that I wouldn't want it short.
I got my DRZ used, but paid a bit more because it was in demand at rhe time and I wanted it from a dealer, may not be as high demand now with all the KTM's, Yamaha's and Kawi's that cover dual sports. You'll probably pay the same for a decent XR-L now.
I maybe dream of a KTM690, but the DRZ is a keeper even if my sport bike was on the block.