Given that it's the producer, particularly in the US, who controls things, leaving aside the input from sponsors, networks etc., that's pretty much the case.
I yield to no one in my admiration of good US tv but even the top notch stuff isn't really dependent on who directed a particular episode...