I think they flow quite well though. If they hadn't gone back to Ashford, then Shirley wouldn't have been Geassed, if she hadn't been Geassed, well...
If said event didn't happen, Lelouch wouldn't have a motive for his next course of actions...
If he didn't go through with his next set of actions, then we would never find out more about C.C., V.V., The Emperor, Etc.
So, it all comes together. Did they need the comedy? No, but I thought it was a nice touch, and it gave us some Milly development.
You say it doesn't flow, I say it does, I suppose it is something we will just have to agree to disagree on.