Yes, this is my dad's favorite riddle, so I (unfortunately) hear it a lot.
The time of day is a metaphor for the stage of life of a human.
We walk on four legs as babies, two legs until middle age, and three legs (counting a walking cane) as old folks.