Yeah I'm more talking about health inputs like nutrition, environment and health practices.
I don't really know much about human ancestors, didn't take any anthro classes. I spose they could have been larger, but they wouldn't have been humans, no? I don't know.
There study that I read that looked at stature (height) suggested that aside from people from Asia, all other continents had the same genetic potential for stature given the same inputs. Of course, this was an economics paper, but it was still interesting none the less.
Edit: this was also based on a large sample sizes, each group had out layers obviously, but over a large population group given the same inputs the averages would be the same.