The people who tolerate milk well as adults are a minority, but not even that minority forms an ethnic group - several different ethnic groups are involved.
Dairy farming was first invented in southwest Asia between 6000 and 5000BCE. It spread somewhat slowly from there, eventually becoming established in continental Europe and south Asia, but it did not reach sub-Saharan Africa and east Asia until relatively modern times, surely the Iron Age if not the late Iron Age. It spread to Scandinavia and the British Isles a little sooner.
The people of the New World had no domesticated animals that could be milked. Llamas produce so little milk it's a wonder that their own babies survive, and the North Americans never tried to domesticate large herbivores like the bison, their own species of cattle--or even the moose or the mountain goat.
So it was the people of southwestern Asia, Europe, and the Indian subcontinent who had milk available as a plentiful and efficient source of nutrition (dairy farming uses pasture land ten times as efficiently as beef farming) and therefore needed a physiology that could digest it.
Like all mammals, humans produce the enzyme
lactase as babies, which is necessary for digesting the lactose in milk. But like all mammals, as we grow older and stop nursing, we no longer need that enzyme so its production attenuates.
It's not clear why any human would have tried drinking cow milk, unless population pressure made the eating of meat cost-ineffective and they had to find another way to survive. Whatever the reason, suddenly people who could digest milk had a tremendous survival advantage over the others. The human population was waiting for a mutation that would make this happen.
The resulting phenomenon is an instance of
neoteny, the retention of a characteristic of infancy into adulthood. Humans whose bodies continued to produce lactase for a few more years had a survival advantage over those who didn't. A community in which a large percentage of the people could drink milk, leaving more meat for their comrades, had an economic advantage over other communities.
Over the course of several thousand years, natural selection resulted in the people of those regions retaining the ability to produce lactase throughout their entire lives. As Europe became more crowded and its cities pushed farmland farther and farther away from the population centers, in an era when there was no refrigeration and the fastest way to deliver food was a wagon pulled by a domestic animal, farmland had to be used more efficiently. Otherwise farms would be so far from the city that it would be impossible to deliver food before it spoiled. Of course preserved meat lasts longer but meat preservation is an expensive process that somebody has to pay for--certainly not the peasants and slaves who comprised most of the population of the Roman Empire.
In fact people began eating more grains--which created horrible nutritional problems and drastically reduced life expectancy in the Roman Era. But they augmented their bread with the amino acids in milk protein, and this kept them alive, at least for a while. (They knew virtually nothing about vitamins and minerals, neither of which are abundant in cereal.)
The rest of the human species, those who do not digest milk well as adults, are very diverse ethnically.
Sub-Saharan Africans, East Asians, Polynesians, Native Australians, and the native people of the Western Hemisphere.
Lactose intolerance is also common (but not universal) among Jewish people, even though they originated in the dairy farming region. Perhaps the proscription in the Torah against eating meat and milk in the same meal discouraged them from eating milk at all--but this is simply a wild guess.