Afro-Americans comprise something like one-tenth of our population,
12.4%, to be exact :]
so it's statistically unlikely to see a large group of people that includes no other ethnicity.
Only if you add the assumption that black people are uniformly distributed (in geographic, class, educational and other socially-relevant terms). But they pointedly are not - the sort of color-blind society that would produce such a distribution likely would not even sustain them as a separate race for long in the first place.
Instead, the lingering effects (and continuance, if attenuated) of centuries of overt, powerful discrimination have put us in a situation where the black population is highly concentrated into certain locations, classes, social niches, etc. There aren't many places in the US with a 12.4% black population. Rather, there are a few places with a very high percentage of black people (supermajorities) amongst a sea of places with very very few black people. Most major cities (not just in the South) feature "black" neighborhoods outside of which black people are not much seen, and in which nobody else is much seen. Truly integrated neighborhoods are a rarity, even in "enlightened" places like coastal CA.
I would be extremely curious if I saw one,
Only if you weren't in one of the places where black people are clustered (which, indeed, would include hoity-toity Harvard-area clubs and the like). And even then, the curiosity would not be why they're hanging out in a group of black people, but why that group has ventured out of their de-facto segregated area.
Harvard and Yale have been racially desegregated for decades,
But the social class of rich, connected achievers that attend such schools has not been desegregated. Not by a long shot.
Why does a large group of their alumni include only Afro-Americans? It does't make sense. Are Afro-Americans segregating themselves, forming societies to which only they can belong? Since it would be illegal for us to do that, I assume it would be just as illegal for them.
Not societies to which only they can belong, but societies to which only they are interested in belonging. I.e., societies explicitly for advocating and advancing the interests of black students and alumni. There is no rule barring anyone from joining such (and I'm sure there are a few non-black people involved), just nobody other than black people has much interest in joining. Such is commonplace at every university in the USA with any minority populations to speak of - my own alma mater featured a dizzying area of "Asian interest organizations," as well as ones for other ethnic groups, and also race-based frats. None of them have any illegal rules barring any ethnicity from joining, and none of them need them since nobody else is interested to begin with.
And such is openly accepted by society at large, exactly because the history and present of racism justifies minority-interest advocacy groups. The truly perverse cases are ones like La Raza, into which all manner of ugly chauvinist ideology (google "Califaztlan" or "reconquista") gets injected under the blanket of the various legitimate minority-interest activities they pursue.
