I think it's not plain wrong or right.
Definitely it is not strictly right - such as, the "gills" on the embryos are not functional gills, but only the starting structures from which functional gills on fishes develop. Ontogeny does not recapitulates phylogeny, but often looks somewhat as if it does because lots of the differences between species were determined at a reasonably late stage of development, because of both viability of development and maintenance of adaptation.
But, for instance, butterflies are not closer ancestors of some worm-like, catterpilar-like invertebrate pre-arthropod than any other insect, they just had this developmental stage stretched, which turned out to be both viable and adaptive.
Likewise, out unicellular stage probably has not much to do with ancestry of primitive unicellular organisms, but with the fact that the new organism has to "start" somewhere, and we are not able to start from, say, the detachment of an arm or a piece of flash, like some organisms do (planarias, among others). These organisms are somewhat intermediate between unicellular and us, but we do not reproduce like them. The origin of twins has some resemblance with that, however, but yet much more akin to the physical disruption of a colony of unicellular organisms in two viable colonies.
The development of arteries, limb and tail buds that start to develop only to degenerate later, these are quite impressive evolutionary remains of the past evolutionary history preserved on the development, however.
Haeckel has some detractors and defenders, I'm just not enough informed to take a stand. I think that it can be from plain falsification from some legitimate mistake, somewhat like orthogenetic evolution ideas on the early 20th century, from small samples of fossils, before the evidence for a branching evolution. I have no idea on how hard or easy is or was to draw the embryos and etc.
It's necessary to keep in mind, however, that "ontogeny recapitulates phylogeny" is not strictly correct, and this is not a sort of "requirement" for scientific mainstream's (i.e. "darwinian") evolution. Darwin was pointing to embryological evidences already in his 1842's sketch (which already has the essence of common ancestry, natural selection, allopatric speciation and so forth), different from those of Haeckel, without the assumption or implication that ontogeny recapitulates phylogeny.
_____________________
About the topic itself:
That's a somewhat tricky issue. You can have a cell or a group of undifferentiated or even somewhat differentiated cells, with all the 46 chromossomes unique to its own, but which wouldn't ever develop into a human, even though it's all human tissue and genes, even a karyotypic/genetic gender.
Sometimes, the egg starts developing by itself, without fecondation (it will make another copy of its 23 chromosomes, ending with a karyotypically normal human zigote), but the resulting organism is always inviable, in the case of humans and other mammals, as far as scientists know.
It will have many specialized tissues with some degree of organization, even thumbnails and nervous tissue, but not organized enough to continue living. I recall reading that the level organization may increase if the development of one of these occurs simultaneously with the one of a normal zygote, at the point to be possible of be mistakenly identified as a genuine twin in the ultrasound (presumably, at earlier stages anyway). It still won't develop into anything viable, however. The nearest thing would be a chimaera, that is, if it fuses with a normal developing embryo and develop as one, kind of the contrary of what happens with identical twins. There is at least one case of a partial parthenogenic (developed from an unfertilized egg) human chimaera in the medical literature, if I recall.
They've managed to produce a rat or a mice from this sort of reproduction, anyway, without the aid of a normal embryo to create a chimera.
Unique human life, in a certain way, is already present in the gametes, since the gametes are living cells, genetically unique, and humans (in the case of human gametes).
