Many believe that wisdom teeth and the appendix are vestigial...that they had uses to our distant ancestors but the function was lost over time when it became unnecessary. Some dispute that with respect to the appendix, arguing that it still serves some role in the human immune response. (Of course some dispute that there is such a thing as a "vestigial" organ, since they dispute that we had non-human ancestors, or that an organ might degrade into non-functionality over time).
The hymen....no one knows. It appears to be useless, but it appears to be useless in a wide variety of mammals. Because it's a common feature with other not so closely related species, it's not "vestigial," but no one has yet figured out why we as a species still have it. I once read the speculation that it was there to make virgins think twice about having sex the first time--to make them more choosy in a first mate (since the first experience might be uncomfortable, they might as well wait until they find someone they really like and who is likely to stay with then if they get pregnant). This theory sounds plausible, but it's not entirely clear then why less picky species that do not pair bond have hymen as well (like, say, horses or certain types of whales).
I've also read that it provided an advantage in humans because it allowed our parentage obsessed ancestors to be certain they were marring virgins (and hence that any children she bears are likely to be the offspring of the husband). This theory is almost certainly wrong, since you can't reliably verify virginity from looking at the hymen in every case. (Still, if our ancient ancestors persistently
believed it were true, women with hymens might have ha d a survival advantage). Again though, it still doesn't explain why non-virgin seeking species, like horses, have them.
Here's a new theory: God designed us all, but He is a terrible engineer and threw a lot of unnecessary code into our DNA. Stupid God.