As a legal matter the issue is not "criminal" liability, but liability in tort for slavery. Conduct does not need to be illegal for a tort claim to apply (for example, if I accidentally hit you with my car, that may not be a crime, but it could still be legal "negligence" which would entitle you to a recovery). There are many tort claims that could apply to slavery, including a false imprisonment claim, and many states have freeform "tort" claims that one could assert.
The problem is not the legal theory, but the impracticality of the claims. The claims should be against the tortfeasor, and they are all dead. One can imagine asserting a claim against the estates of the tortfeasors, but the wealth they accumulated has been so disbursed that you can never separate it from general funds.
Even if you could (or if you asserted the claim against the federal government, which itself never owned slaves), the plaintiffs would be the descendants of slaves, and you can't determine who is owed how much under that standard. Not every black person is descended from slaves. Amongst those who are, some have more white ancestry than black, and almost all have some white ancestry and some ancestry from free blacks who themselves were not slaves. Figuring out which person was entitled to what by ancestry would be impossible.
Even if you could overcome that, though, determining what the damages were to each slave in the plaintiff's ancestry is impossible. Some slaves had it worse than others. There are just too many immeasurables involved.
Besides that, even if we named an arbitrary figure, most blacks would be offended at the size of it. Imagine that we gave $100 billion. That would only be only about $2,800 per black person in America. That is not a small sum, but it's not life changing either.