So what exactly is The Sift?
The name "sift" almost implies filtering of some sort.
We already know that this new dimension is likely directly correlated with souls/soul magic.
It's also interesting that several parts of the nether also have a correlation with soul energy. (Wither, Soulsand Valleys, Soulspeed). So this "sift" dimension likely has some sort of connection with the nether.
I feel like The Sift is a sort of a first stop for souls after death. They experience some form of judgement in The Sift (they get "sifted" through). Then they get sent to different fates. Suffering in the nether, being reincarnated, or maybe even kept in The Sift itself would all be examples of different fates for souls after sifting.
Tldr: The Sift is a big hub for souls before their final destination.