Levi knows, if there's any justice after death then he certainly doesn't deserve heaven.
But Petra will most certainly be there, and he thinks if there's any justice after death then he'll be with her again.
The goddesses must have worked out the paradox, because there she is and he's running so hard and as fast as he can to get to her he can't breathe and his heart is pounding in his chest and his whole body is on fire—
"Levi!"
When Levi opens his eyes and sees Erwin's and Hanji's concerned faces, he knows he's in hell.