At first Castiel thought that being in Purgatory might be a kind of penance, but it hasn't felt that ways so far.
The place weighs on him, a constant reminder of his guilt. After all, these souls were there with him when he went to Heaven, claimed the place of his father, and slaughtered so many of his kin. These souls bore witness to his wrath on earth, as he took the lives of so many humans, so confident that he was doing the right thing. He used these souls, burned them up. And now, in their own realm, while he can't truly kill them, Castiel is still a force of destruction.
He thinks that he may be able to relieve his guilt with pain. If he were to surrender, lie down, let these creatures tear him apart until the ends of time. It would be as easy as it would be just.
But he can't leave Dean.
Not only because Castiel swore to Dean that they'd find a way out. And not only because he owes Dean for what he did to him, or because he owes Dean for the renewed trust and camaraderie that he has begun to show towards Castiel. Even a renewed sense of friendship, despite everything.
Castiel can't leave Dean because he doesn't want to, couldn't make himself want to if he tried. Not now that he appreciates how precious and fragile this friendship is. And Castiel knows that he doesn't deserve it, but he is not letting it go.
On rare occasions the two of them are somewhere safe and sheltered, and there is a fire to keep the ever-present chill of Purgatory at bay. Castiel is exhausted from fighting and flying and bleeding for protection wards, and the place itself draining him of power. But Dean is exhausted as well, dead on his feet, so Castiel says he'll take first watch. And when Dean wakes up, on those rare occasions, he tells Cas to get some rest, and smiles.
He feels so grateful, so glad that Dean is there with him.
And then Castiel feels guilty for that, too.