The problem, when one really got down to it, was that of time.
Castiel could not manipulate time, per se. No angel could. Time itself was a dimension like any other. But he could alter his experience of time, or the experience of others in time. Or he could move himself or others within that dimension. There was a lot that he could do in respect to time, which was advantageous, given that he had lived for eons and, to be frank, there was a lot of boredom during those eons that he would rather not experience to the fullest. Humans made a very large fuss about the Cambrian Explosion, but unless one observed it very quickly, it was just a lot of many-legged bugs wandering about and fighting and dying. The fossils were more exciting than the event itself.
If he knew that a certain event was going to take an inordinate amount of time, he'd change his perception of time - the experience of centuries could be compacted into moments. He had idly done the math once and was surprised to discover that in subjective time, he'd only lived a little over three thousand years, though he could of course perfectly remember the march of millennia.
Perhaps if Dean knew that, since the first moment of their meeting, he had not once altered his perception of time when he'd been in Dean's presence - that it had been a conscious choice to live every moment just as Dean did, both painful and joyous - and that he increased his observation of time tenfold when he was not there. Or if Dean knew that fully one third of his attention was always trained on the Winchesters, wherever they were. Or if Castiel finally admitted that keeping his "ears on" meant a steady drain on his Grace that he did not dare cease for fear of missing one single whispered prayer.
Perhaps if he could only make Dean see that just because he wasn't present didn't mean he was gone, Dean would forgive him.