He knew they all left eventually. He knew that no one was going to stay with him forever, that's just how these things work. He thought about Donna, who had lost all memories of his existence and their time together. He thought of Martha, who left by her own free will. He thought about Rose, and how she was stuck somewhere without him, definitely against her will.
Not a day passed by when he didn't question why Amy and Rory were still with him, after everything he'd put them through. Being stuck outside the universe. Losing their child. Rory dying more times than one could count. And those two had each other, so for every day and every adventure he could have with them, The Doctor was immensely grateful.
But then came those times when Amy mentioned that she wanted to go home, just for a few months or so. And she started to question whether their friends would notice that they aged faster than other people. And The Doctor knew that their time together was running out. He felt more and more like that imaginary friend he'd been for Amy for so many years. He knew there would come a day when the Ponds wanted to stay at home for good. He just hoped he'd be able to keep them alive until then.
And so when they left Mercy, and Amy asked to be taken home, The Doctor made a decision. When the TARDIS stopped and Amy ran outside, expecting to be surrounded by everything she knew as home, she found herself standing on a beach instead.
"Doctor, where are we exactly? I thought we were-"
"I know, we are going home," The Doctor interrupted her. "This is just a slight detour."
Amy frowned.
"It's okay, we're on earth, it's 2012, and it's safe. Don't worry!"
Rory joined them, and the three of them sat down on the sandy beach, the sun was shining, and it was really a beautiful day.
"Where exactly are we?" Rory wondered.
"Norway, not too far from Bergen." The Doctor replied. "We're here because I have a story to tell."
And so The Doctor began telling the story about Rose Tyler. About the girl he'd met and lost. About their adventures, how they'd spent Christmas together. After he was finished, there were a few minutes of silence before Amy dared to speak.
"Where is she now?"
"In a parallel universe. I'll never get to see her again."
Amy regretfully remembered the time when she'd accusingly asked The Doctor about all his female companions, and wondered how many of them he'd lost like this.
"Is this where you last saw her?" Rory guessed.
The Doctor nodded absentmindedly, his thoughts apparently somewhere else entirely.
Neither Amy nor Rory knew what to say, they hadn't seen this vulnerable side of their otherwise so giddy friend.
Suddenly he clapped his hands together, smiled and stood up. "Well, home for the Ponds it is, right?"
With that he went back to the TARDIS, only looking back at the beach once.
Not many minutes later, The Doctor was saying goodbye to his beloved ginger friend and her husband, Rory the Roman. He told them to take care of themselves, and each other. He insisted on extra long hugs and when Rory went to unlock their front door, The Doctor pulled Amy back and whispered to her "Thank you for waiting, I hope it was worth it."
He could have told them that this was the end, that he wasn't coming back, that this was the last they'd ever see of him. But that would've hurt, and deep down The Doctor knew that this was the right decision.
Amelia Pond had grown up, she had a husband, a job, a life. She didn't need her imaginary friend, her raggedy Doctor. She wouldn't be waiting any longer..
And no matter how sad it made him, The Doctor knew he'd done the right thing. Because Amelia and Rory were still alive, breathing and, finally, safe.