"You can't expect him to say goodbye to Susan and forget about her the next minute." -Barbara Wright, "The Rescue"
In the weeks after the Doctor left his granddaughter in a wartorn Earth with a man she had only just met, he wonders how she walked back to the rebel base with only one shoe.
It's these kind of anxieties that makes him regret his decision to force the girl- no, the woman- into adulthood. Did she cut her foot on the way back? Did David help her? Would he take care of her? Love her? Had the Daleks destroyed her already, making all of these worries in vain?
He keeps her shoe a secret in a little drawer under the TARDIS console. That way, Barbara and Ian can't bother with it. They worry enough about him, simply due to his physical appearance, but he can't be bothered to explain to the humans aboard the ship that his true age was far beyond their perspective of his physical age. No, he's far too tired most days for that. They already thought he was senile.
The night after the pair leaves him, he's eternally grateful to the Universe that Vicki and Steven either didn't or pretended not to see the tears glassing over in his eyes, clutching her shoe to his chest in an attempt to cling to the last physical memory of Susan that he had left.
He tries to remember what he told her: no regrets, no anxieties, but with Ian and Barbara gone, there's no one left who can remember her smile, the melodic ring of her laugh, the sparkle in her eyes as she teased him about being too slow. He never minded. Not really. He would give anything to have her back now. But it was too late. All he had now was a fraying, broken shoe and his memories.
They seemed to grow more and more grey by the microspan.