When you lose a soulmate, there are some things you come to know.
John knew Sherlock had been the one for him, the love of his life, and no-one else would ever come close. He also knew that Mary was sweet, pretty, in love with him and that he might like to spend the rest of his life with her, and at the same time he knew that she would never make him feel like Sherlock did. She could never make him feel as happy, as alive, as important, as needed, as loved as Sherlock did. As Sherlock had.
He closed his eyes and, just for a moment, allowed himself to imagine growing old with Sherlock.
They'd have ended up sleeping together at some point, probably after finishing a case, and afterwards John would've confessed how he really felt about his best friend, and it would all have led from there. A wedding, a child - perhaps two. Eventually Sherlock would have left the work behind and they'd have moved out of London to retire - Sussex, maybe. He'd have learnt to make jam, Sherlock would have kept bees (he had never actually told John of his unrealised beekeeping dreams but John had deduced it from the well-worn books and internet history left on John's laptop whenever Sherlock "confiscated" it. He reckoned Sherlock would be proud.) The grandkids would have loved to come visit and listen to their grandfathers' stories, and in the mornings Sherlock would've read the paper, ignoring John whenever he found anything interesting, and they would have argued over experiments that Sherlock didn't need or shopping trips John didn't want to make, and everything would have been different but nothing would have ever really changed...
But he forced himself to stop, and to open his eyes, and to wipe the smile off his face, because Sherlock was dead, and Sherlock would never grow old, and John would never grow old with Sherlock.
This was just one of those things that John Watson knew.