This was just Neal thinking about his life and where he would be if he had made different choices.
It's 4:35 in the morning and all Neal wants to do is sleep. However, sleep is playing hard to get tonight. He's laying in an empty bed, wide awake, cold, wishing Peter where there to keep him warm and talk to him until they both fell asleep. But that's not the way it works. No, Peter is with El tonight, not with Neal. He's at home keeping someone else warm, while Neal is across town, alone.
It's nights like these that hurt the most. It's these nights that he has to much time to think and remember. Or at least try to remember. He thinks about Kate. It's been a few years since her death and he's already forgetting what she smells like, taste like, feels like. And that's what gets the tears going, and then he'll never get to sleep.
He wonders what life would be like if she hadn't died. Wonders where he would be. Would he still be with the FBI? With Peter?
He doesn't know, and he doesn't want to know. He loved Kate with all his heart and soul and whatever else he could give her, but Peter, there was something different in the dynamic of their relationship. He didn't want to think about what path he would be on without him.