Disclaimer: All characters and the universe they inhabit belong to CBS and other people who aren't me. I'm just playing around with them for a while.
The first time Sam ever says it, they're walking to the meet the others at a bar. This pretty woman gives Sam a wink as she and her friends pass by, and Callen remarks on it. Sam merely shrugs and comments that it doesn't matter to him anymore, because he's too in love with Callen to do anything about it. When his partner stops in the middle of the sidewalk to stare at him, he simply looks the other man in the eye and adds that while it's true, Callen isn't obligated to do anything about it. Then he turns and walks into the bar. To Callen's credit, he follows, although he doesn't go home with Sam as he normally would.
Over the next several months Sam casually slips the phrase into their daily lives anyway he can. He mutters I knew I loved you for some reason when Callen hands him coffee on late nights, or announces and this is why I love my partner when Callen comes up with something particularly brilliant. He slips you know I love you but or just because I love you doesn't mean into their bickering. He includes it in text messages and on the little scraps of paper on which he writes directions, phone numbers, and the food orders the team compiles when they stay late.
Otherwise, there is no break in the comfortable routine they have built for themselves. The rest of the team has either not noticed or are treating it as simply an outgrowth of their normal interaction. Both scenarios scare Callen just a little. These little confessions are uttered casually, as easy as comments on the weather. Sam doesn't leave space for a reply afterwords, and it takes Callen nearly six months to realize he isn't expecting one. That he doesn't need one to validate the way he feels, or reciprocation to continue being Callen's friend.
It is that realization that causes him to show up on Sam's porch one Friday night, the first time he has been there since that first confession, and spends the most nights there he ever has (the three nights of the weekend, but still the longest).
He still hasn't been able to make that reply, but he thinks that someday he will be.