The first time Dean ever gets mad at Sammy, really gets mad, they're staying at Bobby's place and they're wandering amid all the cars in the yard. John is off on some hunt or other, making promises about making it back that both boys know he'll never keep. By this time they don't really expect him to anyway. It keeps the hurt from showing in their eyes when he misses a birthday, or forgets the science fair he'd said he would attend.
If anyone would ask him about it later, Dean would claim he doesn't remember, or flat out deny that it happened. But the sound of Sammy's body hitting the old red Ford, followed by the wince he makes when Dean shoves him a little harder until the sun heated metal is searing his flesh even through the layers of shirts, and then the crunching sound of Dean's boots as he stomps away, they would be etched in Sammy's brain forever as The First Time.
The second time ends with John's stern voice and a smack to the back of both boys' heads. They're in Idaho, Sam thinks, although he can't always be sure. The sun had set in a cream-pink sky, leaving behind it a blaze of fury almost foreshadowing the bloody fight that leaves the jagged scar down John's back, and bruises Dean's eye to the point where he can't even open it the next day. But Sam, Sam's taken the brunt of the violence and when Dean finally has his little brother all patched and cleaned, he lays into him.
His voice almost tastes like the blood caked to his split lip as he growls at Sam in that low rough voice he only uses when he's trying to be scary. But Sam can see through it and can tell, just tell, that Dean's more terrified than angry, and when he yells back, yells to make Dean actually show what he's really feeling, John just interferes and destroys any progress either had made.
The third time Sam makes Dean mad Dean leaves him standing in the rain for four hours. Just drives off in the Impala, tires hydroplaning on the black Minnesota asphalt before finally taking hold and stealing the car from Sam's sight. The Third Time Sam calls it, even then, as he watches the tail lights disappear around the corner. He was old enough that he should be counting the times he's been up a girl's shirt, or the times a girl has touched him, no matter how tentatively, and he knows it too. But he can't help the solemn tally he's got locked up in his head.
When Dean comes to get him, he's shivering and wet and almost crying. He fumbles with the handle twice before getting the door open and Dean doesn't bitch about the water Sam is dripping all over the leather seat. He just steels his face and drives the eight blocks back to the one-bedroom apartment they're staying in until John can fetch them after tracking a Phooka for six months before deciding it wasn't worth it.
The fourth time Sam makes Dean mad they don't speak for over two years. Sam leaves him a couple of text messages, and occasionally calls his cell phone just to listen to Dean's gruff drawl on the voicemail recording. He stops when the computerized voice tells him the number is no longer in service.
When Dean finally calls him, Sam's not home. When he checks his messages and hears Dean's voice roll over him, he takes the small tape out of the answering machine and places it in the shoebox he keeps on the top shelf of his closet, along with an AC/DC cassette, and a set bloodstained shoelaces, stolen from the trashcan Dean had tossed them in nearly a decade ago.
He never musters the courage to return the call and Dean never calls him back. Over time the memories are pushed to the back of his mind and he plays make-believe that he doesn't care. He even convinces himself that he likes it better that way.
And then Sam makes Dean mad, really mad, for the fifth time and before Dean has a chance to storm off, or get a really good punch in, Sam's on him, crying into his faded t-shirt, clutching at arms and hair and skin that had been too far away for too long, and Dean just can't stay angry anymore.