Outbursts of emotion, or any manner of drawing attention to oneself really, were not encouraged when Kaeya was a young child. Khaenri'ah was a nation ravaged by war and overrun by monsters of the abyss, danger lurking around every corner. One wrong move and it would swallow you whole.
After his clan became regents to the throne, he was sent to Mondstadt for his own safety and to infiltrate their society should his position be needed when Khaenri'ah sought to rise again and claim revenge on the gods that brought the great nation to ruin.
There was little so terrifying to a child as to be abandoned by his family in a strange land, but gradually he was able to adjust. The Ragnvindr family took him in, cared for him, loved him, comforted him in a way he had never been afforded in his past life. He received a good number of lingering looks and an occasional comment on his unusual appearance - a dark skinned, blue haired boy from another land didn't really blend in - but even half of those could be in attributed, at least in part, to his constant companion, Diluc - a fiery redhead from a notable family. As a whole, Mondstadt was welcoming and friendly even to outsiders though.
In time, he could almost forget his early life of living in fear and darkness. His adoptive family was respected and admired. He was able to begin a promising career with the Knights. He had friends. He had good potential marriage prospects. He was welcomed into the most profitable industry in the city. He had a brother he would do anything for, and who would do the same for him. He couldn't ask for more.
And then it all fell apart.
He, a sinner from a fallen nation, wasn't destined for a life of luxury and happiness, and he had been careless to forget that.
His adoptive father died tragically, and he was unable to prevent it like a good son would have. It brought to mind his biological father, the family he left behind, and lost in a mix of emotions he wasn't supposed to show, he made another tactical error.
He told Diluc the truth of his arrival in Mondstadt. It was a poor choice if he was ever to complete his mission, but he couldn't bear the burden of lies alone anymore. Unfortunately, he had the worst possible timing. And he should have started at the beginning, built up sympathy for his plight. Diluc was such a naive and optimistic young soul though, and he didn't want to taint that with his nightmares any more than he had to. Instead, he stated the truth with little context to soften the blow.
And his brother lost it.
Already grieving and traumatized, the news that the boy he grew up with, who had essentially become a brother to him, was a traitor was too much to deal with rationally. They fought. Angry words turned into actions. And eventually Kaeya faced his burning wrath.
If not for Celestia's intervention, Kaeya would have burned to death that night. Instead, he was gifted a cryo vision, a frozen reminder of his failures, and that he must live the rest of his life bearing the burden of the lies of his past alone.
Now, as he lay in darkness and alone on a bunk in the Knight's barracks, kicked out of the home he grew up in, he let the tears fall, silent sobs racking his burned and scarred body. And if a monster crawled out of the abyss and swallowed him up for the display, all the better.