Their household wasn't safe for Peter, and both he and his ex-wife Tina knew it. He loved the kid, there was no doubt about that, but Mathias and him... They didn't have the most conventional relationship. Besides, right off the bat the state would only give him visitation rights, because he lived with another man. So he took those quietly and without a fight, trying to keep in touch with the boy more and more the older he got. Berwald wanted to be there, wanted to still be some sort of father figure, to still be around and involved.
The two of them had some habits that weren't safe for kids to live around, though. No matter how much they tried to lock the basement door, they could always forget that one time, and then... Then they would have to explain why dad had daddy chained to the wall, or why daddy had a knife to dads chest, or why dad and daddy had been biting each other. They wouldn't know how to deal with that, the two of them hardly knew how to deal with kids in the first place. So it was best to just avoid the problem. It was safer for Peter's sake. The kid deserved a semblance of normal life, after all.