Tulsa was a small town. More often than not you knew someone, and if you didn't then you knew who they hung out with or whatever.
So because it was a small town, if something happened to one individual or family, everyone knew.
Which was why it was torture for Randy to go to school.
"There's that Soc who didn't fight them Greasers..." the Socs would sneer.
"There's that Soc whose too good to fight us..." the Greasers would mutter, shoulders hunched.
So although it was a little sadistic for him to say so, at least his life was Ponyboy's.