One head cannot contain all wisdom.
They see me as all knowing and all powerful. In their eyes I'm a god and, for them, I try to be. But the truth is, I am only a man. I try to see everything and be everywhere, but by looking at the forest so much, I tend to miss the trees. It happened to Sirius all those years ago. Severus also paid a price, thrown in a role he hates. Harry was one of those who suffered the most for my failures. Early in life, by having to survive his relatives, but also later on, when I proved over and over again that Howarts wasn't as safe as I wanted it to be.
Legilimency isn't mind reading and I don't routinely peak in other people's mind. They have a right to privacy. "Those that can trade freedom for a little security, deserve neither freedom nor security" and all that. I never knew about Quirrell and Tom, though I will admit those traps where set up to test Harry, to see where he stood. None of them were truly dangerous though, I had made sure of that.
Harry is the student who scared me the most. It seemed every year he was in danger. Severus came close also with his family and the way James and his clique treated him in school. My heart ached when I learned he had joined the Death Eaters. Tom is the one a failed the most though. I never gave him a chance. Maybe this is why I give more chances than I should now. I don't want to make the same mistake twice.
I play my grandfather persona and act all-knowing so that others feel safer in the world while I fear for those I care and weep for those I failed.