Sandaime
He'd been excited and a bit nervous to be Hokage. Then he discovered that with peace time, Hokage's became glorified paper pushers who had Anbu guarding them to make sure they didn't try to sneak out to smoke their pipes or read about interesting new jutsu.
Taking a genin team, especially with Tsunade on it had seemed like a good idea, what with a new war starting up. True, Jiraiya was the one to understand the bell test, and Tsunade was a damned good kunoichi, best he'd ever seen, but it was the pale quiet Orochimaru who caught his interest.
His whole being focused itself on learning new jutsu and new applications for already existing ones. Everything about the ninja arts fascinated him. Sarutobi admits that he was a bit neglectful of his other two students to train the boy, but he couldn't help it. The boy, at least in his thirst for knowledge was so much like him.
Then, then that thirst for knowledge turned black, for a child, who by age ten had already stained his hands with blood, learned the darkest of ninja arts, things that would bother a grown-up barely phased a child like that. And much like the snakes he loved and resembled he grew darker, and colder, and crueler every day.
Orochimaru's defection was what decided him. If he was so willing to be blind to faults like that, activities like that, yes, it was time to hand the title over to someone else. He was obviously too complacent about things, let the next generation take over.
Besides, now he might actually have time to smoke his pipe, though he didn't think he would ever be able to enjoy studying justsu quite the way he had before.