This is set after the crew meets Luffy's grandfather for the first time. This is a conversation that could have taken place between the two(Luffy and Garp), during their time on the island together.
"I don't like it Luffy. I don't like it at all," Monkey D. Garp's brows furrowed. "No matter how you put it, your subordinate defied your obvious intent; to bring the long nosed sniper back. He not only refused to comply but persuaded the others to act otherwise as well. I don't like the sound of this Roronora chap. I've seen greater pirates fall by the hands of their own-"
"No Gramps," Luffy protested. Garp watched his grandson, the scrawny kid who used to hide behind his older brother, bristle with genuine disapproval against the doubt towards his first mate. "It's not like that."
To conceal his amusement, Garp snorted as he asked.
"Then?"
Luffy paused, but only for a moment.
"He gets to do that," The corner of Luffy's mouth slipped into a wide grin as he explained painfully simply. "He's Zoro."