They all ask that question, each of them, eventually. Friends, feds, family and fellow thieves. At one point in time they all ask was it worth it?
He takes a moment to think about it; pondering whether or not the lies and cons were overshadowed by the betrayal and manipulation. If the heists were worth his time in jail and the years in the tracking anklet. He wonders if all his wrong doings are justified by his good deeds, though he knows they aren't.
He's paid the price more than once, in losing Kate Moreau and almost Mozzie too, straining his relationships with Peter and Sara, and putting El in danger on more than one occasion. He thinks about how everything has unfolded; how he ended up exactly where he is now. When they ask "was it worth it?" he thinks about how in the end everything led back to her and to them, every single time.
When Neal Caffrey answers "It was worth it" he's not thinking about his greatest score or the legacy he left behind, he's thinking about her and him; about stolen treasures and origami flowers.