Wish
Harry sometimes thought it would be easier if the Dursley's straight-out abused him.
Instead, they didn't, which gave him hope that maybe they did love him like a family. That one day, he would be praised like Dudley.
It would be so much easier to hate them, but he couldn't, since they had never actually done anything to him. Maybe some pushing around, but that could easily be taken as a bit of disciplining.
No, Harry didn't hate the Dursley's for the neglect of love while raising him through his childhood. He just wished they hadn't let him hope.