You Had the Wrong Picture!
Once upon a time, there was a town in England where all stories and legends were real, the town of Spinner's End.
Through some, never cleared circumstances the characters from your favourite stories got stuck in our world, in that particular town and lived alongside the natives.
That is how the king and queen of Arendelle, Mr and Mrs Evans, and their two daughters, came to live on the other side of the street from the Snapes.
Mr Evans had been in the same predicament as his illustrious ancestor, which also made him proud: his daughters were magical.
Unfortunately, Mr Evans had additional problem known as relationship with the native magicals, the Wizarding community. He had the chance that his younger had been able to fit right in the community, but his elder also had a form of wild magic which was systematically hunted and eliminated by the Wizards. So Mr Evans asked local trolls for help, and they masked the magic of the elder daughter.
As the years went by, the younger daughter had built her life in the Wizarding community, and the elder in "the normal" world. The elder daughter end up marrying a nice respectable troll and they settled themselves as Mr and Mrs Dursley in a town called Little Whinging.
It was all fine until the younger daughter got herself killed by a terrorist Wizard and her son was placed in the care of his aunt.
The Dursleys had always known that the Wizards would return to claim the boy as their own and were probably even keeping constant surveillance over him, and for that reason they could have never accepted him in their family. Unnatural ways of doing magic were opposite of their ways of doing, and there was also the constant fear they would be found out by the Wizards and exterminated.
That's how the young Harry ended up having to grow up in the environment without love, and also had to suffer bullying from his cousin Dudley who was actually only trying to protect his parents and fight the threat which was hanging over his family.