Thestrals can go a lot more places than just wherever you need to go. Unfortunately for Harry Potter, Voldemort is more than aware of this, and doesn't want to deal with Harry Potter anymore.