"Even through his anger and impatience Harry recognized Hermione's offer to accompany him to Umbridge's office as a sign of solidarity and loyalty" (ch. 32).
Hermione didn't think that Harry's vision was real, and told him as much. She knew he wasn't supposed to have seen it, there was a reason for that, and she was sure it was a trick, playing off his weaknesses.
He blew up at her, of course, and she defended herself, but he was firm in his course.
And even though she thought it was wrong, even though she disagreed, she would go into Umbridge's office with him. And into the Department of Mysteries.
Because she was his friend, and she would support him, because that was what they did.
I don't think this one needs any explanation. Review!