James Potter went five years at Hogwarts without realizing Lily had a little sister. Hermione would have preferred if he never realized she existed. Now she's stuck, in Potter's circle of awareness, and maintaining the timeline. Not like he makes it easy, or something.