When Harry had an affair with Daphne, it didn't mean he stopped loving Ginny. He loved her with all his heart, and she will always hold a special place in his life. His affair with Daphne just happened, and he did not love her. Daphne was an Unspeakable who got partnered up with him. Usually, Unspeakables kept to themselves, but the case that she was assigned was about the Death Eaters that were never caught after the war. These types of cases would usually go to the Aurors Department, but these Death Eaters are very dangerous and deadly. So the Ministry of Magic chose one of their best Unspeakables, which was Daphne, and one of their best Aurors, which was Harry.
They spent a lot of time together during this case, and there was a strong sexual attraction between them. He tried to ignore it, but couldn't. It didn't help that Daphne was very attractive with her curly blonde hair and big blue eyes. The first time it happened was when they were at the bar. They got a lead about the Death Eaters, and the lead was a bartender who, obviously, worked at a bar. They went to go get information from the witness, so they went under a disguise so they wouldn't alarm anyone. At the bar, they drank too much and got very drunk. One thing led to another, and they slept together. Harry vowed and said that it will never happen again. However, that's not what happened, and they slept together many more times. The result of this affair was unexpected.
"What do you mean you're pregnant," Harry asked while running his fingers through his hair. He never meant for this to happen and Daphne being pregnant with his child was definitely not the plan. Ginny was pregnant with their first child, and he did not want to tell her. They could get a divorce, and he loved her. He couldn't go to his friends for help, they would be disappointed with him, and demand him to tell Ginny.
"It's exactly as I said," Daphne responded while drinking her tea. He glanced at her and noticed that she looked like the pureblood she was raised to be. She was calm and collected, but he could see her glowing with happiness.
"You can't keep the baby Daphne," Harry said. He was stressed and wasn't thinking straight. That's probably why, later on, he would regret the words he would say to her. All he could think about right now is that nobody could know about the baby. "I'm married and Ginny's pregnant. I love her. What happened between us was a mistake. The baby was a mistake."
He saw Daphne tense up, and anger appearing in her eyes. "I'm keeping the baby Potter. I'm not asking for your permission, and the only reason I told you was because you're are the father."
Now it was Harry's turn to be angry. "Are you trying to ruin my marriage, Daphne. You know that if anyone ever found out about our affair, my life would be ruined," Harry spat out angrily. Maybe it was his anger clouding his vision that caused him to say, "Do you want money, Daphne? Is that what you want. How much money would you want to not tell anyone about this affair and to abort the baby." He realized his mistake and regretted those words immediately when Daphne slapped him across the face. He saw her face and could see the anger and hurt. She was trying to hide it, but he could still tell.
"I don't need your money Potter," Daphne hissed as she stood up. "But I can see that you don't want this baby. I'll get an abortion as you wish, but never look for me again."
She turned around and apparate out of there. At that time, Harry felt relief flood through him as he registered her words. The relief of Ginny not knowing about this incident. But the relief was also overpowering the guilt. The guilt that he just ordered his child to be killed. It never crossed his mind that Daphne might have been lying about the abortion. She would keep the baby and one of Harry's biggest regrets would be not watching his child grow up.