Howard had always been attracted to Penny. Sure, people thought his friendship with Raj was more than just friendship,
and Howard wasn't the ladies man he made out to be, but Howard liked Penny. A lot.
Howard didn't know why he always came across as a pervert. Most people figured it stemmed from his self esteem issues (the issue being that his self esteem was non existent), and they accepted it. But most of the time,
Howard honestly thought he was complementary towards women.

Penny hadn't always liked Leonard. At first, she liked Sheldon more than she did Leonard.
He was honest, straightforward and didn't care what other people thought about him or his opinions.
Penny soon realised that Sheldon was a little odd, but she loved him for it. Not a romantic love, but she did love him.
Penny thought she loved Leonard. She really did. But, late at night, when she was honest with herself,
she knew they wouldn't work out. She needed to be challenged, not catered to. She needed someone to argue with her, to keep her on her toes, to blow her mind, both intellectually and physically. Sure, Leonard was smart,
but they couldn't have a real debate, as Leonard mostly talked about physics or comic books or science fiction.
That gets boring after a while, especially when you don't understand most of it. The only person who came close to challenging her was Sheldon. And after that, it was Howard.

Howard complimented Penny, albeit in a backhanded way. It was creepy and sweet at the same time.
When she told him to shut up, he did so, but there was always something there. As if he did it to make her happy, but knew he could rile her up again. Not like Leonard. Leonard only wanted Penny to be happy by serving her.
And if he wasn't serving her, he was making her feel stupid with science and words she didn't understand.
Howard always explained things in a way she could understand. Sheldon tried, but seldom succeeded.
If someone were to ask her when Howard Wolowitz invaded her thoughts and became an almost permanent fixture in them, Penny would be unable to answer. She didn't know. However, she did know that she had to keep it to herself, not for fear of ridicule, but for fear of something coming of it.