For all his witty banter, Q makes little reference to his past. There is a certain level of professional conduct, most of which they ignore, and somewhere amoungst the list of rules and procedures that Bond disregards there is something about 'personal information' not being discussed during missions.
They are friends. Despite his numerous defenses and general distrust of people Bond has accepted Q as one of the few and he assumes the status is mutual.
Still, he knows little of the Quartermaster's life before or outside of MI6.
007's history is in his file. The file is electronic by now and that means it belongs to Q. Everything that uses wires or chips belongs to Q and Bond has no doubt that his past is already known. Really, though, that only sets him on edge all the more.
Q knows about him. Knowledge is power, that's a simple enough fact. Bond knows nothing about Q and that means he's the weak one. He could likely bench press Q's weight, he might well put himself in front of a bullet for Q. He does not choose his friends lightly and he is rather done with losing them. Still, his stomach churns at the imbalance between them.
He has to trust in his friend and it scares the shit out of him.