I had an idea about why Jack's hat means so much to him. I do not own any Pirate characters, if I did, Johnny and Orlando. I do own Angel Sparrow.
Jack Sparrow laid in his cabin on the Pearl. In his hand was his hat. Looking on the inside, he found the initials AS, Angela Sparrow, or his angel as he called her, his little sister of 15 years. He remembered when she was born; she had her father and brother's dark brown eyes. As she got older, she grew long brown hair, just like Jack's.
Angel loved hearing the pirate stories their father, Captain Teague, would tell them as bedtime stories. Angel would cuddly up with Jack and fall asleep in his arms.
Then he remembered the day she gave his hat. Jack was about to leave on the Pearl for the first time, when he heard a voice calling him "Jack wait!" Jack turned and saw his running down the dock to him, holding something in her hand.
"Angel love, what's wrong?"
"I made you something" With that she handed him the hat. Jack looked at the three-corner brown leather hat as if it we the holy grail, then he took his sister in his arms and embraced her one last time.
Jack was brought back to the present. He remembered the last time he saw his sister. It was the day she married a fine pirate named Nathaniel. Angel had gone into the family business and became a pirate. Now she and her husband were sailing the seas on their ship the Dragon.
Even though she was married and had her own life, to him she would always be his angel. His angel who him his hat.