Memories belong in specific places. Patrick Jane is well aware of this. He exploits this fact to create his memory palace, but he also finds that it works well with physical locations. For example, his lonely mansion in Malibu houses the memories of his wife and daughter's deaths. He had emptied the place of most homely touches years ago, eliminating nearly every single reminder of how they had lived together as a family. Those wonderful memories shouldn't be tainted by such horrors.
When Jane wants to remember the happy times, he goes out on his red and white yacht. He remembers how the three of them used to go out at night, away from all of the light on land, and look at the stars. Charlotte would usually fall asleep there, and he and Angela would enjoy a nice bottle of wine in the moonlight.