Klaus knew that the cave men came billions of years after the dinosaurs, and had no idea they existed. When everyone found this out they left Olaf, and went on about their lives.