It's not just Palantir, it's fairly common within tech interviews, I've been asked this at at least 5 popular companies when looking for a job out of college.

It's a really good question for testing practicality in the face of a problem that is pretty inefficient regardless of how you do it. I think the major reason is the use of hash tables, which I think industry values a lot more as a practical tool than I think academia gives it credit for.

I think the first part is that they want you to choose the right data structure for the job, even if the algorithm is dead simple. With the generalization part, I think they're really just testing your inductive skills as a computer scientist, which is something any cs program goes over a million times.