Source: Value Differences As Differently Crystallized Metaphysical Heuristics | Slate Star Codex, by Scott Alexander
We need to navigate complicated philosophical questions in order to decide how to act, what to do, what behaviors to incentivize, what behaviors to punish, what signals to send, and even how to have a society at all. Sometimes we can use theories from science and mathematics to explicitly model how a system works and what we want from it. But even the scholars who understand these insights rarely know exactly how to objectively apply them in the real world. Yet anyone who lives with others needs to be able to do these things; not just scholars but ordinary people, children, and even chimpanzees. So sometimes we use heuristics and approximations.
…
Some examples:
…
EXPLICIT MODEL: Trying to count how many calories and milligrams of each nutrient you get
EMOTIONAL EXPERIENCE: Feeling hungry or full
REIFIED ESSENCE: Some foods are inherently healthy or unhealthy
ENDORSED VALUE: Insisting on only eating organic foods even when those foods have no quantifiable benefit over nonorganic
People can choose metaphysical heuristics or explicit models based on their own innate tendencies, their education, their intelligence, their experiences, and what kind of question we’re thinking about. Rather than talking too much about fundamental value differences, we should be asking where a given person has chosen to place themselves on the metaphysical-heuristic-to-explicit-model ladder at any particular moment.
…
Since most metaphysical heuristics are a stand-in for something real, we should expect blocs of allied people to contain some people who want the real thing, and other people who are running metaphysical heuristics that point at the thing.
…
My guess is a lot of people fluidly move back and forth between these levels, just as I would expect people who are very interested in only eating organic food to also be more likely to care about what percent RDA of vitamins are in their food. This isn’t sinister, or a reason to think that people are only claiming consequentialist arguments for their heuristics. It’s just a natural consequences of the way our values get produced and the fuzziness in everybody’s value system.