Eliezer Yudkowsky is on Facebook. To connect with Eliezer, join Facebook today.
Eliezer Yudkowsky is on Facebook. To connect with Eliezer, join Facebook today.

Eliezer Yudkowsky

Xposted w/ edits from a comment on Effective Altruism, asking who or what I cared about:

I think that I care about things that would, in your native mental ontology, be imagined as having a sort of tangible red-experience or green-experience, and I prefer such beings not to have pain-experiences. Happiness I value highly is more complicated.

However, my theory of mind also says that the naive theory of mind is very wrong, and suggests that a pig does *not* have a more-simplified form of tangible experiences. My model says that certain types of reflectivity are critical to being something it is like something to be. The model of a pig as having pain that is like yours, but simpler, is wrong. The pig does have cognitive algorithms similar to the ones that impinge upon your own self-awareness as emotions, but without the reflective self-awareness that creates someone to listen to it.

It takes additional effort of imagination to imagine that what you think of as the qualia of an emotion is actually the impact of the cognitive algorithm upon the complicated person listening to it, and not just the emotion itself. Like it takes additional thought to realize that a desirable mate is desirable-to-you and not inherently-desirable; and without this realization people draw swamp monsters carrying off women in torn dresses.

To spell it out in more detail, though still using naive and wrong language for lack of anything better: my model says that a pig that grunts in satisfaction is not experiencing simplified qualia of pleasure, it's lacking most of the reflectivity overhead that makes there be someone to experience that pleasure. Intuitively, you don't expect a simple neural network making an error to feel pain as its weights are adjusted, because you don't imagine there's someone inside the network to feel the update as pain. My model says that cognitive reflectivity, a big frontal cortex and so on, is probably critical to create the inner listener that you implicitly imagine being there to 'watch' the pig's pleasure or pain, but which you implicitly imagine not being there to 'watch' the neural network having its weights adjusted.

What my model says is that when we have a cognitively reflective, self-modely thing, we can put very simple algorithms on top of that---as simple as a neural network having its weights adjusted---and that will feel like something, there will be something that it is like that thing to be, because there will be something self-modely enough to feel like there's a thing happening to the person-that-is-this-person.

If the one's mind imagines pigs as having simpler qualia that still come with a field of awareness, what I suspect is that their mind is playing a shell game wherein they imagine the pig having simple emotions and that feels to them like a quale, but actually the imagined inner listener is being created by their own minds doing the listening. Since they have no complicated model of the inner-listener part, since it feels to them like a solid field of awareness that's just there for mysterious reasons, they don't postulate complex inner-listening mechanisms that the pig could potentially lack. You're asking the question "Does it feel like anything to me when I imagine being a pig?" but the power of your imagination is too great; what we really need to ask is "Can (our model of) the pig supply its own inner listener, so that we don't need to imagine the pig being inhabited by a listener, we'll see the listener right there explicitly in the model?"

Contrast to a model in which qualia are just there, just hanging around, and you model other minds as being built out of qualia, in which case the simplest hypothesis explaining a pig is that it has simpler qualia but there's still qualia there. This is the model that I suspect would go away in the limit of better understanding of subjectivity.

So I suspect that vegetarians might be vegetarians because their models of subjective experience have solid things where my models have more moving parts, and indeed, where a wide variety of models with more moving parts would suggest a different answer. To the extent I think my models are truer, which I do or I wouldn't have them, I think philosophically sophisticated ethical vegetarians are making a moral error; I don't think there's actually a coherent entity that would correspond to their model of a pig. Of course I'm not finished with my reductionism and it's possible, nay, probable that there's no real thing that corresponds to my model of a human, but I have to go on guessing with my best current model. And my best current model is that until a species is under selection pressure to develop sophisticated social models of conspecifics, it doesn't develop the empathic brain-modeling architecture that I visualize as being required to actually implement an inner listener. I wouldn't be surprised to be told that chimpanzees were conscious, but monkeys would be more surprising.

If there were no health reason to eat cows I would not eat them, and in the limit of unlimited funding I would try to cryopreserve chimpanzees once I'd gotten to the humans. In my actual situation, given that diet is a huge difficulty to me with already-conflicting optimization constraints, given that I don't believe in the alleged dietary science claiming that I suffer zero disadvantage from eliminating meat, and given that society lets me get away with it, I am doing the utilitarian thing to maximize the welfare of much larger future galaxies, and spending all my worry on other things. If I could actually do things all my own way and indulge my aesthetic preferences to the fullest, I wouldn't eat *any* other life form, plant or animal, and I wouldn't enslave all those mitochrondria.

July 22, 2014 at 8:37pmBerkeley, CA, United StatesPublic

Eliezer Yudkowsky

Yes.
2Aug 11, 2014

Eliezer Yudkowsky

Robert: I'm pretty happy to take a stand on it being okay for me to not update on other people believing in P-zombies. If we agree that these issues are of the same kind, then this case is far far more accessible to me---much much easier to communicate. Like "not P-zombies" is 3% of the way to the finish line of the previous issue.
Aug 11, 2014

Rob Bensinger

Robert: 'Zombies! Zombies?' probably doesn't hit hard enough outside the context of the sequences. Inside the sequences, it's one of dozens of posts hammering on one of Eliezer's most basic theses: 'Beliefs are physical states of the brain, useful because we've evolved to be able to put those states in physical correlation with configurations of stuff outside the brain. Acquiring veridical beliefs, then, is a special case of thermodynamic work, and requires the emission of waste heat. Any specific reliable belief about any matter of fact must (in principle) admit of a causal explanation that links the brain state to the worldly circumstance it's asserting.'

Now we're not just picking on Chalmers. We have a general causal theory of evidence, and as a side-effect it predicts that brains can't know about epiphenomenalism (even if it's true!). No event in the brain, including any thought experiment or intuition or act of introspection, can causally depend in any way on epiphenomenalism, the character of (epiphenomenal) phenomenal consciousness, etc. Our brains' miraculously guessing correctly that they are attached to an epiphenomenal ghost would be a pure lucky coincidence. Like guessing 'there's a ghostly wombat 200 feet under my house that has never causally impacted anything' and getting it right; or like guessing 'every diamond lattice in the universe is attached to an epiphenomenal feeling of regret'. The greater causal proximity of our brains to our consciousness shouldn't make any difference, because the causal arrows only go -away- from the brain, not toward it.

If you assign a low prior to 'all diamonds are epiphenomenally regretful', you should also assign a low prior to 'my brain is currently epiphenomenally attached to a visual experience of seeing this sentence, which has no impact on my brain's beliefs or assertions to that effect'. And if you have no prior reason to believe there's a Matrix God who created our world, loves epiphenomenalism, and would build dualistic minds with just the right mind-matter bridging laws (a view called occasionalism), then no argument or experience you have can lend evidential support to epiphenomenalism, for the exact same reason it can't send support to the diamonds-are-regretful thesis. If you can't establish a causal link inside the system, then you need to posit a force outside the system to establish that causal link -- else the physical process we call 'evidence' just won't work.
Edited5Aug 11, 2014

Rob Bensinger

I think it's pretty widely accepted by philosophers that traditional epiphenomenalism is one of the more fraught and hard-to-believe views on consciousness. (E.g., it was abandoned a decade ago by its most prominent modern defender, Frank Jackson.) What Chalmers would say if you brought this criticism to him (and has said before, on LW) is that his view isn't 'traditional epiphenomenalism'. His view allows for phenomenal consciousness to be 'causally relevant' to our phenomenal judgments, because his view identifies consciousness (or its underpinnings) with the fundamental thingies ('quiddities') that play the functional roles described in physics. It's mysterious what electromagnetism is 'in itself' (above and beyond the relations it enters into with other patterned empirical thingies), and it's mysterious what consciousness is, so we combine the two mysteries and sort-of-solve them both. (Which feels elegant and gratifying enough that we're willing to abide the lingering mysteriousness of the view.) This is also Strawson and David Pearce's view. See http://consc.net/papers/panpsychism.pdf.

I don't think this is an adequate response to Eliezer's objection. Aside from occasionalism, I haven't yet seen any adequate response to Eliezer's objection in the literature. The problem is that this view treats 'causal relevance' as a primitive, like we can just sprinkle 'causality' vaguely over a theory by metaphysically identifying phenomena really really closely, without worrying about exactly how the physical structure of a brain ends up corresponding to the specific features of the phenomena. The technical account of evidence Eliezer is giving doesn't leave room for that; 'causal relevance' is irrelevant unless you have some mechanism explaining how judgments in the brain get systematically correlated to the specific facts they assert.

If the zombie argument works, quiddities can't do anything to explain why we believe in quiddities, because our quiddities can be swapped out for nonphenomenal ones without changing our brains' dynamics. If the qualia inversion argument works, quiddities can't explain why we have accurate beliefs about the particular experiences we're having (e.g., as William James noted, that we're experiencing phenomenal calm as opposed to phenomenal agony), because the quiddities can be swapped out for other phenomenal quiddities with a radically different character. The very arguments that seek to refute physicalism also refute all non-interactionist dualisms. I think this has been unclear to professional philosophers because philosophers of mind mostly treat 'causality' and 'evidence' as black boxes, rather than having a detailed theory of how they work (such that they would need to constrain their philosophy-of-mind views to accord with that theory).

Systematic philosophy is relatively unpopular in modern academic analytic philosophy, so different fields often carry on their debates in isolation from each other. And systematic philosophy is -especially- unpopular these days among hard-nosed reductionists -- the sorts of academics most likely to share Eliezer's interests, background, and intuitions.
Edited4Aug 12, 2014

Rob Bensinger

If you want to know how a blind person guessed that you're holding a diamond, it's not enough to say that the diamond is 'causally relevant' to the blind person (e.g., photons are bouncing off of the diamond to hit the blind person). You need at least a schematic account that allows the blind person's brain to systematically correlate with the physical structure and location of the diamond. If your assumptions are telling you that a certain systematic correlation between belief and reality is a coincidence, then, vague 'causal relevance' or no, at least one of your assumptions must be wrong.
3Aug 11, 2014

Eliezer Yudkowsky

I agree with Robby's reply.
Aug 12, 2014

David Pearce

[Robbie, _if_ Strawsonian physicalism is true, then all consciousness, and only consciousness, has causal efficacy because consciousness discloses the intrinsic nature of the physical. Thus causality isn't "sprinkled"; it's integral. For many purposes, e.g. the functioning of a personal computer, the particular values of an information-processor's constituent fields of hypothetical micro-qualia are entirely incidental: swap the silicon chip for a gallium arsenide chip and the programs will execute just fine. No, the particular values of the world's fundamental qualia fields aren't literally arbitrary: they are yielded by the solutions to the quantum field-theoretic equations. No property of the world is missing from the formalism of physics or derivative from it. But in many contexts, these particular fundamental qualia field values are just irrelevant implementation details of information-processing system in question.

Our classical digital computers / silicon robots are (probably!) zombies. They aren't endowed with unitary phenomenal minds. Strawsonian physicalism is not animism. The challenge - _even if_ Strawsonian physicalism is true - is to explain why organic robots aren't zombies too: to explain how phenomenal binding is possible, and how consciousness can refer to itself. I won't rehash my views again here. But indexical thought, e.g. _this_ particular self-intimating thought, and _this_ particular self-intimating pain lies on a continuum. There isn't some radical discontinuity where pain becomes meta-pain - or on Eliezer's conjecture, when insentient noiception becomes self-reflective phenomenal pain. And the fact that self-intimating agony is more evolutionarily ancient than self-intimating indexical thought-episodes doesn't make the agony any less intense: quite the contrary.]

Anyhow, and ethically much more important from a practical point of view...
Suppose, say, I hold a minority position. I believe I have cogent arguments that tomorrow's "mind uploads" lack bound phenomenal experience. "Uploads" are just digital zombies that I can molest at will - with no more compunction than the hostiles of today's violent games. Let's suppose the great majority of the scientific community are unpersuaded by these arguments. What is the ethically appropriate way to behave? Forcefully reiterate my position, lament how researchers haven't understood or properly worked their way though the long but compelling chain of reasoning demonstrating that I'm correct - and then cause grievously bodily harm to [what I'm convinced are] just zombies? Or have the cognitive humility to acknowledge that I could well be wrong? Acting out the consequences of idiosyncratic views can be ethically catastrophic.

Factory farming pigs for the dinner table on the contested assumption they are just zombies is no different.
2Aug 12, 2014

Rob Bensinger

"causality isn't 'sprinkled'; it's integral"

That isn't what I meant by 'sprinkled'. I meant that you need an actual mechanism (at least a toy one, as a proof of concept) for how the brain ends up correlated with its referent; saying the referent is 'causally relevant' or 'integral' doesn't give us such a mechanism, even schematically.

Suppose you claim to know that the 'inner nature' of a piece of bread is the Body of Christ. Without wading into any complex theology, it's perfectly reasonable for me to doubt your claim •simply on the grounds that you haven't given me a mechanism for how you came to know about the bread's 'inner nature'•. (And I can doubt this just as easily whether or not I agree with you that bread •has• an 'inner nature'.)

Now, there are ways to solve this problem. The Matrix God might have told you directly about the bread's inner nature. (This is also the only solution I know of to Eliezer's objection to non-interactionist dualism. The Matrix God is outside the system, and can bridge causal gaps that are impossible inside the system.) Or you say: the 'inner nature' of an object discernibly changes the bread (e.g., it makes it glow) or discernibly changes things near the bread (e.g., it makes nearby crosses glow); this glow then strikes the claimant's retina, which leads to the claimant's belief 'there's something special about this bread', and then (after considering several hypotheses) to the claim 'this bread's inner nature is the Body of Christ'. That's a reasonable schematic account, even if the details are slightly off.

On the other hand, just saying 'the Body of Christ is MAXIMALLY causally relevant, because it just IS the bread, the bread's true nature'... well, that doesn't even begin to address the worry, because it doesn't explain how the true nature differentially produces a change in the claimant's brain. The problem is the same whether you're talking about the 'inner nature' of a sensory object, v. the 'inner nature' of your own brain; somehow the actual pattern of neural firings in your brain has to causally interact with this inner nature in a way that causes the one to structurally resemble the other (like portions of my brain's visual centers structurally resemble the objects I'm looking at).

(I agree with you on the practical question. One's view on the hard problem of consciousness should inform one's view of animal welfare, but most such views agree that animals have a non-negligible chance of being moral patients. Eliminative physicalism denies that animals are conscious, but in a fashion that plausibly makes it •harder• to assert human exceptionalism.)
Edited1Aug 12, 2014

Vadim Kosoy

I feel that the debate surrounding the philosophy of consciousness is somewhat of a red herring in the context of the moral status of animals. It is obvious to me that I place value on the life and well-being of my cat. I get this knowledge directly from the source of value judgements in my brain and it is not conditional on any complex logical argument. Moreover it is rather clear that this value is directly related to the behaviours that my cat displays rather than solely its status as my pet. This makes me conclude that other minds in the smallest natural category including cats and humans have moral value. This conclusion already relies a lot on System 2 however the reasoning is much simpler and robust than the consciousness debate, making it a clear case of modus tollens. One can argue that my emotions towards my "cat" are a bug, a mechanism designed for detecting "consciousness" triggered by something else. However this would be isomorphic to claiming e.g. that non-reproductive sex has no value. I don't have the answers to all questions surrounding consciousness. I suspect people use this word to mean several different things and that some of the questions might be meaningless word games. It is possible that Eliezer's theory indeed cuts reality in the joints but if his notion of "consciousness" is not present in my cat then this notion of consciousness has no relevance to ethics or at least it is not a necessary condition for moral status. However if eating pigs is important for Eliezer's ability to work on AI safety, I prefer that Eliezer continues to eat pigs as the lesser evil by far.
Edited1Dec 17, 2015