On becoming a machine

A regular, TomA, asks: “If you could replace your organic body (in its entirety) with a machine, would you do it?”

This is one of those questions where examining the implied premises is the most interesting thing about answering it…

My shortest answer is “No, unless I were dying and it was the only way to escape mortality.” I have a strong hunch that being embodied as a human is required to understand the minds of other humans. Being posthuman might get pretty lonely. I’d also hate to give up eating and sex.

A slightly longer answer is that the question as posed neglects important issues about the capabilities of the machine. If I get to be an android with a fully human sensorium, that’s a very different and more acceptable case from being a mobile computeroid with tank treads and grippers.

An interesting counter-question is: “How am I not a machine already?” I’m not a vitalist. I regard my body as a machine that happens to use organic molecules and assemblies thereof as parts. This observation takes me back to the question of how much transforming me into a different kind of machine would alienate me from human experience.

I’m not asserting this is true of TomA, but I think people who ask this question often have a sort of clanking Robbie-the-Robot stereotype about what becoming a machine would be like. Well, it would beat dying, but please hurry up the upgrade with the syntheflesh and genitalia, would you?

173 thoughts on “On becoming a machine

  1. We hardly know yet how much of our “thinking” is done with the body parts outside of our brain.

    This is not metaphysical, I mean hormones, non-central nervous system, and using our other body parts do express thinking to ourselves. Your hands are the ultimate “digital computer”.

    Like, it is different if you see your words written down. It is also different if you feel what you make.

    Douglas Hofstadter and Daniel Dennett wrote about your thesis extensively in “The minds I”.

  2. Meh. The idea that “we are all cyborgs” has been around philosophical circles for years. We use fire (and other heat sources) to pre-digest food; clay, paper and now computers to record information, and to assist with memory, and to calculate, etc.; we use lenses to help our eye-sight (either to bring it to same “base” standard, or to extend it); etc. I’m sure you could think of a number of other examples around transport, communication, manufacturing, and so on.

    Wait, you might say. I’m not a cyborg! Well, you may not accept my definition, but still cyborgs walk amongst us. That person! (They have a pacemaker.) Them! (Hearing aids.) You get my point.

    Now, I don’t agree that our bodies are already machines more generally. I would reserve that term for artificial things. For things that we make. But when it comes to cyborgs… I wore glasses. I had to. (I then had laser surgery, something I would strongly suggest you look into if you have to wear glasses. Though it may not be a good idea in your case.) If I could have machinery attached so that I could perhaps run faster, or fly otherwise unaided, or a number of other things… Why I think I would probably take you up on such an offer.

    But to become solely machine? What happens to my brain? Are we talking uploading? Do I get to have multiple copies of me “running around”?

    I really think I would have to think about it. Luckily I’ve got plenty of time, because I can’t see such things happening any time soon.

  3. Robbie-the-Robot? More like Data. More to your point: I wrote a composition for college English class once where I asserted the digestive system is just a very sophisticated filter, allowing the nutrient components of foods into the body and letting the waste pass along.

  4. >Douglas Hofstadter and Daniel Dennett wrote about your thesis extensively in “The minds I”

    As I recall that book, it was less concerned with the question of whether human thought requires a human embodiment and more concerned with other issues in the philosophy of mind – one particularly entertaining one is where “you” are if your viewpoint becomes separated from the physical location of your body.

    My skepticism is less related to Dennet & Hofstadter than it is to Stevan Harnad’s continuing critique of the brain-in-a-jar model implicitly aimed at by most AI research.

  5. @esr
    I think Avatar (the movie) is close to the current consensus: You are where your senses tell you.

  6. Actually, the question occurred to me in regard to the blog post “Holding up the sky.”

    In a sense, open source hackers are becoming vitally essential to the functioning of society. In the pursuit of robustness of the system, might it not be in our interest to “mechanize” these key players and keep them around for a while? Perhaps there could be a virtual Olympus where this talent will remain in perpetuity.

    It seems to me that losing the hackers would be more serious than a routing congestion collapse.

  7. What are you but your body?

    What would it actually mean for you to replace your body in its entirety with an artificial machine?

  8. “please hurry up the upgrade with the syntheflesh and genitalia, would you?”

    If it ever got to that point, just think of the fun you could have at SCA events – just back up your brain contents to your smartphone, then go do battle with real swords! No need to play ‘Call of Duty’ on a computer, give that man a Garand and….

  9. If cogito ergo sum, where is the thinking being done? And no, I don’t necessarily mean where your brain is located. If all five senses are present somewhere else than where the brain is, where are you thinking?

  10. Assuming the “mind” is contained in the organ of the brain (which I don’t believe) how would the personality of the human being transferred to the robot skeletal composition? I would think with that presupposition unless someone can locate and identify the “mind” bit in the brain you could never depart with the organic brain and that in turn would limit the metamorphosis needed.

  11. As you say, it depends on the machine.

    At this point, I don’t trust the people who’d be working on the project to create something with the interesting possibilities of a body. I do T’ai Chi, and part of the fun is finding that the way moving my arms feels can be very much different and better than I thought it could be.

    In principle, it should be possible to invent machines which are more satisfying to live in than bodies.

  12. @Nancy Lebovitz

    more satisfying to live in than bodies.

    Again, I think this betrays an underlying and unstated assumption that there is something more to ‘you’ that your physical body. What would it mean to ‘live in’ another body, when everything that you are is constituted by your body. I think this whole conversation implies some sort of dualism that is unsupported by what little we know about how the brain works. We seem to be assuming that there is some sort of magical essence that is the ‘real you’ that can be taken out of your body and put into some other substrate.

    We know almost nothing about consciousness. We don’t really know what it is or what produces it. I regard this as one of the deep mysteries of human existence. I don’t think that we can assume that ‘in principle it should be possible’ to create a machine that we can ‘live in’, given how little we know about the subject.

  13. Answering this question tells a lot not only of assumed properties of the machine, but also of attitudes regarding death. Esr would prefer being uploaded to dying for an evidently wide selection of machines. Personally, I’d pass for most of the prospective machines presented in SF. Remembering being human and then having to live with vastly degraded capabilities doesn’t compare that favorably to death. That of course means that there are quite a few real life (hihi) survivable scenarios from which I’d prefer not to survive.

  14. two explorations of the subject (one shorter, one longer) in mainstream media that i’ve encountered in recent years:

    “rebuilt: my journey back to the hearing world,” by michael chorost, is a chronicle of the author learning to live with a cochlear implant and dealing with the implications of the associated technology after his frail hearing fails completely.

    the anime franchise “ghost in the shell” is essentially an extended meditation on the following question: if humans can transfer their consciousnesses into completely mechanical bodies and robots begin to approach sentience, what does that do to the line between man and machine? the two single-season TV series balance action and exposition somewhat better than the movies do, and they explore the topic in a more coherent way (IMHO).

  15. Much of what it feels like to be a conscious human has to do with internal, recalled sensory experiences, as distinct from those that you could glean from input devices. An article from a few days ago in the NYT said that

    scientists have come to realize in the last few years is that narratives activate many other parts of our brains as well [as "classical" language regions], suggesting why the experience of reading can feel so alive. Words like “lavender,” “cinnamon” and “soap,” for example, elicit a response not only from the language-processing areas of our brains, but also those devoted to dealing with smells.

    I’ll bet that this is why art works in general.

    I question whether you could have those recalled experiences apart from a body to have them in. Thinking about it another way, this is going to have to be quite a machine in order to duplicate how recollections feel.

  16. An interesting, related tangent to this is the specific process used in moving from meatware to hardware. The discussion so far seems to focus somewhat on straight uploading.

    But, what about a transition instead of a cut-over? I believe it was lobsters that some researchers tied into a hardware neural system and observed said lobsters’ neurons start firing with the artificial ones as though they were part of the same system. If one could create a brain-hardware link that allowed for the existing meatspace system to extend and flow into the artificial one, and if the artificial one is powerful enough, could you then remove the meatware and both a.) maintain continuity and b.) retain a sense of essential humanity?

    Not really a new thought, just adding it to the discussion.

  17. I don’t think that there is a fundamental difference between a mind residing in a human body and a mind residing in a body made out of plastic and metal.
    And if more than only our synapses are important for our mind then a upload will be harder but there is no philosophical difference.

    When you can upload your brain to a machine you could also fork it an run it on two machines. I think that makes some more interesting questions.
    Would you fork yourself to let one copy make a probably deadly activity?
    Would you do the probably deadly activity when you know that there is a fork of you that stays save?
    Would it be ethically to shutdown a fork of yourself if it behaved evil?
    And if you can fork you can probably also merge.
    Would you merge with a fork of yourself so that you can remember two situations which happened at the same time?
    Would you merge a fork which happened 10 years ago?
    Would you merge with someone you love?
    A stranger?

  18. @Emanuel Rylke

    I don’t think that there is a fundamental difference between a mind residing in a human body and a mind residing in a body made out of plastic and metal.

    What do you mean by a ‘mind residing in a body’. For that matter, what do you mean by a ‘mind’. Do you think the mind and the body are two separate things?

  19. We are creatures of our sensory perceptions, by and large, a total package if you will. Who we are as individuals indeed evolves from and are highly influenced by the senses through which we interpret our experiences. But the mind is a powerful thing (or can be anyway) and we have to ask ourselves how “real” are the things we perceive within the capability of our human perception. How “objective” is “objective” really?

    But the more we discover of ourselves and how we function, the more we really do seem like meat machines, organic components integrated to form a series functions.

    An interesting development – Parkinson’s patients have components implanted in their brain to simulate the lost functions caused by the disease. They can update the required software via the internet. I do not think it will be long before the expansion of this sort of toolkit finds increasing appeal WRT integration within human bodies. The ability to integrate the toolsets of my smartphone, store or access a library from within my skull, etc has great appeal, at least to me. The ability to enhance my vision and reflexes, and have my body self-diagnose also has great appeal. But I guess I’m really talking about enhanced humans rather than “replaced” humans here. The advancements in brain-scanning capabilities I think are what will drive a lot of this.

    This development will further our species down the path of philosophy, asking ourselves, what does it mean, really, to be human? The movie, Ghost in the Shell: Innocence takes the GitS universe further into a society where nearly all humans have been cyberized and at a high level attempts to identify pros and cons and ask some interesting questions here.

    It does seem likely that the first steps in serious cyberization would be to understand and replicate the human senses for integration into cyberized bodies – I do not believe that by and large we would simply become “machines” in the sense of tank treads and arm cannons given the trajectory and human inclination, but this is merely an opinion. We have a tendency to try to make everything “more human” from pets to computers to robots, it seems we would continue the trend with ourselves. I can’t imagine huge swaths of people willing to give up sex, hedonistic appetites, and what seems to be the fundamental need for human touch. But who knows where ultimately such an evolution would take us.

    Ray Kurzweil has hypothesized and written much about this. And while I do not share his optimistic, utopian view of what such a world will ultimately be like, he provides much information on the current state of developments, the idea of exponential technological advancements, and a fairly in-depth synth of many components required to come together to make possible such things (developments in computing, robotics, nanotech, medical, gene therapies, psychiatry, etc). it has been interesting to watch his thoughts evolve and see what has panned out, through his books “Spiritual Machines” to “The Singularity is Near” to his latest “Transcend.” I’m not a Singularitarian or whatever the label is, but that doesn’t negate the value of some of the info.

    One subject I frequently encounter in my thoughts on this, is “what does this mean for our individuality?”

  20. I apologize if a double-post appears from me. My post just disappeared on submit (rather than receiving a moderation message) – don’t know what that means.

  21. @tom I think the difference between mind and body is like the difference between a computer and the data that is on its hard drive or in its ram. So yes the data is not separate from the computer but i could copy it to a different computer. In the same way i think it is theoretically possible to copy a mind to a different body.

  22. @Emanuel Rylke

    So yes the data is not separate from the computer but i could copy it to a different computer. In the same way i think it is theoretically possible to copy a mind to a different body.

    Do you think that if you could scan a human body (including the central nervous system) down to the atomic scale, and record all the information you collected on the position of the atoms, what other atoms are interacting with them, what sort of atoms they are, etc and recorded all that information in a database on a computer, that the resulting database would be conscious and effectively be ‘you’?

  23. This discussion (especially the forking/merging bit above) reminds me a lot of “Kiln People” by David Brin, in which temporary, disposable artificial bodies become a part of everyday experience.

  24. @tom yes that would be me. But i think it would only be conscious when it was simulated and not only a saved snapshot. I think consciousness is a process and not a state.
    I think you could even make a copy of me that is more abstract and i would still call it me. Because i don’t actually care how the atoms that store me are specifically configured.

  25. @Emanuel Rylke

    yes that would be me.

    Really? Just a bunch of ones and zeroes sitting on a hard disk doing nothing would be you?

    It’s amazing how people’s intuitions differ on things like this.

    But i think it would only be conscious when it was simulated and not only a saved snapshot. I think consciousness is a process and not a state.

    So it sounds like you are saying that the essence of ‘you’ can be captured without any sort of conscious activity required?

    Is ‘simulating’ consciousness the same thing as consciousness?

    If that sounds weird, is simulating a hurricane the same thing as a hurricane?

    If you think that a computer can actually be conscious then does it not follow that consciousness can be reduced to computations? If so, would it be sufficient to have a large number of people slowly doing the right calculations on abacuses over thousands of years? Would this constitute consciousness?

    What if they stopped they calculations for a millennium, and then somebody found the calculations and restarted them? Would the conscious entity somehow ‘find’ the calculations again and continue the pattern without noticing the break?

  26. ObCaveJohnson:

    “The point is, if we can store music on a compact disc, why can’t we store a man’s intelligence and personality on one? So, I have the engineers figuring that one out now. Brain mapping, artificial intelligence, we should have been working on it 30 years ago. I will say this, and I’m gonna say it on tape so everyone hears it 100 times a day: if I die before you people can pour me into a computer, I want Caroline to run this place. Now she’ll argue, she’ll say she can’t – she’s modest like that. But you make her! Hell, put her in my computer, I don’t care. “

  27. If that sounds weird, is simulating a hurricane the same thing as a hurricane?

    It would be to the simulated consciousness experiencing the simulated hurricane!

    Put another way: If a simulated hurricane could somehow blow my house down the way a real one can, there is no practical difference.

    Similarly, if a simulation of consciousness can do the things real consciousnesses do — that is to say make decisions and act on the real world — from our perspective it’s just as much a consciousness. That was the entire point of the “Turing test”: consciousness is defined extensionally; its behavior is what makes it.

  28. >Remembering being human and then having to live with vastly degraded capabilities doesn’t compare that favorably to death.

    Take the long view. If your consciousness can be moved once, it can be done again. Even if your sensorium sucks, the thing to do is hold out for the next upgrade.

  29. @esr

    My mind and memories.

    But aren’t these things part of the body? If not, then where and what are they? Are they sort of ‘floating around’ near the brain?

    @Jeff Read

    Put another way: If a simulated hurricane could somehow blow my house down the way a real one can, there is no practical difference.

    But, this is the whole question. A simulated hurricane cannot blow down your house. We can simulate a hurricane (at least in principle) on the computer to a very high resolution, but this simulation is in no way a hurricane.

    Similarly it may well be possible to simulate the brain, but will that simulation actually *be* a brain?

    Similarly, if a simulation of consciousness can do the things real consciousnesses do

    This is question begging. The whole point of my questions is to get at the point that it may well not be possible to actually replicate the physical effects (including, presumably, consciousness) of a brain just by simulating it on a computer.

  30. @Jeff Read

    That was the entire point of the “Turing test”: consciousness is defined extensionally; its behavior is what makes it.

    That’s what *some* people think. But I think these people are copping out of dealing with the hard question. What I mean by ‘consciousness’ is basically phenomenological. That is to say, it’s the ‘what it is like’ to be something. That is not behavioural.

  31. @tom
    > If you think that a computer can actually be conscious then does it not follow that consciousness can be reduced to computations? If so, would it be sufficient to have a large number of people slowly doing the right calculations on abacuses over thousands of years? Would this constitute consciousness?
    yes. yes. yes.
    But it would suck if a couple million years goes by and you only have thought one sentence. Basically the world would change way faster than you can comprehend it.

  32. >But aren’t these things [minds and memory] part of the body? If not, then where and what are they? Are they sort of ‘floating around’ near the brain?

    Is a program identical with the arrangement of magnetic domains on a disk that records it?

    In fact, I do think human consciousness is entangled with the phenomenology of being in a human body. That doesn’t mean that I ‘am’ my body. To see this, try a succession of thought experiments:

    1. You step into the transmitter end of a classic SF teleporter, a machine that destructively scans your body and recreates the same arrangement of atoms a thousand miles away. A being with all your memories steps out of the receiver side. Is it you? If not, who else is it?

    2. You step into something you think is the transmitter end of a teleporter, but it’s actually an uploader. Moments later a pattern of information encoding the state of your body (including your brain state and your memories) appears in a simulation computer. Is it you? If not, who else is it?

  33. For the benefit of the discussion, there’s a theory of the “self” that is very interesting to me. Basically, it is stated that the sense of self that we have is tightly bound to a particular part of our brain and its interaction with the rest of our body. If so, then if we trick that part of the brain into interacting with another body as if it was our present body, our sense of self would be consistent.

    Someone might want to check it out here, particularly because it sheds light on the poorly understood topic of consciousness.

    http://www.ted.com/talks/antonio_damasio_the_quest_to_understand_consciousness.html

  34. Remembering being human and then having to live with vastly degraded capabilities doesn’t compare that favorably to death.

    Take the long view. If your consciousness can be moved once, it can be done again. Even if your sensorium sucks, the thing to do is hold out for the next upgrade.

    The first iteration had better be good enough that you can find a way to earn the upgrade. Otherwise they just turn you off when the money runs out.

  35. Based on what I pointed at just a minute ago, I think consciousness transfer requires brain transfer. Also, that brain should be hooked up with a body state simulator of some sort.

    Otherwise, if the brain state is transferred to a body state simulator, it would work.

  36. It seems there’s two different proposals floating around here– upgrading peripherals (replacing all the organs) or forgoing hardware acceleration entirely (the Matrix.)

    I’ve been playing video games long enough to know that software rendering sucks. I think I’d rather go the cyborg route.

  37. @esr

    1. You step into the transmitter end of a classic SF teleporter, a machine that destructively scans your body and recreates the same arrangement of atoms a thousand miles away. A being with all your memories steps out of the receiver side. Is it you? If not, who else is it?

    Intuitively, no. However, thinking about it rationally I have to conclude ‘yes’. There is no difference between the two entities.

    It’s interesting to think abut the slight variation where you don’t destroy the original.

    2. You step into something you think is the transmitter end of a teleporter, but it’s actually an uploader. Moments later a pattern of information encoding the state of your body (including your brain state and your memories) appears in a simulation computer. Is it you? If not, who else is it?

    I don’t know if it’s anybody. My whole point is that we don’t know that it is possible to create consciousness using a computer. A simulation of something isn’t the thing itself.

    It *might be* that consciousness can be reduced to mere computation, but when you really think about it that would be an awfully strange thing. Would it really be the case that if I did the calculations necessary for consciousness on an abacus, one operation per week, over thousands of years, that some sort of consciousness would arise?

    Maybe. But that would be pretty weird, and it’s not all obvious that such a thing is possible. We really know nothing about what creates consciousness, so right now we just cannot know the answer to your second scenario.

  38. Let me try to put this a slightly different way.

    Imagine that I had written down all the information that encodes the state of your body – at atomic or better resolution – in a huge set of tables in thousands and thousands of books. I also have a huge set of books that contains tabular data representing the atomic state of a ‘world’ that we say your body is ‘in’.

    Is there some process of calculations (representing the physical laws of the world) that I could perform (by hand) on this data, such that a conscious ‘you’ would somehow emerge from the process?

    The process would be something like going through each atom in my huge set of books and carefully applying a set of transformations to it to advance the whole system to the next instant in the ‘simulation’. The next instant would be a new set of books, to which we can apply the same transformations, again and again.

    Functionally this would be exactly the same as being uploaded into your computer simulation.

    Can this produce you?

  39. “Ray Kurzweil has hypothesized and written much about this.”

    Anyone know when the movie “The Singularity is Near” is finally coming out? The delays have been crazy. How can a movie be shown at film festivals in 2010 and still not be available in art houses even in the Bay Area? I haven’t had any luck finding out what the issue is.

    I’m very sympathetic to Singularitarian ideas, and I suppose I’m an optimist that we’ll eventually have a world where individuals how power like those in Vinge’s “Marooned in Realtime” or “The Gentle Seduction” by Marc Steigler. Whether I live to see it is depended on when life extension technologies are developed, as I don’t see it coming in a normal human lifetime.

  40. >It’s interesting to think abut the slight variation where you don’t destroy the original.

    Yes. I think the only reasonable conclusion is that the original and the copy are the same person, but cease being the same person as their memory histories diverge.

    >I don’t know if it’s anybody. My whole point is that we don’t know that it is possible to create consciousness using a computer. A simulation of something isn’t the thing itself.

    I think this is naive essentialism – you’re letting your intuitions run your thinking when you shouldn’t. To see why, suppose we upload you without destroying the original. I am now put into communication with both copy and original, but not told which is which. Both have your memories and your mental landscape. Both claim to be you.

    Now, possibly I can spot the uploaded version because there are defects in his simulated sensorium that he reports. But to sharpen the question, suppose I can’t do that because the simulation environment is really good. How do I decide which one is really you? If I can’t, which one is “real”? Am I warranted to conclude that either version is not really conscious? If so, how – what consequential test should I apply?

    We’re not done. How do you know that I am not such a simulation? All you have of me is words on a browser display. You form a theory of mind from them, but you have no way to know, in principle, that I am the meatware ESR rather than an upload made last week and running on some mad scientist’s hardware.

    And we’re not done yet. Suppose you are in fact such a simulation? You might be and not know it. Some people semi-seriously propose that the Planck-scale granularity of the universe is a clue that we are all inside a gigantic discrete simulation.

    In order to decide that any being you can communicate with and form a theory of mind about is not conscious, you need a behavioral test that can distinguish between “consciousness” and “mere computation” – and you need one robust enough to cope with the possibility that the being could be either lying or deceived about the nature of its own substrate.

    (Yes, I’ve thought about this a lot. :-))

  41. @Cathy

    Anyone know when the movie “The Singularity is Near” is finally coming out?

    I saw it in London at the Science Museum, and Kurzweil was there and gave a talk. It’s a very interesting film, and you get to see a lot more about his childhood and how a lot of his hopes for immortality and brain uploading are motivated by a desire to bring his dead father back to life. He has a huge archive of materials about his father’s life that he believes he will one day be able to use to simulate his brain.

    I am sympathetic to a lot of Kurzweil’s arguments, and I regard myself as a ‘singularitarian’. I think his ideas about the exponential nature of technological advance are sound. However, I get off the train when it comes to brain uploading and strong computer-based conscious AI. I don’t say that these things are definitely impossible, but I think (for reasons I discuss above in other posts) that they are far from a certainty.

    I think it will be possible to create artificial life forms that are conscious, but my feeling is that they won’t be computer based.

  42. @esr

    Yes. I think the only reasonable conclusion is that the original and the copy are the same person, but cease being the same person as their memory histories diverge.

    That is EXACTLY what I think as well. I think that the instant the copy ‘wakes up’ and experiences something different from the original then they are no longer the same person. Of course, if you have killed the original then there is some sense in which you can see the copy as a seamless continuation of the original pattern.

    Both have your memories and your mental landscape. Both claim to be you.

    You’re assuming that such a thing is possible, and part of what I am claiming is that it might not be.

    My earlier example of the hurricane is relevant here. We can put into the computer a lot of numbers that describe the exact physical state of the hurricane at time t0, and we can then perform calculations on them that will work out what the hurricane will look like at time t1. And we can do that again and again really fast. But the wind doesn’t blow as a result! Just because we can describe something in great detail on a computer and predict its next state, it doesn’t mean that all the physical effects of the real-life thing will be created by the simulation.

    We might be able to store the exact state of a brain in a computer, and update all the numbers that constitute that representation to its next state (given a set of laws), but that doesn’t mean that the simulation will be conscious.

    The next question is ‘is it possible to behaviourally mimic human behaviour without consciousness?’

    Maybe. Maybe not. But you’ve got to ask yourself, from an evolutionary perspective why did we develop consciousness? It seems doubtful that something so seemingly complex and resource hungry would be an accident. So it also seems likely that if we could have developed our complex behaviour without consciousness then we would have.

    So, if a computer can’t be conscious (and I don’t think it’s clear that consciousness supervenes on computation) and human behaviour requires consciousness then it might not be possible for a computer to mimic human behaviour.

    There are a lot of ifs and maybes there. The fact is that we don’t know what consciousness is, how it is produced, or why we have it (evolutionarily), so I don’t come to any firm conclusions. But I don’t think it is at all certain that brain uploading is possible.

    How do I decide which one is really you? If I can’t, which one is “real”? Am I warranted to conclude that either version is not really conscious? If so, how – what consequential test should I apply?

    Right. This is the old behavioural argument that basically all that matters is behaviour, and if I can’t distinguish between two entities based on behaviour then they are the same. But I reject that. Just because you cannot tell the difference between the two entities it doesn’t mean that their internal experiences are the same. I accept that we might have no choice – pragmatically – other than to assume that an entity that acts as if it is conscious is in fact so. However, that doesn’t mean that the entity is in fact conscious.

    We’re not done. How do you know that I am not such a simulation? All you have of me is words on a browser display. You form a theory of mind from them, but you have no way to know, in principle, that I am the meatware ESR rather than an upload made last week and running on some mad scientist’s hardware.

    That’s true. It’s possible given our current state of ignorance, that such a thing could exist. But all I am saying is that we really don’t know the answer to that yet.

    Some people semi-seriously propose that the Planck-scale granularity of the universe is a clue that we are all inside a gigantic discrete simulation.

    And I take such a possibility seriously. Maybe the universe is all just mathematics. Maybe. But, again, we don't know.

    In order to decide that any being you can communicate with and form a theory of mind about is not conscious, you need a behavioral test that can distinguish between “consciousness” and “mere computation” – and you need one robust enough to cope with the possibility that the being could be either lying or deceived about the nature of its own substrate.

    Are you going down the Wittgensteinian route here?

  43. Some people semi-seriously propose that the Planck-scale granularity of the universe is a clue that we are all inside a gigantic discrete simulation.

    Also, this gives rise to one of my favourite things to say to non-programmers, which is that if there is a God then He is a programmer.

    My only wish is that he is not a C++ programmer.

  44. > My only wish is that he is not a C++ programmer.

    Probably more like a deranged sysadmin. The real bastard operator. “Didn’t follow the rules? userdel!” See also: make world

  45. “Some people semi-seriously propose that the Planck-scale granularity of the universe is a clue that we are all inside a gigantic discrete simulation.”

    …and there are millions of Hindus who propose that the delusion of the world is merely a dream of Vishnu as he slumbers on his bed of cobras. They had the idea long before….

    Does it really matter? The world seems real enough to us. If you could prove that it’s all a giant simulation, how would your life change? Would you try to sabotage the program in protest? I have to say, that if life is just a program, I’m pretty much enjoying it, and I’m not looking forward to my deletion.

  46. Indeed, if you accept a few premises then it follows that we almost certainly are living in such a simulation.

    The premises are something like:

    1. It is possible for consciousness to emerge through computer simulation.
    2. It is possible to create a computer simulation of sufficiently high resolution.
    3. For any possible simulated or non-simulated universe it is more likely than not that more than one simulated reality will be created by its inhabitants.
    4. The number of non-simulated universes is static.

    I’m typing this off the top of my head. I had it more precisely figured out a few years ago, but the upshot is that given a world consistent with these premises any given universe is more likely than not to be simulated. In fact it might be FAR more likely to be simulated.

  47. >The next question is ‘is it possible to behaviourally mimic human behaviour without consciousness?

    In modern philosophy of mind, “philosophical zombie” is the term used for a being that mimics human behavior without consciousness. I think the term originated with Daniel Dennett – it’s in his style, anyway.

    You are asking whether philosophical zombies exist. What I am trying to show you is that that question is unanswerable without a decision procedure that tells you whether the beings you actually observe, including me, are actually philosophical zombies. Nor is there is anything special about “is a philosophical zombie”; any predicate is meaningless without an analogous decision procedure. The meaning is the decision procedure.

    So. Tell me what your decision procedure for “is a philosophical zombie” is. Or equivalently, tell me what your decision procedure for “is conscious” is. If you can’t specify one, then both terms are only noises devoid of content and you should avoid using them.

  48. @esr

    I think the term originated with Daniel Dennett – it’s in his style, anyway.

    I thought it was Chalmers, but I’m not sure.

    You are asking whether philosophical zombies exist.

    Actually I’m just saying that it is maybe impossible for them to exist.

    Nor is there is anything special about “is a philosophical zombie”; any predicate is meaningless without an analogous decision procedure.

    I think that ‘is conscious’ actually is unique, because it is the only predicate I know of that we all have direct personal subjective experience of but cannot formulate a scientific test for (yet, anyway).

  49. >I had it more precisely figured out a few years ago, but the upshot is that given a world consistent with these premises any given universe is more likely than not to be simulated. In fact it might be FAR more likely to be simulated.

    Sorry to have to be the one to tell you this, but this is a well-known argument in some circles. I believe the fully-developed form is normally associated with Nick Bostrom, though it’s pretty clear that he was riffing off some ideas proposed much earlier by Edward Fredkin and Stanislaw Ulam.

    The general reaction among people who have thought about this (including me) is that Bostrom’s argument is logically sound, but uninteresting because you can’t draw observable consequences from it.

  50. @esr

    Sorry to have to be the one to tell you this, but this is a well-known argument in some circles.

    Oh, believe me, I was under no illusion that I was the first one to think of it. It’s a pretty natural and obvious line of thought once you start thinking about the possibility of living in an artificial universe.

    It’s just a curiosity really.

  51. Assuming we are in one, how to hack the sim?

    Is science merely exploratory hacking, and engineering ai run amok?

    To what end?

    God always seems to be constructed in the image of the familiar, powerful and mysterious of any given age.

  52. Setting aside for the moment the nuts & bolts of morphing your original organic self into another format, there is an aspect to this subject that is more fundamental.

    Eric and others are a key support system for modern technology centered on internet data transfer, telecommunications, GPS, and other fundamental information systems upon which society has become dependent. Lots of people, institutions, and businesses are working to improve the robustness of the underlying hardware and software.

    Is the hacker support system robust? Can we assume that new hackers will always arrive on scene to replace the old ones as they die out? If you answer yes, is that a logical certainty or a leap of faith?

  53. @TomA

    Is the hacker support system robust? Can we assume that new hackers will always arrive on scene to replace the old ones as they die out? If you answer yes, is that a logical certainty or a leap of faith?

    I wouldn’t worry about it. Every generation grows up, makes its contributions, dies, and then is replaced by the next generation. We’ve never had much trouble replacing talent before.

  54. Feeling like you have senses and motion is probably mandatory for mental health. Sensory deprivation and paralysis do *weird* things to your perception.But I could definitely imagine a machine with augmented physical capacities that still felt like I was moving “my body.” And I’m not even sure “feeling like you have a body” requires physical contiguity — what if I were a flock or a swarm, and I could move all my parts and still feel “embodied” in them? There’s a lot of options out there, and I’d probably like to try some of them.

    The more interesting question, I think, is which is a harder problem: creating a mechanical body with sufficiently advanced movement capabilities for you to “live” in, or creating a digital simulation of motion and sense perception. I’m inclined to think the robot body is an easier problem…it takes fewer resources to move a machine than to make a digital simulation complete enough to be mistaken for reality.

  55. @Tom: What if we could “save” the state of the brain and then “write it” into a previously grown brain + body (a la “The 6th Day”, iirc)?

    What if the state could be evolved in a simulation, then written out to said brain + body? Could you say that the lapse of time in which the state of mind evolved without a physical body+brain was not spent in consciousness?

  56. >Is the hacker support system robust?

    You’re on fire, dude! This is worth another blog post.

  57. @iajrz

    What if we could “save” the state of the brain and then “write it” into a previously grown brain + body

    I doubt that it would be possible, because the state of the brain is deeply wrapped up in the actual physical configuration of the brain. What makes you *you* and not somebody else is the physical form of the brain. So to say you can ‘save’ that and write it onto another brain doesn’t really make sense.

    People like to think of the brain as a computer, but I don’t think it’s really that way at all. You can’t just read off data the way you can from a hard drive and dump it onto another empty hard drive and have a nice neat copy.

    People’s brains are different sizes, have different internal connections, different numbers and layouts of synapses and have a million other analog features that are widely varying between individuals. You can’t just export that to disk and dump it into another brain.

  58. As no definition of machine was given then I’d be very happy to be uploaded into a spaceship. Time would be irrelevant, just go into sleep mode for 1000 yrs and wake up in a new solar system like it was a new day. In your face FTL drives!

    In all seriousness, integrating sensory input might have complications. What if you upgraded your eye to perceive the complete electromagnetic spectrum, not just visible light? Our program (brain) is just not designed for that kind of input. And if you did add a patch to your program to support the new input, it would be kinda trippy for a while. Imagine looking, literally, through the eyes of a baby.

    I dont see why sex would be a problem. If I can get uploaded, so can the opposite sex. My program would still have all the rules for love. Sex would be different. But children? Merging two programs to generate a third? That might not be possible. Would reproduction become asexual, where you copy your brain, wipe all memories and upload into another machine? An evolutionary deadend.

  59. “People’s brains are different sizes, have different internal connections, different numbers and layouts of synapses and have a million other analog features that are widely varying between individuals. You can’t just export that to disk and dump it into another brain.”

    There’s another aspect of that. When a baby is born, it has a surplus of neurons, and little control over its body. As it gains experience, various neural circuits are reinforced, others die off. The early brain is programmed to program itself, adapting itself to the body. I think that it’s most likely that that ability is largely lost once the process is complete. Just as adults can never pick up a language the way small children do, you might really have a hard time adapting to a new body.

  60. @LS
    I think when we are capable to upload a mind, we will be capable to change it in a way that it can adapt to a new body.

    @tom
    I don’t think consciousness is more special than our ability to learn. My pet hypothesis is, that consciousness is our ability to model others, applied to ourself.

  61. @Tom Not in the “read and write from disk” sense of the expression; you could, theoretically, store the information of the state of the brain, including connections, chemical balance, and foldings in the brain.

    And about writing: the brain is naturally “chemistried” into it’s configuration. What would stop us from getting a “virgin” brain and apply chemicals and electricity till we’ve got it into the shape we want?

    If we managed to do that, what then? And why wouldn’t we be able to nurture a brain into a specific configuration?

  62. @iajrz: You’re on the right track, but you haven’t gone far enough. ‘We’ are not just the state of our brains. Our nervous system extends to all parts of our bodies. You would have to duplicate *all* of it, which means that you would actually have to clone the body entirely.

  63. > God wrote in Lisp. Didn’t you know?

    So it’s parentheses all the way down, eh? Makes sense.

  64. >but uninteresting because you can’t draw observable consequences from it.

    It depends on how good the simulation is. Watching for flaws, inconsistencies in the simulation, could tell us that we are in a simulation, though I can’t see any way to show that we aren’t.

  65. “Watching for flaws, inconsistencies in the simulation…”

    Some time ago, the “Is it all a simulation?” question was debated on John Tierney’s old science blog. One of the comments: “Kudos on Jennifer Beals!”

  66. @Emanule Rylke

    I don’t think consciousness is more special than our ability to learn. My pet hypothesis is, that consciousness is our ability to model others, applied to ourself.

    But that doesn’t explain the most mysterious thing about consciousness, which is the ‘what it is like’ phenomenological, inner subjective experience of having a perspective and viewpoint on the world.

    The question is, how can something like this arise from non-conscious objective physical stuff? It’s deeply weird.

  67. @William B. Swift

    It depends on how good the simulation is. Watching for flaws, inconsistencies in the simulation, could tell us that we are in a simulation, though I can’t see any way to show that we aren’t.

    Not really, because how would you know what was a flaw or an inconsistency? If we found something inconsistent with our current understanding of universe we would ascribe it to a flaw in our theories, not a flaw in the universe. There would be no way to tell which was which.

  68. >The question is, how can something like this arise from non-conscious objective physical stuff? It’s deeply weird.

    Um..why? That’s like saying “electrons are weird”. At best it’s just a statement about the prejudices wired into your nervous system, at worst just noise. Electrons aren’t weird, consciousness isn’t weird, they just are; “weird” is something you added.

  69. @esr

    Um..why? That’s like saying “electrons are weird”. At best it’s just a statement about the prejudices wired into your nervous system, at worst just noise. Electrons aren’t weird, consciousness isn’t weird, they just are; “weird” is something you added.

    Because there’s a huge explanatory gap. Why do particular brain states (particular arragenments of matter and energy) produce certain phenomenal experiences and not other phenomenal experiences, or no experiences at all?

    It’s hard to understand how subjective experiences can be determined by brain states. Why do we suddenly go from a lump of matter and energy with no inner subjective experience to something with an awareness and a point of view. Subjectivity is something completely different from everything else we see in the universe.

    There is just no way to explain consciousness in terms of the science we know today.

  70. William B. Swift says: If we found something inconsistent with our current understanding of universe we would ascribe it to a flaw in our theories, not a flaw in the universe. There would be no way to tell which was which.

    Every experimental test so far has shown that General Relativity makes accurate predictions. Likewise for Quantum Mechanics, yet, they are inconsistent with each other. Relativity is a local theory, while QM is not. Tell all those string theorists that they are probably wasting their time chasing a program design flaw.

  71. >Why do particular brain states (particular arragenments of matter and energy) produce certain phenomenal experiences and not other phenomenal experiences, or no experiences at all?

    I think this is is a very silly question. Your “phenomenal experience” is the operating noise of your cognitive machinery. If you had different cognitive machinery, it would make different noises. Whatever operating noises the cognitive machinery of any being (human, robot, alien, hamster) made, that would be its “phenomenal experience”. You’re mystifying something that’s actually trivial.

  72. I would agree with what ESR is saying. In fact I would go further to
    explicitly say that consiousness is not a state, but a process.

    I see no theoretical reason that my ‘mind’ couldn’t be uploaded into a
    ‘computer’ and my personality and memories go on from there. And the
    computer version would be ‘me’. This presumes a sufficently exact
    upload and an appropriate platform to run ‘me’ on. The engineering
    details are left as an excercise for the student.

    Also, if my body could be duplicated in a sufficently small amount of
    time, with an exact (or at least sufficently close) duplication of all
    parts of my body as it exists now, that duplicate would be ‘me’.

    If the duplication were destructive, it would be ‘me’ off over there,
    if it were a non destructive duplication, ‘I’ would exist in two
    places. As ESR said, the two of ‘me’ would become different
    individuals right away as our new experinces diverged.

    As a simpler version of the issue, water bears and some nematodes can
    be dried out, then frozen at liquid nitrogen temperatures, on down to
    liquid helium temps, then stored in a vacuum for a long time.

    At this point, while I might not label them ‘dead’, it’s hard to make
    a case that they are alive.

    However, you can then warm them up, sog them up, and a significat
    fraction will go wiggling off to do their things. The survival
    percentage is signifiantly greater for those that got frozen hard and
    stored in the vacuum, than if they were just dried out for the same
    amount of time, at least if the time is fairly long.

    Again, at the moment, there are a few picky little engineering
    details to be worked out to make this work for humans — but again, I
    see no theoretical reason why it can’t be done. And if so, it looks
    like ‘I’ am a process and a pattern rather than there being anything
    sacred about this piece of meat that currently runs ‘me’.

    Like ESR, if in a relative few years, if there is a method of
    uploading ‘me’ into a new platform — assuming a reasonable cost and
    acceptable capabilities of the new platform, I will take it. If
    nothing else, I can always ‘die’ later. Why not give it a shot? It
    might work. (I don’t expect it to be possible in the twenty year
    window that will probably be my limit, but who knows. Things have
    certainly changed in the last twenty and the twenty before that.)

  73. @esr

    How does calling it ‘operating noise’ explain anything? We all have a subjective ‘inner life’ that we call consciousness. You can call it ‘operating noise’ if you want to, but most people use the term ‘consciousness’. If you want to use ‘operating noise’, then let me rephrase the question: ‘how does operating noise arise from certain arrangements of physical matter? And why do these certain arrangements produce these certain sorts of operating noise and why do other arrangements produce other sorts of operating noise, and why do still other arrangements produce no apparent operating noise at all?’

    The weird thing is that consciousness exists only as experienced by some animal ‘I’ creature. Everything else in the universe is just there, it has an objective ontology. But consciousness (sorry, ‘operating noise’) uniquely has a subjective first person ontology. It only exists as experienced by a person. Everything else exists because it is just there. There is a fundamental difference of category. One category of things (objective matter) somehow (apparently) gives rise to the other category (subjective experience). The question is how? and the answer so far is that we have no idea at all. That is the mystery.

    There is nothing in our physical theory of the universe (so far) that explains why some things are conscious and some things are not. And yet consciousness is something we all have direct experience of every moment of our waking existence. In a way it is the most important fact about our lives, and yet we have no way of integrating it into our model of the world.

    I think you’re just trying to dismiss the problem as ‘operating noise’ because you don’t have the tools to grapple with it. Well, that’s what makes it such an interesting problem!

  74. @Jim Hurlburt

    Do the revived nematodes exhibit behaviors conditioned before they were dried and frozen?

  75. @Jim Hurlbert

    This presumes a sufficently exact
    upload and an appropriate platform to run ‘me’ on. The engineering
    details are left as an excercise for the student.

    The ‘engineering details’ and your presumption of an ‘appropriate platform’ are the whole question!

    The point of what I was saying is that we just don’t know whether everything the brain does supervenes on computation.

    Until somebody can answer that question we cannot know whether it will be possible to run your brain in all of its technicolour phenomenological glory on a computer.

  76. >how does operating noise arise from certain arrangements of physical matter?

    You can ask that question about truck engines, too, and it’s not interesting there, either. It has an answer, but the answer is trivial – it doesn’t tell you anything about what makes the engine go or what it’s for.

    >The weird thing is that consciousness exists only as experienced by some animal ‘I’ creature.

    How do you know that? How do you know that rocks aren’t conscious? See “decision procedure, importance of”.

    You’re just floundering – mystifying yourself by tossing around predicates for which you have no tests.

  77. It’s hard to understand how subjective experiences can be determined by brain states. Why do we suddenly go from a lump of matter and energy with no inner subjective experience to something with an awareness and a point of view. Subjectivity is something completely different from everything else we see in the universe.

    That’s vaguely the intuition most creationists have: it’s hard to see how something very, very complicated (us) can come from something very, very simple (amino acids, or whatever started it all). So they leap to the conclusion that “us” and “them” are in entirely different categories, and that you can’t get from there to here.

    The intuition is false for creationists. Why shouldn’t it be false for dualists?

  78. @Desmond

    it’s hard to see how something very, very complicated (us) can come from something very, very simple (amino acids, or whatever started it all).

    It *is* hard to see how that can be so. And that was a deep mystery at one time. But we have developed a very beautiful and full explanation for it: evolution.

    We have no such explanation for consciousness, which is why it is still a mystery.

    That’s not to say that we won’t one day come up with a scientific explanation, but right now we don’t have it.

    @esr

    You can ask that question about truck engines, too, and it’s not interesting there, either. It has an answer, but the answer is trivial

    How do you know that? How do you know that rocks aren’t conscious? See “decision procedure, importance of”.

    It might well be so that truck engines and rocks are conscious. And some people take that seriously and contend that one part of the explanation of consciousness is that *everything* in the universe is conscious on some level; that consciousness is a fundamental property of matter, just like mass.

    Why some things produce one sort of consciousness and others produce other kinds is still unknown.

  79. @esr

    You’re just floundering – mystifying yourself by tossing around predicates for which you have no tests.

    Is it your contention that there is no such thing as consciousness?

  80. “It might well be so that truck engines and rocks are kerflobble. And some people take that seriously and contend that one part of the explanation of kerflobbleness is that *everything* in the universe is kerflobble on some level; that kerflobbleness is a fundamental property of matter, just like mass.”

    This is exactly as meaningful as what you said. In both cases, the grammatical subject of the sentence is a predicate which you cannot connect to any evaluation procedure. The only difference is that you fool yourself about the word “conscious” in a way you don’t fool yourself about the word “kerflobble”, because you’re attached to ungrounded intuitions about it.

  81. @esr

    This is exactly as meaningful as what you said. In both cases, the grammatical subject of the sentence is a predicate which you cannot connect to any evaluation procedure.

    Yes, this is what makes it a hard problem to grapple with. But unless you actually deny that you are conscious, and that consciousness exists then there is still a problem to be solved.

    Do you actually deny these things?

    The unique thing about consciousness is that it’s the one predicate that we can’t develop a test for but that we all have experience of. We all know that *we* have the property of consciousness. We all know that we each have an inner life and a subjective viewpoint. We know that there is ‘something it is like’ to be us. And yet we have no way of evaluating that property in other things because by its very nature consciousness has privileged access. Only the entity subjectively experiencing the consciousness has access to it, because consciousness is subjective experience.

  82. Well, I would be surprised if a nematode were trainable — they’re down at the size level where they’re hard to see with the naked eye. Beyond that, I doubt anyone has asked the question before. It’d be an interesting question to investigate, if nothing else, can they learn at all.

  83. And to add a comment to the nematode, to me the difference between me and a nematode is entirely degree, not kind. Do I believe that a nematode is in any sense ‘inteligent’ or ‘self aware’? No On the other hand, I see no clear dividing line between the nematode, an insect, which can learn at least a little, a domestic turkey (which is about as dumb as anything made of meat), a dog, which clearly thinks and can learn, dolphins and chimps which seem to be possibly sentient, to me — I manage sentient and self aware at least occasionally.

  84. @Jim Hurlburt

    Perhaps they should try scaling the process up to flatworms instead. There’s some evidence flatworms can be conditioned.

    The idea being to verify that aquired neurological changes survive the experience.

  85. >But unless you actually deny that you are conscious, and that consciousness exists then there is still a problem to be solved.

    From what Eric’s been saying, I don’t think he’ll deny that, as far as his own definition of “consciousness” is concerned, but I think it’s fairly likely that he’ll maintain that it is unknown (and that he can neither confirm nor deny) whether he is conscious as far as *your* definition of consciousness is concerned.

    How close am I Eric?

  86. >How close am I Eric?

    Pretty spot-on. My objection to Tom’s way of talking is not that I think “consciousness” is necessarily a meaningless term. but that he is mystifying it and himself by tossing it around without having in mind a meaning – that is, an experiment that can be performed to distinguish conscious entities from non-conscious ones.

    In fact my test is simple. An entity is “conscious” if it shows behavioral evidence of introspection. That is, it has a world-model which it treats as distinct from its phenomenal field, and it compares the two as a response to surprise. We see such evidence not only in humans but in various other mammals – not reptiles or older classes, though.

  87. @esr

    Let us be clear. Are you saying that you do not have a subjective experience of the world? That you do not have an inner life that is not accessible to others but only to yourself? That there is nothing it is like to be you?

  88. >Are you saying that you do not have a subjective experience of the world?

    I show behavioral evidence of introspection.

  89. “We have no such explanation for consciousness, which is why it is still a mystery.”

    Yes we do, and it’s the same one. Evolution. Organisms that do not take positive steps to preserve themselve until they reproduce simply die and remove their genes from the pool. Conciousness is self-preservation writ large.

  90. @LS

    Yes we do, and it’s the same one. Evolution.

    That’s like saying that evolution explains how flight works. Birds developed flight because it has survival value, but that doesn’t explain what the mechanism is that actually makes flight happen.

  91. >You’re evading the question. That’s not what I asked.

    You’re being dense. What else do you think your “subjective experience” looks like from the outside?

    I’m talking the way I am so as not to encourage you to baffle yourself with semantic spooks.

  92. @esr

    What else do you think your “subjective experience” looks like from the outside?

    That’s my whole point. I’m not talking about what it looks like from the outside. I’m talking about what it’s like from the inside.

    Indeed, my point is that the very fact that there is an ‘inside’ is mysterious.

    All I am asking you to admit is that you have an inside.

    I assume you do, but you seem strangely reluctant to say so.

  93. >I assume you do, but you seem strangely reluctant to say so

    Because the moment we start speaking in those terms I expect you will start babbling meaningless noises. Sigh…

  94. @esr

    Because the moment we start speaking in those terms I expect you will start babbling meaningless noises. Sigh…

    Well, you can deny it to yourself if you want to, but I know that I, at least, have an inner life. When I look up at the blue sky my brain doesn’t just take in visual data, process it, and produce behaviour. There is something it is like to be me seeing that sky. I have a subjective experience of that blueness that nobody else can have access to. My brain doesn’t just process the blueness data, I have a sensation of blueness.

    I understand why you are reluctant to deal with this, because science is all about being objective. It’s hard to be objective about something so inherently subjective. Science looks at the world from the outside. We look at a person as a body with a brain that produces behaviour. That’s why you like to define consciousness in terms of behaviour. Because it’s nice and neat that way, and you can integrate that into what science already knows about the world.

    But I think if you go down that route you are missing the biggest question about consciousness. And perhaps one of the biggest questions about being human, which is the inner subjective experience of life.

    Well, I don’t want to avoid that issue, because it’s too interesting. We all know we have a subjective life, so I want to know how we can explain these subjective experiences in terms of the objective processes that are known to science.

  95. @Tom: Think about how, once self-awareness appeared, it became possible to make survival a result of plans that the individual made, rather than just reactions to stimuli that happened to work towards survival. This is powerful survival tool, and, once started (possibly by some random brain mutations), natural selection would have honed it to a fine edge. Ideas that led to individual survival might have laid the foundation for the concept of ‘beauty’, later extended to art appreciation, etc. Other primates seem to have self-awareness, concepts of fairness, etc., so we are not unique in this. Conciousness is just another brain function, selected by the Blind Watchmaker.

  96. @LS

    Think about how, once self-awareness appeared, it became possible to make survival a result of plans that the individual made, rather than just reactions to stimuli that happened to work towards survival. This is powerful survival tool, and, once started (possibly by some random brain mutations), natural selection would have honed it to a fine edge. Ideas that led to individual survival might have laid the foundation for the concept of ‘beauty’, later extended to art appreciation, etc. Other primates seem to have self-awareness, concepts of fairness, etc., so we are not unique in this. Conciousness is just another brain function, selected by the Blind Watchmaker.

    I have no disagreement with the idea that consciousness has survival value and that it was selected by evolution. That’s all perfectly fine.

    My point is that we have no idea at all how the brain creates consciousness. How is it that some particular brain states give rise to subjective experience? This is the part we have no explanation for.

    Back in the days before we understood the mechanics of flight, we could still look at birds and conclude that they have flight because of the survival advantage it confers. But we still didn’t have any idea of how lift is generated and what the exact mechanism was that allowed birds to fly.

    See the difference?

  97. @Tom & esr
    “We look at a person as a body with a brain that produces behavior. That’s why you like to define consciousness in terms of behavior.”

    In the USA, you will most likely have been influenced by Behaviorism. Behaviorism was a failed attempt to build behavioral sciences on pure empiricism. Behaviorism cannot cope with consciousness.

    There is a whole different branch of ethology (the study of animal behavior), which replaced behaviorism. The now current consensus in ethology assumes that animals have internal states, feelings and emotions, which will affect their behavior, but themselves are not observable. Some might even have consciousness. The famous “mirror test” has shown that the big apes, whales and dolphins, elephants, and just maybe, a European Magpie, might have a level of self-awareness. (I see European Magpies daily around my house, they are amazing birds)

    In hacker terms, the fact that I cannot observe the states of a computer, just it’s outputs, does not mean I cannot refer to such states in my model.

    Science can study human feelings, happiness, and consciousness objectively. They can be related to behavior, self reports, hormone and neural activity, evolution, etc.

    So, when putting Eric’s head into a jar, we can connect his sense organs to appropriate sensors. But we can also make sure that the relevant levels of Adrenaline, Oxytocin, and other hormones will circulate as a kind of extra-neural computation. The same for oxygen, CO2, and immune reaction. These are necessary because we know that they are part of Eric’s emotional system without which he will not be Eric anymore. In the end, we would probably much better off keeping his body intact.

    I am pretty sure the experience will change Eric in fundamental ways anyway.

  98. >I am pretty sure the experience will change Eric in fundamental ways anyway.

    I agree. That’s why I have indicated skepticism about whether human consciousness is separable from human embodiment.

    I’m not a behaviorist; I have no issue with the claim that animals have an inner life. But I’m also not a mysterian – I accept only observable behaviors of the animal (or human) as evidence of that inner life.

    I think human intuitions about consciousness are even more unreliable than our intuitions about physics, which is why I’ve been giving Tom a hard time about ungrounded language. The domains where we instinctively fool ourselves the most are precisely those where we need to be most careful about insisting on sharp operational definitions and tossing out terms or statements that we cannot unpack to those.

  99. @esr
    “I’m not a behaviorist; I have no issue with the claim that animals have an inner life. But I’m also not a mysterian – I accept only observable behaviors of the animal (or human) as evidence of that inner life.”

    My point exactly.

    In hacking terms, living things compute. To compute, you need inner states. To say something sensible about those states, you need the “source” and the “design” (not distinct in living things). All these are material and physical objects (or patterns, if you think everything is information).

  100. >My point is that we have no idea at all how the brain creates consciousness. How is it that some particular brain states give rise to subjective experience? This is the part we have no explanation for.

    Personally, I think consciousness is a side-effect of any sufficiently rich information processing system. Someday soon we will probably have the means to test this and see.

  101. @LS: William B. Swift says: If we found something inconsistent with our current understanding of universe we would ascribe it to a flaw in our theories, not a flaw in the universe. There would be no way to tell which was which.

    >Every experimental test so far has shown that General Relativity makes accurate predictions. Likewise for Quantum Mechanics, yet, they are inconsistent with each other. Relativity is a local theory, while QM is not. Tell all those string theorists that they are probably wasting their time chasing a program design flaw.

    1. I didn’t write that. That was someone else responding to what I did write.

    2. GR and QM are not inconsistent in the logical sense (which is what I meant); they provide different levels and different formats of description, which physicists are busy trying to reconcile. If they were “inconsistent” laws, there wouldn’t be much hope for the physicists’ success.

  102. This is an issue which science fiction has discussed for more than a century — but almost always in the wrong way. We SF writers have treated cyborgization as a metaphor rather than a real possibility. The “joining of man and machine” is always treated as a big philosophical issue, a major change from how we live, and a big symbol of whatever you need a big symbol for.

    And it’s all bullshit. Humans have been cyborgs since the first australopithecus picked up a rock. Cooking food changed the way we eat and digest food — so much so that it’s very difficult for a modern human to live on a raw food diet. Our teeth and nails got small and fragile because our bodies became targeting and delivery systems for rocks and sticks. Even the most “primitive” societies are steeped in technology. Humans can no more live without machines (even if they’re simple ones) than we can live without oxygen.

    As cyborgization moves from metaphor to reality it gets less problematic. My wife uses an insulin pump because her pancreas doesn’t do the job any more. She has a MACHINE sticking to her with a NEEDLE EMBEDDED IN HER FLESH, pumping CHEMICALS made in FACTORIES into her veins! In a 1930s Weird Tales story she would obviously be the villain, a creepy, barely-human monstrosity. In the real world of 2012 an insulin pump is about as remarkable as dentures.

    So, Eric, if it’s ever possible for you to inhabit a mechanical body, you’ll do it. And neither you nor anyone else will think it’s particularly strange. Will it change the way you think? Probably — just as writing and mechanical clocks change the way you think. Will it change who you are? You aren’t the same person you were when you started reading this comment. Of course new experiences will change you.

    Will it make you less human? Nope.

  103. I somewhat like the criterion “shows behavioral evidence of introspection” to the extent it bears a component of objectivity. However, I still don’t like that it could be faked…. or could it?

    We’re getting closer and closer to writing programs that could pass a Turing test in the text-only division. Suppose we actually get there. A human poses a question to the program that it cannot answer without showing behavioral evidence of introspection, e.g. “do you think you have a soul?”. It answers. It has now passed the consciousness test – but it’s just a fairly advanced algorithm, probably running on something no more powerful than a desktop computer. (Paging Dr. Searle… Dr. Searle, please report to the Chinese room…)

    So what’s wrong with this scenario?

    1: We’ll never quite accomplish writing a program that passes the Turing test.
    2: This is insufficient behavioral evidence of introspection; I’d need to chat longer.
    3: This is insufficient behavioral evidence of introspection; it requires audio and/or visual interaction as well.
    4: This criterion isn’t quite sufficient to prove consciousness.
    5: There’s nothing wrong. That running program is conscious, good sir.
    6: Something else I overlooked.

    Which one is it?

  104. >Which one is it?

    Probably 2 or 5. But then I don’t think consciousness is either complicated or ontologically special, so opting for 5 isn’t a big deal for me.

  105. any predicate is meaningless without an analogous decision procedure. The meaning is the decision procedure.

    Or, as James Blish’s Spock put it in “Spock Must Die!”:

    “A difference which makes no difference, is no difference.”

  106. >“A difference which makes no difference, is no difference.”

    This quote is originally due to the psychologist and philosopher William James, deployed in describing the philosophy of C. S. Peirce. When I wrote “Any predicate is meaningless without an analogous decision procedure. The meaning is the decision procedure.” I was consciously recasting Peirce in slightly more modern language. So these two maxims are closer together than you knew.

  107. Does anybody have an experiment that can distinguish between the following hypotheses?

    1) All human beings are conscious.

    2) 75% of human beings are conscious. The other 25% have two copies of the recessive zombie gene, mimicking consciousness well enough to fool other humans without actually possessing it.

    3) All humans of your sex are conscious, all those of the opposite sex are zombies.

    4) All human beings except you are zombies; you’re the only conscious being in the universe. What you consider signs of consciousness actually have no connection to consciousness.

    No? You don’t have such a test?

    Then you either need to redefine what you mean by “consciousness”, or admit that debating if a program that passes the Turning Test is “conscious” is as fully pointless as debating if it has been endowed by God with a soul.

  108. I wrote: “I was consciously recasting Peirce in slightly more modern language.”

    Sadly, Peirce was like Algol – a considerable improvement over most of his successors.

  109. I don’t think consciousness is ontologically special

    The question that immediately came to my mind when reading this is whether you claim the existence of natural rights a la Ronald Dworkin, or if you think there’s no such thing, a la Jeremy Bentham. I’m not 100% sure whether that’s a relevant question, but I’m still trying to fully grok that stuff, and I have to hit the road here in a minute, so I’ll toss it out here for now.

    I’m going in the direction of asking whether there’s any conclusions one can draw about the rights inherent in any entity due solely to its being conscious. The concept of a right is itself open to debate in this context. Nevertheless, this is the sort of utterance that I see people typically drawn to at this point. Also, are there any interesting properties ascribable to a consciousness other than rights? Presuming for now the definition of “shows behavioral evidence of introspection”?

  110. >The question that immediately came to my mind when reading this is whether you claim the existence of natural rights a la Ronald Dworkin, or if you think there’s no such thing, a la Jeremy Bentham.

    That’s complicated.

    But first, I want to note that your question implies a belief that taking consciousness off the ontological pedestal a lot of philosophers put it on has strong implications for moral philosophy. I don’t think that’s true; it’s a variant of the “If God is dead, anything is permissible” mistake. Consciousness does not have to be any kind of special to make conscious beings (potential) participants in an ethical web of trust.

    I don’t think philosophical zombies can exist – I think the concept relies on incoherent and necessarily false premises, my critique being essentially Daniel Dennett’s with verificationism as a second punch. But if they could, they too would be potential participants.

    To address your question, I don’t believe there are any a-priori natural rights – I’m a conseqentialist. But I think the language of natural rights becomes appropriate very early in the development of any sound political philosophy because there are some rules that are simply too consequentially dangerous to ever violate. I am willing to assert, for example, natural rights to free speech and the bearing of arms.

    This is actually parallel to my positions on ontological realism and virtue ethics. I’m not an ontological realist – having premises about ‘reality’ before you have a confirmation theory to check them with is just nuts. But if you don’t get to where you can recognize objective reality as a feature of your system fairly early, you’re doing it wrong. I don’t think “virtue” has any kind of prior existence either, but virtue language serves the same important function in personal philosophy as natural rights language does in political philosophy.

  111. @esr

    But first, I want to note that your question implies a belief that taking consciousness off the ontological pedestal a lot of philosophers put it on has strong implications for moral philosophy.

    I interpreted what he said as implying that natural rights fit into the category of a ‘difference that makes no difference’.

    And I would agree.

  112. @esr

    But I think the language of natural rights becomes appropriate very early in the development of any sound political philosophy because there are some rules that are simply too consequentially dangerous to ever violate

    Why natural rights and not civil rights?

  113. >Why natural rights and not civil rights?

    I prefer the term “natural rights” to “civil rights” because it is less susceptible to the interpretation that these rights are a gift of society or government that can be withdrawn. The whole point of natural-rights theory, whether in its traditional deontological version or my consequentialist one, is that they’re not revocable.

  114. @esr

    I prefer the term “natural rights” to “civil rights” because it is less susceptible to the interpretation that these rights are a gift of society or government that can be withdrawn.

    I understand that from a propagandistic or communications perspective. Perhaps you’d like people to think that way. But, in reality, is it not actually the case that rights *are* granted by governments. The only thing that gives meaning to rights is that they are enforced by something (usually a government). Without something backing them up, rights are meaningless.

    The term ‘natural rights’ to me implies rights that are not man-made, that are somehow inherent in the universe, or perhaps given by a god.

    Obviously such rights do not exist. Indeed, the very term is meaningless (unless you assume some sort of natural or divine enforcement).

    So, although you might want to use ‘natural rights’ because it makes people think that governments cannot take the rights away, in reality, if we are honest, governments can take them away. I think it is more accurate to use the term civil rights, or legal rights.

  115. >No? You don’t have such a test? Then [...] debating if a program that passes the Turning Test is “conscious” is as fully pointless as debating if it has been endowed by God with a soul.

    Excellent. It was as a result of conducting thought experiments like this that I became a hard-shell verificationist – the term I’ve previously used is “brutal operationalist” but the distinction between verificationism and operationalism is accidental and historical, not essential (as is the distinction between those two schools and C. S. Peirce’s predictivism).

    Is this one original with you? If so, applause – it’s a clearer and more elegant argument than most philosophers can manage. If you’re quoting or paraphrasing, I’d like to know your source.

  116. The phrasing’s original with me, I believe. (I will not swear I did not ever read something similar in the deep past, merely that I don’t recall such and was not consciously quoting or paraphrasing.)

    There’s a version of the thought experiment itself I posted on a LessWrong thread last year, but I left the conclusion implied there.

  117. @esr:
    “I don’t think philosophical zombies can exist – I think the concept relies on incoherent and necessarily false premises, my critique being essentially Daniel Dennett’s with verificationism as a second punch. But if they could, they too would be potential participants.”

    I’m curious to hear your thoughts on the theme of the novel “Blindsight” by Peter Watts. He isn’t arguing that humans are non-sentient, but that intelligent aliens are normally non-sentient, and humans are a fluke. (I personally think he weakened his point by introducing non-sentient vampires, but that’s just my own opinion. They are too close to humans in their behavior, which makes hash of the core thesis.)

    If humans are necessarily sentient — not philosophical zombies — can we imagine an intelligent alien species that is not? How would it behave differently than one that is sentient?

    Certainly that book has made me think more about what sentience, or consciousness if you will, really is…though without coming up with satisfying answers.

  118. >I’m curious to hear your thoughts on the theme of the novel “Blindsight” by Peter Watts.

    Note that the “Blindsight” aliens are not philosophical zombies – they can’t emulate introspective consciousness. So this question is actually going somewhere else. But it’s an interesting somewhere.

    SF author Karl Schroeder, a friend of Watts’s, turned me on to Blindsight, and when I told him by email that I found the novel gripping and thought-provoking but the premise implausible he told me he agreed and brought Watts in on the conversation.

    My objection was that introspective, narratizing consciousness as we find it in humans is likely to be common rather than rare because it’s actually adaptive. “Consciousness is a hack for anticipating the behavior of competitors, especially in the presence of deception.” is how I put it.

    Hm, I just found my email to Karl on the topic. Here is some of it:

    OK, I’m not certain I know [what consciousness is good for]. But there
    is a family of answers, all consistent with the biological and
    behavioral evidence, that have been well explored since at least
    Julian Jaynes’s “The Origin of Consciousness in the Breakdown of the
    Bicameral Mind” in 1976. Consciousness is a hack for anticipating the
    behavior of competitors, especially in the presence of deception.

    Self-awareness, in this account, a necessary consequence of developing
    the machinery to solve the other-minds problem — so you can
    outmaneuver them. My own elaboration on this, which owed something to
    William Calvin’s “The Ascent of Mind” (1990) but was well formed long
    before “Blindsight”, is that human self-awareness developed from
    social tactical modeling but has all the hallmarks of the sort of
    gaudy overelaboration you see in peacock’s tails, and developed from a
    similar sort of intra-species arms race. The tie to sexual selection
    may be just as close.

    The consciousness arms race in hominids (if that’s what happened) had
    a lot of phylogenetic precedent to build on. The range of animals
    that exhibit experimental signs of self-awareness (that is, of knowing
    that their representation of reality differs from reality and
    including themselves *in* that representation) is actually quite
    broad. My own personal experiences back up the folklore that includes
    dolphins and elephants in that range. Heck, even my *cat* has
    occasional flashes of self-aware behavior, though they’re dim and
    infrequent compared to (say) an elephant’s. Animal ethologists
    confirm that you get this sort of thing clear down to the level of
    rabbits and rats. And I know of a parrot who apologized to its owner
    after beaking her when startled — by saying “I love you”.

    The amount of phyletic spread on behaviors indicating at least
    rudimentary solution of the other-minds problem makes me strongly doubt
    Watts’s bleak picture of a universe in which self-awareness is
    exceptional. I’d bet on it being common in what he calls “residents”,
    and social animals in general.

    So, the right question to answer Watts’s with is: can you solve the
    other-minds problem without being aware of your own? I’m inclined to
    think not. But even if hyper-zimbos like the Rorsach intelligences
    *are* competitive with fully self-aware sophonts, that doesn’t make
    them a likely evolutionary outcome. It may be that it’s simply easier
    (lower Kolmgorov complexity) to genetically describe a self-aware
    being than a zimbo with equivalent tactical-modeling capability.

    And you both need to read Paul J. McAuley’s “400 Billion Stars”.
    Maybe Peter has; I remember that you hadn’t when we discussed
    “Blindsight”.

    Later in the exchange I developed a more general explanation of introspective consciousness based on the cognitive requirements for rehearsal of learned skills. I’ll probably clean that up and blog it.

    Watts half-agreed with me but held out for the proposition that introspective consciousness might be just a local adaptive peak, not a global one.

  119. >But I think the language of natural rights becomes appropriate very early in the development of any sound political philosophy because there are some rules that are simply too consequentially dangerous to ever violate.

    But how do you define consequential danger? How do you define bad consequences? You can certainly derive the best means of achieving a given goal, and label actions whose consequences are counter to that goal “bad” or “dangerous”, but what’s to stop us from choosing a set of goals such that freedom of speech and the right to bear arms hinder the achievement of our chosen goals and are therefore “bad”? What’s to say (for example) that spreading suffering isn’t better than spreading pleasure? Or that the fusion/fission of as many atomic nuclei as possible into nickel-62 isn’t better than the establishment of any human emotional state?

  120. >But how do you define consequential danger?

    The same way Sam Harris proposes in The Moral Landscape; the minimization of total suffering over time. He does a pretty good job of rescuing this from the usual anti-utilitarian objections.

    In political philosophy, my touchstone is preventing the conditions that produce genocide. I became an anarchist when I concluded that genocide is the natural destination of the logic of state collectivism. See Why I Am An Anarchist for discussion.

  121. “”1. You step into the transmitter end of a classic SF teleporter, a machine that destructively scans your body and recreates the same arrangement of atoms a thousand miles away. A being with all your memories steps out of the receiver side. Is it you? If not, who else is it?”

    An even more interesting version of this (to me) is to eliminate the word “destructively” from the scenario, so that there are two (initially) identical copies of “you”. Do they have equal claim to being you? Does the locational discontinuity of the duplicate weaken its claim to be the “real you”? If so, what does this imply about the same locational discontinuity that would occur in the destructive scan case?

  122. Oops, look like this variation was already discussed. The hazards of responding to a post before reading the subsequent posts.

  123. @Tom

    … But, in reality, is it not actually the case that rights *are* granted by governments. The only thing that gives meaning to rights is that they are enforced by something (usually a government). Without something backing them up, rights are meaningless. …

    I think you have it backward. Humans give up some of their liberty in order to associate with other human beings. “Your right to swing your fist ends at my nose.”

    We own or rights naturally. We can give up some of them or have them taken from us. Otherwise we are in the unenviable position of being grateful for the scraps thrown to us by our betters.

    You do have a point about people needing a way to enforce their rights. Time to practice our archery.

  124. @Bob

    We own or rights naturally. We can give up some of them or have them taken from us.

    What does it *mean* to have a natural right? How could you tell whether a person had natural right x or natural right y?

    Does a person in Iran have a right to free speech? What would the difference be if that person did not have that right?

    Do you have a right to free speech? Do you have a right to drive a car?

    If your answer to those last two questions is ‘yes’ and ‘no’ then you need to explain why that is the case.

    How can you tell what natural rights you have and which ones you don’t have?

    To me a natural right is a difference without a difference.

  125. I haven’t read Blindsight, but you might find the aliens in Jeffrey Kooistra’s novel, Dykstra’s War, interesting. The aliens are not sapient in a human sense, instead they are an incredibly ancient species that has evolved their technology and instincts for working in space colonies and everything else over billions of years. (Note that otherwise the novel isn’t very good science fiction, about on Star Trek level science.)

  126. @esr:
    “Note that the ‘Blindsight’ aliens are not philosophical zombies – they can’t emulate introspective consciousness. So this question is actually going somewhere else. But it’s an interesting somewhere.”

    I agree, which is precisely what makes them more interesting. I share your belief that no experimental test can be applied to distinguish philosophical zombies from sentients, which makes them uninteresting to me. But non-sentient intelligent creatures at least a logically self-consistent construct that would clearly behave differently from sentients, although it might be difficult to construct a plausible evolutionary path that would lead to their development.

    Good stuff. I don’t think I could be having this conversation anywhere off the Net, as my friends and acquaintances wouldn’t have any interest in this stuff and might not even get the concepts. They certainly haven’t read “Blindsight”. The Net is a powerful way to let low-incidence affinity groups locate each other and exchange ideas with minimal friction costs…though it still doesn’t quite erase my regret that I don’t get to have these conversations face-to-face with other humans at my physical location. Ah, well.

  127. Tom: The core of natural rights, and what makes then natural as opposed to civil, is that they are rights you can exercise in a world where no other human being exists.

    Obviously, that needs a lot of systematic elaboration when extended to a multi-person reality, which is not particularly suited to presentation in a blog comment. But that’s okay, people have been elaborating on the basic philosophical premise for more than three hundred years, so it’d be pretty easy for you to find full elaborations if you’re actually interested.

  128. @Steven Ehrbar

    The core of natural rights, and what makes then natural as opposed to civil, is that they are rights you can exercise in a world where no other human being exists.

    How can it make any sense to have a right in a world where nobody else exists?

    What is a right other than a rule that says nobody is allowed to either take a particular thing away from you, or a rule that says society must provide some particular thing to you?

    Rights are inherently about how other people can treat you.

  129. What is a right other than a rule that says nobody is allowed to either take a particular thing away from you, or a rule that says society must provide some particular thing to you?

    The whole point of natural rights theory is to answer that question. If you’re actually interested, again, I urge you to seek out the last 350 years of writing on the concept. If you’re not actually interested, that’s fine, but you’re not going to convince anyone who believes in natural rights by merely asserting they can’t exist.

  130. @Steven

    you’re not going to convince anyone who believes in natural rights by merely asserting they can’t exist.

    I’m not merely asserting that they can’t exist. I’m asking simply for a definition of what it means for somebody to have some particular natural right. In other words, what test can we apply to a person to find out whether they have natural right x?

  131. What does it *mean* to have a natural right? How could you tell whether a person had natural right x or natural right y?

    Best I can tell, there are multiple theories for natural rights, including some that claim they don’t exist.

    If I try to apply consequentialism, I get this result: an entity E has a “natural right” to a privilege P to you, if you perceive a benefit in not trying to block E from exercising P. In other words, the consequences to you of not interrupting E’s exercise of P are favorable.

    For example, if you see a dog sitting in a yard, and you see benefit to yourself in not going over to kill it, it has a right to life. You don’t have to go out of your way to feed it to recognize that right; you just have to not want to deprive it of the privilege of living that it is currently exercising. On the other hand, suppose a dog (maybe even the same dog) is attacking you, and you see benefit to yourself in killing it; in that case, the dog has no right to life (by you). Then again, if you decide it would still be worse for you to kill the dog, but better to simply cuff it and then restrain it, then the dog has a right to life, but no right to free action.

    The trouble with this is that it is subjective – rights are relative to the perceiver – and we’re conditioned to thinking rights are objective. I don’t see either as necessarily more correct in this context. The subjective model seems more elegant to me, though.

    The objective model could be constructed on top of this by claiming that someone may see benefit in not blocking E from exercising P, nor anything in some ontological class of which E is a member. For example, I see benefit in not blocking any dog from living; therefore, all dogs have a right to live by my standards. If every member of a group of people sees this benefit, then all dogs have a right to live by the group. It gets complicated in just the ways one might expect: we may retract that right from one dog attacking someone; one of our group may simply stop seeing the benefit of not blocking that privilege; and so on.

    What Americans often think of as rights are actually duties imposed on everyone – everyone enters into a contract not only to behave as if there is a benefit in not blocking a privilege, but to act to block anyone not behaving that way. For example, we agree to act to block someone from blocking a dog’s living privilege – if we see someone trying to kill a dog, we intercede directly, or we gather resources to enable intercession (get a gun or call the police), or we report it after the fact to block future blocks.

    I wonder if this could cover all of my current intuitions about rights. Or all readers here, for that matter.

  132. Meanwhile, from ESR’s email:
    Self-awareness, in this account, a necessary consequence of developing the machinery to solve the other-minds problem — so you can outmaneuver them.

    Yikes. This is ominously similar to something I’ve thought for a while – power deriving from the ability to step into another’s shoes. You’ve phrased an aggressive, pure-competitive conclusion of this.

    It makes me wonder if this could be harnessed, in a sense similar to how capitalism harnesses competitiveness to create goods and services. Take a population of animals that have individually demonstrated, say, passing the mirror test, and devise environments in which they are rewarded for being able to predict and anticipate what their cohorts do. (Or anyone, really – if they can anticipate what the researcher doing, that also works.) Breed the ones that are best at this. Record any other signs of intelligence, but don’t worry about them past that. See what develops.

    You’d probably want to do this with as many species as affordable, focusing on those with a short time between birth and giving birth in order to see any effects quickly. This might indicate whether introspection is purely learned, or can be inherited genetically (I suspect a combination of both).

    If this is what happened with homo sapiens, depending on the numbers I plug in, I guess a result of 1500-10000 generations needed to see recognizable advancement of the trait; five times that to see a specimen routinely carving stone tools. Probably less given how focused the breeding would be.

    One could carry out a similar experiment for humans themselves, employing the Howard model. One problem I see there is that everyone passes the mirror test; we’d need a better test. Who are the best humans at identifying with other humans? Poker players? Actors?

  133. what test can we apply to a person to find out whether they have natural right x?

    In natural rights theory, all you need to do to test if a person has natural right x is determine if they’re a person; if they are, they do. It’s inherent and automatic. In a world with only one person, there would be no need to define, enumerate, or enforce natural rights, because they would be impossible to infringe, but they would still exist, because the person who exists could still exercise them.

    When you say “rights are inherently about how other people can treat you”, you are invoking a definition of “rights” that inherently denies the possibility that natural rights can exist. In natural rights theory, rights are inherently about how you may act, what you have the right to do.

  134. @Steven Ehrbar

    In natural rights theory, all you need to do to test if a person has natural right x is determine if they’re a person; if they are, they do. It’s inherent and automatic.

    But that makes the term completely meaningless. This parallels in some ways our earlier discussion about consciousness. You are asserting the existence of a predicate that has no meaning. To use Eric’s earlier strategy:

    In kerfloffle theory, all you need to do to test if a person has kerfloffle is determine if they’re a person; if they are, they do. It’s inherent and automatic.

    Has exactly as much meaning as what you said: none.

  135. But that makes the term completely meaningless.

    No, not any more than any other “ought” statement is completely meaningless.

    Natural rights theory is a theory of morality (as applied to politics), not a statement about what counts as people. It is an easily demonstrated fact of the universe that one cannot derive an “is” from an “ought”; no matter how hard one thinks Mercury should orbit Jupiter, no matter what laws are passed on the matter, there is no effect. Since a right in natural rights theory is an “ought” about people, it cannot be used to determine whether someone is a person, and it cannot be found in the physical world except insofar as personhood can be found in the physical world.

    Now, it is a reasonably common interpretation that natural rights theory attempts the reverse, to derive “ought” from “is”. And quite common among philosophers is the opinion (originally stated by Hume) that deriving “ought” from “is” can’t be done. Thus, Jeremy Bentham, who held both positions, declared that natural rights were “nonsense on stilts”. However, the belief that derivation of “ought” from “is” can’t be done is not itself an easily demonstrated fact of the universe, but merely a common intuition about what, well, ought to be.

  136. I guess what it all boils down to is that if rights were ‘natural’, there would be no need to enforce them, and no need for all those ‘bills’ and ‘declarations’.

  137. @Steven

    Natural rights theory is a theory of morality

    So, it sounds to me as if you are claiming that to say ‘People have a natural right to free speech’ has the meaning ‘People ought to be free to say whatever they want’.

    Which is all well and fine, but I don’t think that couching this in terms of ‘rights’ language is helpful at all. In fact it is obscurantist. You’re saying that people ‘have’ something that doesn’t map on to anything real. You’re perverting language.

  138. @LS

    if rights were ‘natural’, there would be no need to enforce them, and no need for all those ‘bills’ and ‘declarations’.

    Quite so.

    Talk of ‘natural rights’ obscures that fact that bills and declarations are necessary for rights to actually exist. They lead people to say things like ‘such and such a right cannot be taken away from me’. But actually, in any real and meaningful sense, they can be taken away from you.

  139. Talk of ‘natural rights’ obscures that fact that bills and declarations are necessary for rights to actually exist. They lead people to say things like ‘such and such a right cannot be taken away from me’. But actually, in any real and meaningful sense, they can be taken away from you.

    I keep reading slightly related text about rights, duties, and other legal concepts in the course of my job, while not studying it specifically. Even so, it’s possible that I understand the basic idea behind rights and just don’t know the terms. I *think* my conceptualization of rights follows consequentialist lines, like Eric, and probably like Bentham, but I’d have to read further to know for sure. See my comment above (search for “best I can tell”) for the gist of it. Again, I could probably use it to model any scenario put forth in this thread so far.

    I could even use it to express a monotheist’s version, or any version relying on an ultimate authority (be it a god, a king, or a written agreement such as the US Constitution – anything John Austin would refer to as a sovereign). Essentially, any individual’s assessment of the consequence of blocking the exercise of privilege by another with the right to that privilege, is factoring in reprisal due to the existence of the sovereign. (E.g., if you believe someone has a God-given right, your blocking that right might get you sent to hell.)

  140. Which is all well and fine, but I don’t think that couching this in terms of ‘rights’ language is helpful at all. In fact it is obscurantist. You’re saying that people ‘have’ something that doesn’t map on to anything real. You’re perverting language.

    Are you arguing that I am not using the word “right ” to mean “A legal, equitable, or moral title or claim to the possession of property or authority, the enjoyment of privileges or immunities, etc”, a manner in which the word has been used for more than a thousand years (according to the OED)? Or are you objecting to “natural”, which similarly has been used for six hundred years to mean “in political and legal philosophy and theology, doctrines based on the theory that there are certain unchanging laws which pertain to man’s nature, which can be discovered by reason, and to which man-made laws should conform”?

    You can reasonably hold that a “natural right” is a term without a nonfictional referent if you like, in the same manner as “leprechaun” or “pegasus”. But it’s ridiculous for you to claim it’s a perversion of language or obscurantist to use a term according to centuries-established meaning, simply because you were ignorant of it.

  141. @Steven

    are you objecting to “natural”, which similarly has been used for six hundred years to mean “in political and legal philosophy and theology, doctrines based on the theory that there are certain unchanging laws which pertain to man’s nature, which can be discovered by reason, and to which man-made laws should conform”?

    it’s ridiculous for you to claim it’s a perversion of language or obscurantist to use a term according to centuries-established meaning, simply because you were ignorant of it

    I am aware of the tradition around the use of ‘right’. I happen to be a philosophy graduate and I am well acquainted with the “theory that there are certain unchanging laws which pertain to man’s nature, which can be discovered by reason, and to which man-made laws should conform”. That theory is wrong.

    I object to the use of ‘right’ to mean a moral wish because it is confusing. To say that there are natural rights is ambiguous, because people are led to infer the existence of rights enforced and inherent in nature in the same way that there exist rights enforced and codified in law. This is false.

    When you say ‘there exists natural right such-and-such’ you are just saying that there ought to be a civil right such-and-such. If that is what you think, then say so plainly.

  142. @Paul Brinkley

    if you believe someone has a God-given right, your blocking that right might get you sent to hell.

    Right. If there were actually some non-artificial (either divine or natural) structure that codified and enforced rights, then we would have a basis for the existence of natural rights. However, lacking any evidence for such a thing we must conclude that the only real rights are those codified and enforced by law.

  143. I object to the use of ‘right’ to mean a moral wish because it is confusing.

    Oh, so you’re just being that most ridiculous of creatures, a linguistic prescriptivist who objects to long-established educated usage. Sorry I wasted my time: English is what it is, not what you think it ought to be. I’ll know to skip your comments in the future.

  144. So, anyone else here has read MINDSCAN, by Robert J. Sawyer? The whole novel is exactly about this scenario.

  145. Tom, you’re being a bit thick.

    ‘Talk of ‘private property’ obscures that fact that bills and declarations are necessary for property to actually exist. They lead people to say things like ‘my property cannot be taken away from me’. But actually, in any real and meaningful sense, it can be taken away from you.’

    Your rights are your first property. You are BORN with them and you learn to exercise them as part of maturation. They can be stolen from you, just as your property can be, but that doesn’t mean they don’t exist or that their theft is moral. You can choose to give them to others (either as a Power of Attorney, or by voting for a representative who will exercise YOUR rights to democracy on YOUR behave) but as with property, voluntary surrender or delegation is the only MORAL way that this can happen.

    And by viewing natural rights this way, it becomes obvious what the government CANNOT do. E.g. it cannot kill somebody without due process because that is the only situation in which we as citizens have agreed (ahead of time) that our rights may be taken from us, and you and I cannot vote for Eric to have the power to kill Winter. We cannot morally give Eric any right on our behalf that we do not ourselves posses, just as we can’t sell Eric Winter’s car.

    By believing that rights are a construct of our social contract, you surrender morality to popular vote and create a guaranteed tyranny of majority situation.

    (With apologies to Winter ;^).

  146. @Don

    Your rights are your first property. You are BORN with them and you learn to exercise them as part of maturation. They can be stolen from you, just as your property can be, but that doesn’t mean they don’t exist or that their theft is moral.

    If you think that natural rights can be ‘stolen’ then I assume you disagree with Steven’s interpretation? His interpretation seems to be that to say ‘so-and-so has a natural right to free speech’ means something like ‘so-and-so ought not to be prevented from saying whatever he likes’. It is impossible to ‘take away’ a right in that sense, since all the right consists in is an assertion that a person ought to be able to do a thing (or be free from a thing). Even if you prevent so-and-so from speaking freely, he still ought to be able to speak freely. So, his ‘right’ (in Steven’s sense) has not been stolen or taken away.

    You seem to be asserting a realist conception of natural rights. If you think that natural rights exist in the world (and can be ‘owned’ or ‘stolen’) then you need to provide an account of the meaning of the term ‘natural right’ in such a way as to provide a method for distinguishing entities with natural right such-and-such from entities without that right.

  147. If you think that natural rights exist in the world (and can be ‘owned’ or ‘stolen’) then you need to provide an account of the meaning of the term ‘natural right’ in such a way as to provide a method for distinguishing entities with natural right such-and-such from entities without that right.

    No, I don’t. A natural right is any right that you as a sentient being can assert for yourself and defend. If you have one, you have them all. Your rights end only where another’s rights begin.

    I’m not sure I can give a more satisfying answer than that. I’m far from a scholar (or even a student) of this subject and my grasp is courtesy of the knowledge shared by others.

    What I do understand is that you’re requesting a positivist definition of natural rights. There is no such animal, because positivism says rights only exist when instantiated and codified into law. Natural law says that those rights are yours regardless of what some legislative law or code says. The Declaration of Independence (which is the first law of the United States of America) says that when positive laws conflict with natural laws, it is the duty of the citizens to defend their natural rights. That is the justification for the Revolutionary War.

    I would recommend reading some of Andrew Napolitano’s writings on the subject.

    Here’s a quick link of Napolitano speaking on the subject to the Mises Institute.

  148. @Don

    A natural right is any right that you as a sentient being can assert for yourself and defend.

    The Declaration of Independence (which is the first law of the United States of America) says that when positive laws conflict with natural laws, it is the duty of the citizens to defend their natural rights.

    I don’t think you can be talking about the same sort of natural rights as the Declaration of Independence, because the declaration says that natural rights are inalienable, but your conception allows for the possibility that rights can be stolen, and that you only have those rights that you can assert and defend for yourself.

    If natural rights are real then they must make some difference in the world. What difference would there be if all the natural rights in the world suddenly vanished?

  149. @Don

    http://youtu.be/GXVG2oaJ1c0

    Well, the guy in this video says that natural rights come from our humanity. What that means is unclear, and he doesn’t explain.

    He gives the example of British soldiers writing their own search warrants and breaking into American houses to conduct searches (presumably as an example of a natural right being impinged?). But on your own account of natural rights, the British had the right to search those houses because, as you claim:

    A natural right is any right that you as a sentient being can assert for yourself and defend

    The British were able to assert and defend their right to search American houses, so on your account they had the right to do so. The Americans in those houses were unable to defend their right not to be searched unfairly, and so on your account they did not have such a right.

  150. Uh, you have a selective reading problem there Tom. Here let me help:

    No, I don’t. A natural right is any right that you as a sentient being can assert for yourself and defend. If you have one, you have them all. Your rights end only where another’s rights begin.

    There, that’s better.

    And I think the Americans DID assert their rights (in the Declaration) and defend them (by sending the Red Coats packing).

    I’m sorry if I was not clear.

  151. @Don

    Uh, you have a selective reading problem there Tom. Here let me help:

    No, I don’t. A natural right is any right that you as a sentient being can assert for yourself and defend. If you have one, you have them all. Your rights end only where another’s rights begin.

    I don’t know that this helps. You say that ‘your rights end only where another’s rights begin’. In the case of the British soldier invading an American house, whose right ends? The British soldier’s right to search the house or the American’s right to not have their house searched? Without a specific set of enumerated rights how do you decide whose right takes precedence? ‘Your rights end only where another’s begin’ doesn’t really mean anything.

    The fact that America eventually won the war of independence is irrelevant to the individual case of the American household being invaded. Those people were unable to defend their right, so on your account they had no right.

    Your model of natural rights is essentially the idea of “might is right”. Whoever can force their will on another person is in the right. If the victim is unable to defend themselves, then they have no right.

    I am sure this is not what you want to say.

  152. @Don

    And I think the Americans DID assert their rights (in the Declaration) and defend them (by sending the Red Coats packing).

    You have almost made my point for me here, by the way. Yes, after the US made the declaration, won independence and the ability to enforce their laws, and set up their constitution, then American citizens had some rights. Specific, real, civil rights; the only sort worth the name.

  153. @Tom

    I’m sorry it has taken me so long to reply.

    All animals, including humans, instinctively defend their property.

    Do you have a dog? He regards the food you put into his bowl as his property. He will defend it from everyone of status equal to or lower than his. He really doesn’t like it when someone of higher status takes it away from him.

    Do you have two dogs? You probably give them separate bowls when you feed them for this very reason.

    I repeatedly cautioned my daughter not to bother our dogs while they were eating. She didn’t get it. Then my mother’s dog came to live with us. When she got too close while he was eating he nipped her. Then she understood.

  154. We did not win our rights in the American Revolution. We won recognition of them.

    Just as my mother’s dog won recognition of his property rights to his food from my daughter.

  155. Thought I’d drop my two cents in here. If the opening post is asking whether or not I would copy my consciousness/memories/inclinations into another body in a bid for vicarious immortality, the answer is no. If this became possible, it would not represent the possibility for a continuity of experience but rather a way for me to carve my name in the stone and say once and for all that I was here, and so was I! Not a thing to gain really, just the arrogance of saying that no matter who is experiencing life, someone just like me should be.
    If we are talking about a more incremented transformation, say from my current self to a self with younger and possibly genetically enhanced features, to a self composed of cells designed from the ground up to be superior to the evolutionarily contrived cells they are to replace, then I wouldn’t hesitate to sign up, at least if the data were there to convince me it would work. The sticking point for me would be transformation by increment. If I could replace my body piece by piece with superior hardware, before finally replacing my brain one cell at a time as the old cells died out, I see no down side. Continuity is maintained. As I do not contain any of my zygotic or even infant cells at this point I do not fear this sort of process, but a sharp discontinuity that returns to an equilibrium similar to my self is a sort of chemical murder, like melting a coin and casting the metal in the original mold.
    It is this very same reasoning that means I would never go anywhere near a teleporter and would defend myself with violence if I were forced towards one. I just don’t think identical is good enough. Of course, it would be a victimless crime. Me not would not complain about me being dead, just the soul crushing realization that he was moments old and had never made a single decision that had led him to be the person he was, and depending upon the remoteness of the destination, the realization that he would need to commit suicide in the near future if he were ever to fulfill any of the obligations that he held closer to home.

  156. “Because there’s a huge explanatory gap. Why do particular brain states (particular arragenments of matter and energy) produce certain phenomenal experiences and not other phenomenal experiences, or no experiences at all?”

    Joining the discussion late. On this one, if I had to guess, it’s because brains are maintaining and updating an internal representation of the external world. That internal representation is your or someone’s experience of the world, hence consciousness and all the stuff that goes with it. A random soup of state variables doesn’t necessarily mean anything, but a pattern encoding the experience of a world is someone’s viewpoint.

  157. PS- wrt the teleporter – why is continuity important?

    Instinctively, we obviously struggle to maintain continuity because in the natural world there aren’t any convenient teleporters to reconstruct us if continuity fails to be maintained. Failure of continuity equals death.

    On the other hand, every night when we go to sleep, we suspend our conscious processing without any of the fearful anticipation of death associated with teleporters, or other means of being atomized. The concious pattern containing our awareness goes away, and comes back in the morning (with an intervening period of other activation states that aren’t strictly necessary wrt continuity (deep tranquilization/comas ect. produce the same subjective experience)). We are effectively turned off wrt. our concious experience. No one finds this weird, because it doesn’t lead to death.

  158. PPPS – would I become a machine?

    Unfortunately our most powerful present machines are probably barely capable of maintaining and turning over all the state variables necessary to encode something like a human brain. (Something that’s surprising, considering that neurons only operate on the order of 100 Hz). On the other hand, there are billions of them, and they each interconnect to thousands of others, so it’s quite a bit of state to be maintained.

    And that’s a separate issue from the question of robotics. Again, unfortunately, our reach exceeds our grasp at present in this department. Our robots are frail and coarse things in comparison to our bodies, even if they can outperform them in certain narrow aspects.

    But I think we’re pretty far along the right track to getting there eventually, with more refinement of the basic abstract ideas of turing machines/nanotechnology devices.

  159. PPPPS –

    Can’t leave this discussion alone -it’s something that has interested me for a while.

    With the picture of an internal representation of a world, the ‘minds-eye’, if you will, representing consciousness, (barring philosophical zombies, it’s pretty close to the language we use when describing our own internal experiences and what we mean when we say consciousness, so I think I’m pretty close), the question of “where” thinking takes place is another interesting one that I saw being asked further up the thread.

    I’m not so sure “where/when/how” are valid properties applied to this process (the abstraction, as opposed to the implementation), hence the confusion. For example, you could have a time-varying series of state variables representing someones experience of a soccer game. They could be implemented in an actual brain responding to an actual soccer game in a physical universe. They could be implemented in a Matrix-simulation of a soccer game being applied to a Matrix simulation of the state variable complex. They could be implemented in an abstract cellular automata thingy in Platonic space somewhere. These could all be happening at the same time, or at different times and places.

    However, I would contend they all represent the same subjective experience without respect to multiplicity, location, duration, ect.

    I don’t know. Just throwing this underrepresented perspective out there – it makes sense to me. What do you guys think?

  160. @ ams

    It’s true that we lose some time when we sleep and sometimes as a result of head trauma or excessive drinking and no one considers these occurrences to be examples of death and the generation of a new nearly identical individual. Intuitively the situation seems different when you are talking about being completely disassembled and not existing at all for a brief period of time.
    Perhaps intuition is not sufficient to understand such a situation. After all, people undergoing open heart surgery are technically dead and partially disassembled for extended periods of time and yet no one is under the impression that a man who underwent a bypass surgery is anyone other than the man who he was before. Similarly people plunged into freezing water can “die” for upwards of an hour and still return to life (though they do seem different afterwards).

    Wrt PPPPS: I’m not sure I understand what your getting at but it sounds like you’re saying that the medium material constituting a mind and the source of the information it receives is irrelevant as long as the information it receives is identical and the pattern of connections and potentialities in the mind is identical.
    If that is a fair appraisal I’d say that while that sounds like a truism, I would find it difficult to imagine a situation in which it could be implemented in reality. Changing the medium in which consciousness is being created by something as little as say reducing the amount of myelin in the central nervous system could result in anything from slower reflexes to blurred vision and lack of equilibrium. Of course this would offer different inputs to the system and break the rules but that seems likely if you change anything, especially the medium the mind is operating in.

    PS: I could have sworn I read an article a few years ago about scientists teleporting mid sized organic molecules, a staggering idea, but in practice useless because small viruses were still tens of thousands of times too large to be moved this way. A cursory look on google only reveals articles about the teleportation of light, energy and information.

  161. @esr
    “You might be and not know it. Some people semi-seriously propose that the Planck-scale granularity of the universe is a clue that we are all inside a gigantic discrete simulation.”

    One of the things that I keep coming across wrt quantum physics is talk of “granularity”. At the level which I understand such things right now, QM provides a planck’s constant limited *resolution* to the diffraction of particle-wavefunctions but resolution limits don’t necessarily imply any sort of discretization.

    If there were some sort of underlying discretization to space, how could you arrange it and still preserve the isotropy of physics with respect to rotation *and* relative motion (or even lorentz boosts)? Is there any evidence of directional or movement based anisotropy?

  162. >If there were some sort of underlying discretization to space, how could you arrange it and still preserve the isotropy of physics with respect to rotation *and* relative motion (or even lorentz boosts)?

    Sorry, I don’t know a good answer to that question either. It’s among the reasons I don’t actually take the Bostrom/Fredkin/Ulam idea that the universe is actually a discrete simulation very seriously. Sure, it’s probably possible to invent some way to preserve isotropy, but this would likely also have the effect of making the resulting theory untestable

  163. I’ve always enjoyed thinking about becoming a machine, if only for the cognitive aspects. As a moderately intelligent person whose brother is extremely intelligent, I always thought it was completely unfair that he could experience pleasures (e.g., mathematical elegance, the joys of working through a particularly abstract topology proof, and the beauty quantum mechanics) which I could only get a taste of. His experience is more profound than my own. As Hofstadter might say, he has a larger soul than I do. In short, I would prefer to be myself than a pig. Thus I see no reason why I wouldn’t prefer to be a large-souled robot than a man.

  164. excellent discussion. Ask musicians about finger memory. I can’t find it now (of course) but I recall reading how some musicians’ fingers moved faster than there was time for the instructions and feedback to go from finger to spine to brain and back.
    We think thinking happens in the head because our eyes and therefore our sense of self is located there, but it might be as the ancient Greeks believed, that thinking occurred between the ears and emotions really did come from the heart.

  165. “but I recall reading how some musicians’ fingers moved faster than there was time for the instructions and feedback to go from finger to spine to brain and back.”

    How do you test for this? It doesn’t seem to me like there’s necessarily a lot of feedback (particularly from the fingers, rather than the ears) involved in most instruments. And without that, all you’ve got is pipelining.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>