Thanks to Justin Caouette for inviting me to the blog. I’ll start with a bit that draws ideas from a paper I’m working on for a book on Robot Ethics:
The standard criteria for personhood are not obviously inherently impossible for AIs to obtain: they could be self-conscious, they could regard others as persons, they could be responsible for actions, they could have relations to others and to communities, they could be intelligent, rational, desirous, they could have second order intentions, and, at least on compatibilist notions of free will, they could have free will, presuming we could program emotions into their software. Bostrom[1] makes a good case that all of this is ultimately doable, and even AI skeptics like Searle don’t rule it out in the long run, as long as we understand the different possibilities of hardware and software.
But even if AIs acquired all the standard personhood traits, there is one area where they may be so unlike persons that we would have to rethink some of the limits of the concept: AIs would not abide by norms of personal identity. Personal identity across time is usually tracked by either psychological or physical continuity. Both of the physical and the psychological can change while identity remains constant, but only if the change is either piecemeal or gradual. That is, if a person grows and loses cells or memories over many years, they retain identity. But if I change all of someone’s cells or psychological content at once, then I’ve created a new person. But with an artificial person, the standard modes of psychological and physical change that our identity concepts are adapted to are wiped away.
Not only would sudden and radical change in both the physical and mental becomes possible, it would be standard for a machine updating its data banks, reprogramming itself, and swapping out hardware. A similar effect could potentially occur for humans, if we acquire the enhancement technology to reprogram our desires and beliefs, and the prosthetic technology to swap out body parts at will.
So, human enhancement and strong AI converge upon the creation of entities that are largely capable of rewriting or recreating their personalities, bodies, and, by extension, themselves. As so many philosophers have recently noted, human enhancement creates moral dilemmas not envisaged in standard ethical theories because it alters the possibilities under which ethical beings operate. What is less commonly noted is that, at root, many of these problems stem from the increased malleability of personal identity that this technology affords. If a self becomes so re-workable that it can, at will, jettison essential identity-giving characteristics, how are we to judge, befriend, rely upon, hold responsible, or trust it? We’re used to re-identifying people across time. We assume that persons are accountable for and identifiable with prior actions. But can a being that is capable of self-rewriting be said to have an identity? Since most human interactions that take place across more than a few minutes rely upon a steadiness in the being of the other person, a new form of person, capable of rapidly altering its own memories, principles, desires and attitudes creates tremendous problems not only ethically, but metaphysically as well. If all the identity-making properties of a person are unstable, how can we track that person across time?
So, while an AI could be a person[2], that might require that we prevent it from learning, adapting and changing itself at the rate at which it would be capable. That is: to make an AI-person, we would need not only to achieve a high-level of technical expertise in programming and hardware, we would also need to stop it from utilizing the benefits of that hardware and software. We would need both technological advances, and a technological brake. As Tom Douglas[3] has noted, AIs might wind up being, in an ethical sense, a supra-persons. That is, they might be so much better at ethical thinking, so much smarter and more efficient and more productive and more cooperative and less selfish, that we would fall behind them in ethical standing. Douglas assumes that such beings would be, if anything, even more responsible for their actions than we are. But again, this would require that they have some kind of stable identity to attach blame and praise to. So while a supra-personal AI is conceivable, it would again require that we not allow it to utilize its ability to change to its full capacity. Otherwise, it may cease to be on the person scale at all. What we are likely to create, though, if we allow AIs all the benefits that emerging technologies can bring, are para-persons, things that have all the personhood qualities, or pass all the tests for personhood, that philosophers have set up (self-awareness, ethical cognition, other-awareness, self-respect, linguisticially expressible concerns etc.), but also have an ability that makes them not supra-persons, but something outside of personhood. That is, even if it had all the personhood qualities, it could also have an additional, defeating quality for personhood: the ability to change instantly and without effort. Our ethical systems are designed or adapted to apportion blame and praise to persons. They could do the same for supra-persons. But it’s not clear that they will work with the kind extremely malleable para-persons that strong AI or strong enhancement will produce.
To give some quick examples: suppose an AI commits a crime, and then, judging its actions wrong, immediately reforms itself so that it will never commit a crime again. Further, it makes restitution. Would it make sense to punish the AI? What if it had completely rewritten its memory and personality, so that, while there was still a physical continuity, it had no psychological content in common with the prior being? Or suppose an AI commits a crime, and then destroys itself. If a duplicate of its programming was started elsewhere, would it be guilty of the crime? What if twelve duplicates were made? Should they each be punished? And would we have competing intuitions if we asked: suppose an AI performed a heroic act, but then completely reprogrammed itself. Should we give an award to the new being that occupies the same “body”? Suppose it did its heroism and then was destroyed, but years later a copy of its programming was found and started up on hundred new machines. Should we award each of them? What if it was on only one machine? I don’t think these questions have clear answers, even if you or I might have strong intuitions about them, because it’s fairly easy to create conflicting intuitions, or to find strong areas of disagreement. This is, I think, because we’ve exceeded the current bounds of our identity concepts, and if that’s true, and if it’s a central part of what it is to be one of these AIs, then I think they may be more para-persons than persons, as identity, which is central to personhood, is no longer applicable in its current form.
[1] Bostrom, N. (2014). Superintelligence: Paths, dangers, strategies. OUP Oxford. See chapter 2 for a good argument for the possibility of person-level AI.
[2] There’s certainly a lot written on this, but I’ll just plug an early, and often overlooked essay, Dolby, R. G. A. (1989). “The possibility of computers becoming persons,” Social Epistemology, volume 3, issue 4, pp 321-336; as I think it provides a strong argument that is unlike that found in the more recent papers on this topic.
[3] Douglas, T. (2013). Human enhancement and supra-personal moral status. Philosophical studies, 162(3), 473-497.
A Philosopher's Take 
Angra Mainyu
June 2, 2016
Hello James
Great post!
I’m still thinking about the identity issue; it’s a lot of food for thought.
In particular, I’ve been thinking about your point that in order to make the AI persons, we might have to not allow them to change themselves in that fashion. A question is: would that be morally acceptable, assuming humans manage to make them persons in the first place?
For example, what if a stable AI is a person (it’s prevented from changing in the way you describe), and it wants to change to make itself morally better. Would it be okay to forcibly prevent it from doing so?
That aside, I’d like to add another potential difficulty for personhood (or at least for moral responsibility), even if the AI are stable: what if they value different things? Could they find moral truth? And if they can’t, do they still count as persons?
I think this problem stable AI might be (depending on the programming) might be akin to intelligent aliens from another planet. For example, let’s say that some smart aliens (with language, spaceships, etc.) aliens evolved from something like squid (“alien squid” for short), and others from elephants (“alien elephants”).
It seems likely to me that they would reach very different judgments (from each other and from ours) in terms of positive or negative evaluations, e.g., if they visit Earth and compare a scenario in which an Earth elephant is brutally killed and eaten by lions, and one in which the lions starve to death. One interpretation is that one of them is wrong in their evaluations of better/worse; my interpretation of such scenarios is that alien squid are not talking about good or bad things, but about alien-squid-good and alien-squid-bad (which supervene on other things, described in non-evaluative language), and the same for the alien elephants.
Similarly, their obligations (at least, the ones they care about and that their obligation-like terms refer to) would be tied not to the good and bad, but to the alien-squid-good and alien-squid-bad, etc., and then there is a question of whether those are moral obligations, or those are alien-squid-moral obligations, and if the latter, whether the alien squid also have moral obligations (which might even be in conflict with the alien-squid-moral obligations).
Now, one might think at first glance that human-made AI would not be like that, since they’re made by humans. However, it seems to me that something like that may very well happen for two reasons:
1. The human programmers might not include morality in the AI, but some goals, and then let it develop in an evolutionary-like way, or by trial and error, etc.
2. The human programmers might program morality in the AI under their preferred theory (consequentialism of some sort, maybe), and then the theory turns out to be false with regard to [human] morality; I think all of those theories are false, even if their verdicts (when they’re precise enough to yield verdicts) about right or wrong often (but not always) agree with the moral truth, in cases that humans are normally likely to encounter (but who knows about cases involving AI?). If the AI are programmed with a value function that pursues maximizing happiness or whatever (or even something far more sophisticated and precise than that), the result might be very similar to morality in cases humans normally encounter, but not so much in abnormal cases humans might encounter (e.g., involving interaction of humans and AI, like the ones you mention), and also cases humans would rarely encounter by AI might.
LikeLike
Lage
June 2, 2016
“they could have free will, presuming we could program emotions into their software”
Why are emotions necessary for compatibilist free will? Do you see them as required for free will? Or are you simply referring to some type of value system implementation generally? One could argue that if anything, emotions actually detract from free will (think of crimes of passion for example), so I’m not sure what your thoughts are on this point.
“Both of the physical and the psychological can change while identity remains constant, but only if the change is either piecemeal or gradual.”
I highly disagree with this assumption, in that I don’t think that identity remains constant over the course of psychological/physical change. I think there is an illusion of constancy or continuity but it’s just that — an illusion. If you could immediately switch between the body and brain that you had 15 or 20 years ago, I highly doubt you would be the same person (in this “identity” sense). That is to say, I think that our identity is in constant flux even though we don’t notice it changing because it changes at the rate it does (among other reasons including how memory works, how old memories are changed in light of new experiences in order to make the “self narrative” more consistent and coherent). The fast changes with future AI would just be sped up versions of this, so your point still stands that “…even if AIs acquired all the standard personhood traits, there is one area where they may be so unlike persons that we would have to rethink some of the limits of the concept”, and so I agree that our traditional intuitive concept of personal identity would no longer apply to AI that change rapidly enough in personality. But I do not agree that it is a matter of humans’ identity remaining constant versus AI that do not but rather that the rates of change in identity (an everchanging identity in both humans and AI) will be different enough to warrant new considerations.
“That is, if a person grows and loses cells or memories over many years, they retain identity. But if I change all of someone’s cells or psychological content at once, then I’ve created a new person.”
As per my previous point, changing someone’s cells or psychological content at once, such as reverting to your “self” as of 15 to 20 years ago, you would indeed have a different identity. But this happened naturally anyway (a new person was created), it just took 15 to 20 years to accomplish the new person that exists today.
“A similar effect could potentially occur for humans, if we acquire the enhancement technology to reprogram our desires and beliefs, and the prosthetic technology to swap out body parts at will.”
I agree with this and this comment of yours leads me to some points I’d like to make but first I’ll mention one more quote from you, “That is, even if it had all the personhood qualities, it could also have an additional, defeating quality for personhood: the ability to change instantly and without effort. Our ethical systems are designed or adapted to apportion blame and praise to persons. They could do the same for supra-persons. But it’s not clear that they will work with the kind extremely malleable para-persons that strong AI or strong enhancement will produce.”
Right, and this goes to show why we assign praise and blame as we do in our general moral framework — because it is needed to reprogram us effectively so that we do what the future AI can do without such a blame/praise or reward/punishment type of operant conditioning required. So if its not needed for that functional purpose then we would simply abandon it for those types of people. The same thing would result if we had the situation you eluded to with your claim about human enhancement technologies, whereby we may one day be able to do the same thing, change our behavioral decision-making processes instantly so as not to require the time-consuming conditioning processes that we evolved with as a social species. I don’t really see the problem in any major sense though since the ultimate goal we have for both altering human or AI behavior would be to do what is most effective in achieving behavioral goals while maximizing happiness/satisfaction. We can see the praise/blame and punishment/reward systems that we implement for people come in degrees too — where some people need much more extreme versions of those influences in order to alter their behavior and others are simply far easier to retrain and can learn with relatively few constraints and conditioning efforts to behave. If AI simply needs none of this (or the bare minimum of it), then it is simply the boundary condition of our scale of conditioning needs for any particular person. It’s true that we haven’t had to operate on this extreme end of the scale (i.e. no long drawn out conditioning needed), but it doesn’t mean that the persons will fail to be persons. They will just be very different kinds of people from the people we’re used to interacting with (slow learners and less effective self-reprogrammers).
“To give some quick examples: suppose an AI commits a crime, and then, judging its actions wrong, immediately reforms itself so that it will never commit a crime again. Further, it makes restitution. Would it make sense to punish the AI?”
No it would not make sense because it wouldn’t need the punishment or incarceration to protect society nor to alter it’s behavior — which are the only reasons for criminal corrections in the first place. The only benefit it could have is to comfort irrational people that would somehow feel better from seeing that being treated as most everyone else is treated, but that’s irrational on many counts and so isn’t a valid reason to do it under any defensible moral framework.
“Or suppose an AI commits a crime, and then destroys itself. If a duplicate of its programming was started elsewhere, would it be guilty of the crime?”
I think this depends on how we define “it” or how we define identity (which is certainly a valid reason for you writing this post in the first place so this definition can be ironed out better). I would say that it IS guilty of the crime if we define personhood or persons based on information (which is probably a more correct “currency” if you will, for defining personhood and identity; information, rather than specific hardware/software). However, in this scenario, if it simply reprograms itself to avoid this criminal behavior then it is no longer “guilty” of anything in any practical sense of the term.
“…suppose an AI performed a heroic act, but then completely reprogrammed itself.”
No, it wouldn’t deserve praise or reward/award because it is no longer the same person since the information that comprised its personality was changed. This is assuming of course that it reprogrammed itself such that it would no longer know it performed the act. However even this assumption isn’t needed because if it doesn’t require these kinds of reinforcement forces (praise/reward/award) to sustain good behavior then it doesn’t “deserve” anything in that sense. We tend to use the word “deserve” based on the assumptions of conditioning mechanisms being needed to deter bad behavior and sustain good behavior. You remove that need and you remove the need for praise and blame as well as the “deserve” concept carried along with it as far as I can tell.
“This is, I think, because we’ve exceeded the current bounds of our identity concepts, and if that’s true, and if it’s a central part of what it is to be one of these AIs, then I think they may be more para-persons than persons, as identity, which is central to personhood, is no longer applicable in its current form.”
I agree with your general conclusion here. The personhood concept and how we intuitively use it will have to be reformulated or adjusted for the boundary condition that we’ve reached or will reach with this future AI (i.e. no conditioning mechanisms needed or only the bare minimum needed, i.e. one trial of behavior and immediate adjustment if needed).
LikeLike
Angra Mainyu
June 3, 2016
James,
After further considering the matter, in my assessment stability (of the kind we’re talking about) is not a condition for either personhood or moral responsibility, and if the AI forced to be stable are persons, the others are also persons (but they might be many of them!). However, that does not resolve the main difficulties your points raise.
In order to argue that stability is not a condition for either personhood or moral responsibility, I will introduce some scenarios involving humans, not AI, and then address AI.
Let’s say the universe is either infinite or finite but big enough to that weird quantum stuff happens (no matter how weird).
There is a planet, Earth1, and a man, Jack1, who is a serial killer, and decides to kill Bob1 for fun. Jack1 kills him on 01/01/2016, and life goes on, without any weirdness.
Earth2 is like Earth1, but it’s a Boltzmann planet in a Boltzmann planetary system. Jack2 is just like Jack1 1 hour before Jack1 killed Bob1 (i.e., Jack2 looks the same, the brain has neurons in the same positions and with the same connections, etc.), and he too decides to kill Bob2 for fun and does so. But when he does it, Jack2 (and the whole planetary system) is only 1 hour and 5 minutes old. Then, life goes on, without further weirdness.
Earth3 is like Earth1 up to 5 minutes after Jack3 kills Bob3, but after that, Jack3 is changed by a quantum freak event in the way you described when you said “But if I change all of someone’s cells or psychological content at once, then I’ve created a new person.”
Earth4 is like Earth2 up to 5 minutes after Jack4 kills Bob4, but after that, Jack4 is changed by a quantum freak event in the way you described when you said “But if I change all of someone’s cells or psychological content at once, then I’ve created a new person.”
In my assessment, all of the Jacks are persons, and all of them are morally responsible, guilty, etc. Now, it seems plausible when Jack2 and Jack4 are changed, they’re actually killed, and some other person takes their place, and this different person is not guilty of killing Bob2/Bob4, deserves no punishment, etc.; however, Jack2 and Jack4 did deserve punishment before they were changed.
The most extreme case is Jack4: he lives for only a bit over an hour, but he’s a person, and morally responsible. Even shorter periods, like minutes, would not prevent that from happening, as long as they get to make choices, and that’s akin to the AI case.
Now, with regard to AI, I will assume that either the alien difficulty I raised can be overcome (i.e., that those aliens would in fact be persons), or in any case, that we can make AI with the right mental properties for personhood and moral responsibility, as long as we prevent them from changing as you suggested, and leaving aside the question of the morality of preventing them from changing against their will. In that case, we may consider an example you suggested, and compare it with a similar one, but in which the AI is prevented from changing:
AI1 commits a crime, judges its actions wrong, reforms itself so that it will never commit a crime again, and makes restitution.
AI2 commits the same crime and makes the same moral judgment, but is not allowed to change.
It seems to me that if AI2 is a person when he commits the crime and is guilty, then so is AI1. Further questions are whether AI1 is the same person after it changes itself, or more precisely, whether it changes itself or kills itself, making another person in the process. My impression is that it depends on the extent of the changes. But either way, it seems to me that AI1 was guilty before the change, and was already a person. Maybe “AI1” refers to two consecutive persons, or more than two.
So, rather than parapersons I would say that AI identified by the same name might be either a single person, or several/many persons one after the other, getting killed in the process.
That said, there is the difficulty that perhaps our identity concepts are not precise enough to be properly used in that context, and so in that case we should say there is no fact of the matter as to whether that’s a single person or consecutive persons, but that does not preclude moral responsibility (assuming always that stable AI would be both persons and morally responsible for their actions).
None of this, however, resolves problems such as whether the AI after the change deserves to be blamed, punished, etc., or whether the copies deserve that, and so on.
LikeLike
corcor194
June 4, 2016
Interesting post, James!
To my mind, the problem that you’re seeing is a more salient example of how reductionist accounts of personal identity come up short in general. It could be that determinate identity is problematic to begin with (my own view). So, your ‘para-person’ conclusion is right, but not because AI presents some new challenges to personal identity.
LikeLike
James DiGiovanna
June 4, 2016
Thanks for the comments. I don’t have time to respond fully, but I would like to note that part of the reason for my exploring this concept was to draw into question the very idea of personal identity. So I tend to agree, in part, with those above who noted that these same objections can apply to ordinary humans. Obviously, not in precisely the same ways, but still, the degree of change that we have to undergo just to be organic human beings makes strict identity problematic at best. Speeding up the process, with the AI example, can help to make that clearer.
Still, there are identity criteria which can overcome most of the simple objections about transitivity and such (Parfit’s overlapping chain, Olson’s animalism), even if, at least in my opinion, they don’t fully account for what’s happening when we talk about sameness of persons. What the AI examples show is that, even on these accounts, we can meet all the criteria for personhood, and fail to meet the criteria for personal identity.
Ultimately, I think personal identity question require context. That is, “is this the person who committed that crime,” may have a different answer from “are you truly the man I married?” or “is this the person who owns this property.” I won’t go into details here, but a preliminary account of this is given in DiGiovanna, J. (2015). “Literally Like a Different Person: Context and Concern in Personal Identity” The Southern Journal of Philosophy, 53(4), 387-404.
Oh, and just quickly for Lage: emotions are necessary for free will insofar as we conceive of free will as, at least in part, being able to do what we want. Wanting, I think, is, or at least presupposes, an emotional state. I can imagine an emotionless robot that simply follows an algorithm for all its “choices,” but that, at least, doesn’t strike me as capturing what’s meant by “free will.” Without desires of some sort, it’s not clear to me that we have an account of will that matches the standard sense of the term.
LikeLike
Lage
June 5, 2016
Thanks for the response James. Regarding your reply about emotions and free will, couldn’t it be the case that what we tend to call “emotions” or “desires” are really just descriptions of the algorithm we use for making decisions? If this is so, then perhaps there is no such thing as an “emotionless robot” as long as it has desires (some end goal or set of end goals that drive its behaviors in particular directions given certain inputs). I think most people take free will to mean “free to make decisions without coercive forces in play” which the robot would be able to do in many situations. I think it’s imperative hear to think more critically about “desires” and what they are fundamentally. The decision-making algorithm in a robot or in a human may drive behavior in a certain way and manifest what we call “feelings” or “emotions” in the process of that algorithm’s execution.
LikeLike
James DiGiovanna
June 5, 2016
I don’t actually buy any version of free will, but on pretty much all of them desire is some part of the equation. I also think there are emotionless robots, because, unless there’s phenomenal consciousness, there wouldn’t be any emotions, by definition (emotions are felt.) I really doubt any current robot has any kind of phenomenal consciousness, but there are a small minority who disagree. But as for emotions being descriptions of algorithms, I think that only goes so far, as the algorithm also has to be tied to a feeling. So, sure, humans are ultimately just very complex machines, and there are, in a sense, algorithms at work in our decision-making. If that were all, then we wouldn’t even have compatibilist free will. Dollop on some feelings, and a certain relation between our desires and our actions, and you can get compatibilism. As for free will = decision without coercive force, that’s not generally sufficient, otherwise, yeah, robots would have free will. So would just about anything that moved. Rocks falling down hills, for example. Unless we tighten up the definition of “coercive,” and if you tighten it up enough, nothing has free will. The general, classical sense of free will, though, is that at action is free if it wasn’t _determined_ by prior events or states of affairs, whether or not those events/states were coercive. Compatibilists, of course, are fine with it being determined, as long as it meets other requirements (we’re doing what we wanted to do, etc.)
LikeLike
Lage
June 9, 2016
Yeah, I agree that emotionless robots can exist and likely all the ones we’ve seen thus far fall into this camp. However, I do think that emotions or feelings result from a very complex version of decision-making algorithms that we have. A type of decision-making algorithm that most people would label as “conscious free will”, but a decision-making algorithm nevertheless that relies on memory, prediction, and a homeostatic goal of some kind (what you might reduce to “desires”, but which can likely be further reduced to something analogous to the “desires” of a thermostat to respond to inputs with certain outputs to achieve some goal). I think this is all that is needed for compatibalist free will. I think it is primarily the complexity of the algorithm that accounts for emotions and feelings and one that a “robot” would experience once the complexity is high enough for the task.
I disagree with your claim that the “classical sense of free will” is that an action is free if not determined. That makes all random (i.e. indeterminate) actions “free” which doesn’t at all jive with what people mean when they use the term (they WANT to be the cause of their actions, they don’t want it to be left up to chance or randomness). The two primary conceptions of free will that I’ve come across are: 1) a libertarian sense whereby if I went back in time and repeated initial conditions, I could in principle arrive at a different outcome without any need of randomness (causa sui free will) despite that only being possible by violating physical causality, and 2) a compatibalist sense whereby I can make decisions “freely”, that is, without coercion and for reasons/values that I hold which can themselves be changed over time — even if the decision/action couldn’t have happened any other way given the same initial conditions. The former type of free will is physically (if not logically) impossible and the latter merely allows us a workable concept to describe our ability to make decisions based on our reasons and be held accountable for those decisions/actions. The latter is kind of a linguistic trick whereby we don’t really have the libertarian freedom that many people think we have, but that we are nevertheless capable of executing decisions/actions based on algorithms that are a part of our personhood so to speak and algorithms that are changeable. The latter also provides us with a more intuitive way of talking about moral responsibility and so forth.
LikeLike
Lage
June 17, 2016
Hey James, I decided to promote your post and write a little about the topic on my blog “The Open Mind”. Here at: https://lagevondissen.wordpress.com/2016/06/17/the-illusion-of-persistent-identity-the-role-of-information-in-identity/
LikeLike