Category: Ethical consideration of non-humans and future persons

  • “One of [Peter Singer’s] ideas is that all suffering matters, that somebody else’s suffering is in fact also our suffering. And he doesn’t limit this view to people. As opposed to idealistic altruism, this kind of altruism is not incompatible with the findings of evolutionary psychology. But it’s kind of like saying, “Given that our minds work this way, shouldn’t we try to be as scientific and systematic about altruism as we can be?”

    Jeremy Lauer, Interview with Jeremy Lauer, Senior Economist at the World Health Organization (Jul 13, 2015).

  • “[W]e need to make sure future generations care about moral philosophy and have the capability to do a better job than us.”

    Robert Wiblin

  • “Being a utilitarian, my interest is in the universe.”

    Torbjörn Tännsjö, Narrow Hedonism (2007), p. 80

  • “[G]etting AGI right is the most important project from an infinite-ethics standpoint, but there may be reasons that better understanding the infinity problem in the short term would make a difference. For example, the AI’s conception of infinity might be constrained by its design – e.g., AIXI assigns zero probability to uncomputable universes.”

    Brian Tomasik

  • “In addition to reducing suffering in the short run, we should consider how our actions will affect the future, including the far future. We appear poised at a crucial period in history, where the trajectories of our technology and society may make a lasting impact on intelligence in our region of the universe for billions of years. It looks likely that artificial general intelligence (AGI) will be developed in the coming decades or centuries, and its initial conditions and control structures may make an enormous impact to the dynamics, values, and character of life in the cosmos. Colonization of space seems likely to increase suffering by creating (literally) astronomically more minds than exist on Earth, so we should push for policies that would make a colonization wave more humane, such as not propagating wild-animal suffering to other planets or in virtual worlds.”

    Brian Tomasik, Risks of Astronomical Future Suffering

  • “I care about reducing suffering regardless of whether it reduces the total amount of suffering in reality. Consider: If there are two buttons in front of me, one resulting in 3^^^^3 tortures and the other resulting in 3^^^^3 years of happy, exciting, meaningful and fun human lives, I would like to press the second button. But imagine if next to the buttons there were an indestructible locked box with infinite negative utility in it, and that I couldn’t do anything about it. Would this box suddenly make my choice between the buttons futile? I do not think so, because I care about the utility regardless of what’s in that box.”

    Timo Timo

  • “[T]he things I care about that I can’t influence do not change what actions my decision theory recommends.”

    Timo Timo

  • “Not only in probability theory, but in all mathematics, it is the careless use of infinite sets, and of infinite and infinitesimal quantities, that generates most paradoxes.”

    E.T. Jaynes, Probability Theory: The Logic of Science

  • “When some principle requires us to act in some way, this principle’s acceptability cannot depend on whether such acts are often possible.”

    Derek Parfit, Justifiability to Each Person (December, 2003)

  • “[T]he things we have created will eventually vanish once human beings are no longer around to preserve them. However, achievements are events, not things, and events that have occurred cannot be undone or reversed. Therefore, it will continue to be true that our achievements occurred even if humanity ends. One disadvantage of having an unalterable past is that we cannot undo a wrongdoing that occurred. However, an unalterable past is also an advantage in that our achievements can never be undone, which may give some consolation to those who desire quasi-immortality.”

    Brooke Alan Trisel, Human Extinction and the Value of Our Efforts (2004), p. 390

  • “What interests me most are the metaphysical questions whose answers can affect our emotions, and have rational and moral significance. Why does the Universe exist? What makes us the same person throughout our lives? Do we have free will? Is time’s passage an illusion?”

    Derek Parfit (1993)

  • “Does life have meaning? Are there objective ethical truths? Do we have free will? What is the nature of our identity as selves? Must our knowledge and understanding stay within fixed limits? These questions moved me, and others, to enter the study of philosophy. I care what their answers are. While such other philosophical intricacies as whether sets or numbers exist can be fun for a time, they do not make us tremble.”

    Robert Nozick, Philosophical Explanations (1981), p. 1

  • “The story of the first artificial Ego Machines, those postbiotic phenomenal selves with no civil rights and no lobby in any ethics committee, nicely illustrates how the capacity for suffering emerges along with the phenomenal Ego; suffering starts in the Ego Tunnel. It also presents a principled argument against the creation of artificial consciousness as a goal of academic research. Albert Camus spoke of the solidarity of all finite beings against death. In the same sense, all sentient beings capable of suffering should constitute a solidarity against suffering. Out of this solidarity, we should refrain from doing anything that could increase the overall amount of suffering and confusion in the universe. While all sorts of theoretical complications arise, we can agree not to gratuitously increase the overall amount of suffering in the universe – and creating Ego Machines would very likely do this right from the beginning. We could create suffering postbiotic Ego Machines before having understood which properties of our biological history, bodies, and brains are the roots of our own suffering. Preventing and minimizing suffering wherever possible also includes the ethics of risktaking: I believe we should not even risk the realization of artificial phenomenal self-models.”

    Thomas Metzinger, The Ego Tunnel

  • “There are known puzzles and unknown future discoveries in physics, neuroscience, anthropics, infinity, general epistemology, human (and nonhuman) values, and many other areas of life that are likely to radically transform our views on ethics and how we should act in the world. Taking object-level actions based only on what we know now is sort of like Socrates trying to determine whether the Higgs boson exists. Socrates would have been at a complete loss to solve this question directly, but one thing he could have done would have been to encourage more thinking on intellectual topics in general, with eventual payoff in the future.”

    Brian Tomasik, Charity Cost-Effectiveness in an Uncertain World (Foundational Research Institute)

  • “Suffering exists only because it was good for our genes. Conditionally-activated negative emotions were fitness-enhancing in the ancestral environment. In the current era, apologists for mental pain are serving as the innocent mouthpieces of the nasty bits of code which spawned them.”

    David Pearce

  • “Sometimes life is too uncertain to have regrets.”

    Son-Goku

  • “[T]he roots of all evil can be seen in natural selection, and are expressed (along with much that is good) in human nature. The enemy of justice and decency does indeed lie in our genes.”

    Robert Wright, The Moral Animal: Why We Are, the Way We Are: The New Science of Evolutionary Psychology (1994), p. 151

  • “[E]volution does not give us a best of all possible worlds.”

    Daniel Dennett, Three Kinds of Intentional Psychology (1987)

  • “It is difficult to believe that there is no such thing as an all-or-nothing self—no “deep further fact” beyond the multitude of small psychological facts that make you who you are. Parfit finds that his own belief is unstable—he needs to re-convince himself.”

    Larissa MacFarquhar, How to be Good (The New Yorker, September 5, 2011)

  • “[Derek Parfit] doesn’t believe that his conscious mind is responsible for the important parts of his work. He pictures his thinking self as a government minister sitting behind a large desk, who writes a question on a piece of paper and puts it in his out-tray. The minister then sits idly at the desk, twiddling his thumbs, while in some back room civil servants labor furiously, come up with the answer, and place it in his in-tray.”

    Larissa MacFarquhar, How to be Good (The New Yorker, September 5, 2011)

  • “If I am a momentary conscious self, and might have (numerically) the same experience as I do now even if I were not causally connected to anything more remotely past or future than the before and after internal to my momentary experience, [t]hen it seems that my continuant personal identity should not be of all that much special interest to me now. For if the way that I am a continuant is by being a collection of, say, segments of continuing physical processes coming together into integrated systems of neural events at one moment only to come apart the next, why should I identify with the future of some of these causal processes rather than with others? Why not care equally about other momentary consciousnesses that I can causally affect, rather than just about that which bears my name? Why not about those that carry the effects of my deeds, or of my social interaction, equally? None of these will be the same momentary consciousness as I this moment am. All will be tied to my present consciousness by causal connections.

    These considerations seem plausible to me. [I]f this scene of thought undermines egoism and the egocentric fears (such as, perhaps especially, the fear of death), it might seem equally to undermine morality, too – by weakening the grip that our biologically and socially based perception and attitudes toward persons as such have. For it seems to be here that morality finds its natural ground – on which the existence of moral facts and motivation, and the application of the distinctive normative force of morality (irreducible to that of seeking pleasure or any other form of welfare or good) depends. But philosophical hedonism, while perhaps undermining morality and self-interest together in this way by suggesting the momentary view, could also provide some justification for self-interest and morality in those moments in which we wonder how it all matters, at a fundamental level, by showing a deeper ground and point to human living – a ground in the momentary experience of pleasure (no matter whose), a ground beyond self interest and morality that lies deeper in the nature of things than does our perception of persons or of prudential and moral norms.”

    Leonard Katz, Hedonism as Metaphysics of Mind and Value (1986), pp. 177-179

  • “[On Parfit’s view], the boundaries within lives are like the boundaries between lives. So we do not regard people as the morally significant units. This only means that if we are concerned with distribution at all, we shall be concerned with distribution between what are the morally significant units – namely, person-segments or whatever these divisions of a person are. So certainly the fact that a person has suffered more in the past will not make us give extra weight to relieving her suffering now. But if she is suffering more now, we may give extra weight to it. We may be concerned to equalize the distribution of good between person-segments. So all this argument does is remind us that we have changed the units of distribution. It does not suggest that we should be less interested in distribution between them.”

    John Broome, Utilitarian Metaphysics? (Interpersonal Comparisons of Well-Being, 1991, p. 94)

  • “There can be an experience-oriented and a person-oriented version of hedonism. On the former view, it is the experience of happiness that is good, wherever it occurs; on the latter view what is good is that people are happy. On the former view people matter, so to speak, only as containers of happiness – it is the total quantity of happiness that really matters. On the latter view the starting point is impartial concern for the happiness of actual people. Real and important ethical differences can flow from this very deep contrast.”

    John Skorupski, The Philosophy of John Stuart Mill (2007)

  • “Why, Parfit wonders, are we so biased toward the future? Was this tendency produced by natural selection? We are upset when we are told that in the future we shall have to endure a day of great pain, but many people do not care at all if they are told that they endured pain in the past that has been forgotten; and yet the past pain is just as real. We don’t have the same bias with other people: if we learn that a loved person suffered greatly before he died, we are upset by this, even though it’s over. The past is just as real as the present. If someone we loved is dead, that person isn’t real now. But that’s just like the fact that people who are far away aren’t real here.”

    Larissa MacFarquhar, How to be Good (The New Yorker, September 5, 2011)

  • “I can imagine a more enlightened society responding to many of our mistakes with exasperation and disappointment, but I have a hard time imagining that they’d react with abject horror and disbelief to the discovery that consumers contributed in indirect ways to global warming – or failed to volunteer at soup kitchens. I have a much easier time imagining the “did human beings really do that?!” response to the enslavement and torture of legions of non-human minds for the sake of modestly improving the quality of sandwiches.”

    Rob Bensinger, Revenge of the Meat People

  • “I am now inclined to believe that time’s passage is an illusion. Since I strongly want time’s passage to be an illusion, I must be careful to avoid being misled.”

    Derek Parfit

  • “As a vegan I’m not really interested in discussing what type of diet our caveman ancestors ate, or whether God put animals here for us to eat, or if we can obtain enough protein from a plant-based diet, or if humans are biologically herbivore or omnivore. These are tired arguments. What we should be discussing is how we can move forward and evolve as a race. The killing of billions of animals every year for food and clothing has helped put this planet and its inhabitants in the crisis situation that we’re now faced with. And veganism offers a solution. Global warming, the declining health of our population, polluted waterways and depleted soil, fished out oceans, starvation in poor countries, species extinction and rain forest destruction. [T]he human race has backed itself into a corner, veganism offers a way out. Let’s move forward.”

    Rich Lysloff

  • “One of the biggest flaws in the common conception of the future is that the future is something that happens to us, not something we create.”

    Michael Anissimov

  • “Three pigs were brought into the town,

    They all began to squeal;

    A man with long and pointed knives

    Their fate was set to seal.

    They kicked and pushed and shook in fear,

    Yet could they know just why?

    Perhaps it was a hidden sense

    Told them they were to die.

    But just as they got to the place

    Which was their journey’s end,

    The pigs shoved hard with all their might

    And posts began to bend.

    The fence fell down and off two went

    As fast as they could go,

    And that they swam from bank to bank

    The world was soon to know.

    As word got out of their escape

    Folk came to have some fun;

    To catch a sight of two young pigs

    Who now were on the run.

    The press came in from near and far,

    The T.V. cameras too;

    With ‘copters flying overhead,

    What would our two pigs do?

    They hid and ate in field and copse,

    Rejoicing to be free;

    They led the press a merry dance,

    What was their fate to be?

    “They’re for the chop, they will not live!”

    Their owner said aloud;

    ‘Twas something that he said most clear,

    Almost as if quite proud.

    Oh No! Oh No! They must not die!”

    The cry was heard all round,

    “They’ve won their right to live in peace,

    A new home must be found.”

    So when they’re caught and that man says

    They will not have to die,

    The fact he got some fifteen grand

    Could be the reason why.

    Of those two pigs we heard a lot,

    But not so much their mate;

    What was to be the end of him?

    What was to be his fate?

    At five months old unlike his friends

    His future was less sweet;

    With fear and pain, then blood and guts,

    He ended up as meat.

    No matter just how far it is

    From abattoir to plate,

    The suffering of those who die

    Is always just as great.

    What right have we to take the lives

    Of those who are so mild?

    To sex, to fix, to cage, these ones,

    When each is like a child?

    Our brains and might give us much power

    O’er all that is around;

    We must make sure we live our lives

    On principles more sound.

    If who shall live and who shall die

    Is based on power and taste,

    ‘Tis surely not their lives alone

    That we do choose to waste;

    For when we hurt and maim and kill,

    And then the victims eat,

    Something inside each one of us

    Will also face defeat.

    What would be lost, I ask you all,

    But chains and ties that bind,

    If we should choose a way of life

    That is not cruel but kind?

    For health, for wealth, for man or beast,

    Please contemplate the choice;

    I write these lines as best I can

    For those who have no voice.”

    Michael Pearce, Three Little Pigs

  • “You are in a terrible accident. Your body is fatally injured, as are the brains of your two identical-triplet brothers. Your brain is divided into two halves, and into each brother’s body one half is successfully transplanted. After the surgery, each of the two resulting people believes himself to be you, seems to remember living your life, and has your character. (This is not as unlikely as it sounds: already, living brains have been surgically divided, resulting in two separate streams of consciousness.) What has happened? Have you died, or have you survived? And if you have survived who are you? Are you one of these people? Both? Or neither? What if one of the transplants fails, and only one person with half your brain survives? That seems quite different—but the death of one person could hardly make a difference to the identity of another. The philosopher Derek Parfit believes that neither of the people is you, but that this doesn’t matter. It doesn’t matter that you have ceased to exist, because what has happened to you is quite unlike ordinary death: in your relationship to the two new people there is everything that matters in ordinary survival—a continuity of memories and dispositions that will decay and change as they usually do. Most of us care about our future because it is ours—but this most fundamental human instinct is based on a mistake, Parfit believes. Personal identity is not what matters.”

    Larissa MacFarquhar, How to be Good (The New Yorker, September 5, 2011)

  • “There is no way anyone can completely control what his or her offspring does. Even a vegan who has children cannot guarantee that his or her child will be vegan for his or her entire life. It is even less likely that the grandchild of a vegan will be vegan, and yet more of a remote possibility that a great grandchild of a vegan will be vegan. If you are vegan and have just one child who is not vegan, you will have countered the positive effects of your entire life of veganism. [I]f you knew that having a child would result in even one animal being tortured and killed, would you knowingly get pregnant or get someone else pregnant?”

    anonymous, Vegan Parenting: Adoption vs Breeding (2012)

  • “Personal identity is not as deep, and simple, as most of us take it to be. Even if we did not know whether we would ever wake up again, we might know the full truth about what was going to happen. Since we have such false beliefs about what is involved in our continued existence, we may misunderstand the rational and moral importance of personal identity. On the true view, I claimed, though we have reasons for special concern about our future, these reasons are not given, as we assume, by the fact that this will be our future. Nor will our death be as significant as most of us believe. In my somewhat misleading slogan, personal identity is not what matters.”

    Derek Parfit, We Are Not Human Beings (2012)

  • “By adopting a child, you will not only be giving a needy child a good home, but you will also be able to have a positive influence on his or her life. Not adopting and insisting on having a biological child or children is not only detrimental to a child who needs a good home, it is detrimental to the thousands of animals that may die as a result.”

    anonymous, Vegan Parenting: Adoption vs Breeding (2012)

  • “At any rate it would mean that self-interest was not a rational motive for action. How could it be, if there is no “self” to have any interests? If there are no such beings are myself or others, there can be no reason to put my interests above those of others.”

    Eric Olson, What Are We? A Study in Personal Ontology (2007), p. 203

  • “It’s easy to change a child but hard to keep him from changing back. Instead of thinking of children as lumps of clay for parents to mold, we should think of them as plastic that flexes in response to pressure – and pops back to its original shape once the pressure is released.
    Bryan Caplan, Selfish Reasons to Have More Kids: Why Being a Great Parent is Less Work and More Fun Than You Think (2011), p. 5

  • “Consider the fact that, in a few years, I shall be dead. This fact can seem depressing. But the reality is only this. After a certain time, none of the thoughts and experiences that occur will be directly causally related to this brain, or be connected in certain ways to these present experiences. That is all this fact involves. And, in that description, my death seems to disappear.”

    Derek Parfit, The Unimportance of Identity (1995)

  • “One particularly poor argument in defence of eating meat is that if humans did not eat animals, those animals would not have been brought into existence in the first place. Humans would simply not have bred them in the numbers they do breed them. The claim is that although these animals are killed, this cost to them is outweighed by the benefit to them of having been brought into existence. This is an appalling argument for many reasons. First, the lives of many of these animals are so bad that even if one rejected my argument one would still have to think that they were harmed by being brought into existence. Secondly, those who advance this argument fail to see that it could apply as readily to human babies that are produced only to be eaten. Here we see quite clearly that being brought into existence only to be killed for food is no benefit. It is only because killing animals is thought to be acceptable that the argument is thought to have any force.”

    David Benatar, Better Never to Have Been: The Harm of Coming into Existence

  • “My death will break the more direct relations between my present experiences and future experiences, but it will not break various other relations.”

    Derek Parfit, Reasons and Persons (1984)

  • “This is all there is to the fact that there will be no one living who will be me. Now that I have seen this, my death seems to me less bad.”

    Derek Parfit, Reasons and Persons (1984)

  • “Some people will remember [Derek Parfit]. Others may be influenced by his writing, or act upon his advice. Memories that connect with his memories, thoughts that connect with his thoughts, actions taken that connect with his intentions, will persist after he is gone, just inside different bodies.”

    Larissa MacFarquhar, How to be Good (The New Yorker, September 5, 2011)

  • “A self, it seems, is not all or nothing but the sort of thing that there can be more of or less of. When, in the process of a zygote’s cellular self-multiplication, does a person start to exist? Or when does a person, descending into dementia or coma, cease to be? There is no simple answer—it is a matter of degrees.”

    Larissa MacFarquhar, How to be Good (The New Yorker, September 5, 2011)

  • “[T]he way we act toward future generations will be partly determined by our beliefs about what matters in life, and whether we believe that anything matters at all.”

    Larissa MacFarquhar on Derek Parfit’s views regarding the future

  • “You have just celebrated your 170th birthday and you feel stronger than ever. Each day is a joy. You have invented entirely new art forms, which exploit the new kinds of cognitive capacities and sensibilities you have developed. You still listen to music – music that is to Mozart what Mozart is to bad Muzak. You are communicating with your contemporaries using a language that has grown out of English over the past century and that has a vocabulary and expressive power that enables you to share and discuss thoughts and feelings that unaugmented humans could not even think or experience. You play a certain new kind of game which combines VR-mediated artistic expression, dance, humor, interpersonal dynamics, and various novel faculties and the emergent phenomena they make possible, and which is more fun than anything you ever did during the first hundred years of your existence. When you are playing this game with your friends, you feel how every fiber of your body and mind is stretched to its limit in the most creative and imaginative way, and you are creating new realms of abstract and concrete beauty that humans could never (concretely) dream of. You are always ready to feel with those who suffer misfortunes, and to work hard to help them get back on their feet. You are also involved in a large voluntary organization that works to reduce suffering of animals in their natural environment in ways that permit ecologies to continue to function in traditional ways; this involves political efforts combined with advanced science and information processing services. Things are getting better, but already each day is fantastic.”

    Nick Bostrom, Why I Want to be a Posthuman When I Grow Up

  • “No science and no analysis of the future consequences of various actions taken today can in itself tell us what to do. We need, in addition, to factor in what kind of future we value, and to what extent we care at all about the future compared to more immediate concerns here and now. The later aspect is usually modeled and economics by the so-called discount rate, which has played a prominent role in discussions of climate change on a decadal and centennial time scale, but hardly at all in the context of longer perspectives or the various radical technologies[.] We are less used to thinking about ethical issues on long time scales, so our intuitions trying to fail us and lead to paradoxes. These issues need to be resolved, because dodging the bullet would in my opinion be unacceptably irresponsible.”

    Olle Häggström, Here Be Dragons: Science, Technology and the Future of Humanity, Chapter “Science for good and science for bad”, p. 13

  • “As he searched the physics literature on the long-term future of the universe, Dyson noticed that the available papers on the subject shared a certain strange peculiarity. “The striking thing about these papers,” Dyson recalled afterward, “is that they are written in an apologetic or jocular style, as if the authors were begging us not to take them seriously.”

    It was not a proper use of your time, apparently, to imagine what might or might not happen to the universe some billions of years down the road – a prejudice that was rather surprising in view of the fact that many physicists nonetheless lavished huge amounts of recycled paper, time, and attention on what had happened billions of years in the past.”

    Ed Regis, Great Mambo Chicken and the Transhuman Condition: Science Slightly over the Edge, London (1991), p. 270

  • “A human life is a human life. If you have that moral point of view that future generations matter in proportion to their population numbers, then you get this very stark implication that existential risk mitigation has a much higher utility than pretty much anything else that you could do. There are so many people that could come into existence in the future if humanity survives this critical period of time – we might live for billions of years, our descendants might colonize billions of solar systems, and there could be billions and billions times more people than exist currently. Therefore, even a very small reduction in the probability of realizing this enormous good will tend to outweigh even immense benefits, [w]hich would be tremendous under ordinary standards.”

    Nick Bostrom, We’re Underestimating the Risk of Human Extinction (The Atlantic, 2012)

  • “The future is not the realization of our hopes and dreams, a warning to mend our ways, an adventure to inspire us, nor a romance to touch our hearts. The future is just another place in spacetime.”

    Robin Hanson, The Rapacious Hardscrapple Frontier (Year Million: Science at the Far Edge of Knowledge, 2008, p. 168)

  • “One valuable asset [for reducing existential risk] would be a donor network comprising individuals devoted to rational philanthropy, informed about existential risk, and discerning about the means of mitigation. [I]t could be worth foregoing some technical advances in the short term in order to fill the ranks with individuals who genuinely care about safety and who have a truth-seeking orientation (and who are likely to attract more of their own kind).”

    Nick Bostrom, Superintelligence: Paths, Strategies and Dangers (2014), Chapter 15: Crunch Time

  • “If the time horizon is extremely short, the benefits of continued higher growth will be choked off and will tend to be small in nature. Even if we hold a deep concern for the distant future, perhaps there is no distant future to care about. To present this point in its starkest form, imagine that the world were set to end tomorrow. There would be little point in maximizing the growth rate, and arguably we should just throw a party and consume what we can. Even if we could boost growth in the interim hours, the payoff would be small and not very durable. The case for growth maximization therefore is stronger the longer the time horizon we consider.”

    Tyler Cowen, Caring about the Distant Future: Why it Matters and What it Means (2007), p. 28

  • “But there is a huge difference between, on one hand, admitting that there are severe difficulties, and, on the other, throwing our hands in the air and fatalistically declaring the problem to be unsolvable. We don’t know that they are in solvable until we tried, and tried really hard. Given the magnitude of what’s at stake, just giving up on the problem is in my opinion unacceptable. The extent to which we are currently neglecting the problem is shocking. Nick Bostrom, in a recent paper, illustrates this with a diagram showing how the number of academic publications on snowboarding outnumbers those on risks of human extinction by a factor of 20 or so, while those on dung beetles beat those on snowboarding by another factor of 2. This should not be taken as a suggestion that too much effort is spent on academic studies of snowboarding and dung beetles, but rather as an indication that current efforts into the study of existential risks to humanity could easily be significantly scaled up without major destruction to the current academic landscape as a whole.”

    Olle Häggström, Here Be Dragons: Science, Technology and the Future of Humanity, Chapter “Science for good and science for bad”, p. 10

  • “Einstein’s theory of relativity suggests that there is no fact of the matter as to when “now” is. Any measurement of time is relative to the perspective of an observer. In other words, if you are traveling very fast, the clocks of others are speeding up from your point of view. You will spend a few years in a spaceship but when you return to earth thousands or millions of years will have passed. Yet it seems odd, to say the least, to discount the well-being of people as their velocity increases. Should we pay less attention to the safety of our spacecraft, and thus the welfare of our astronauts, the faster those vehicles go? If, for instance, we sent off a spacecraft at near the velocity of light, the astronauts would return to earth, hardly aged, millions of years hence. Should we – because of positive discounting – not give them enough fuel to make a safe landing? And if you decline to condemn them to death, how are they different from other “residents” in the distant future?”

    Tyler Cowen, Caring about the Distant Future: Why it Matters and What it Means (2007), p. 10

  • “When you look at human character, it’s hard to be confident humans will survive. To me, it’s crazy to be confident, I have to say. To think that it’s highly likely we will survive nuclear weapons, climate change, artificial intelligence, genetic engineering, biological warfare… to be confident like that is to be either totally ignorant of the nature of humanity – which most people are – or to be crazy.

    So, thinking it’s likely we’ll survive? I can’t believe that. I think it’s unlikely, very unlikely. But not impossible. My age and experience doesn’t permit me to be confident that there’s no way out here. Because humans are adaptable, and things do change, and the changes I’ve mentioned are possible.

    We are on the Titanic, going at full speed on a moonless night into iceberg waters. Have we hit the iceberg yet, and made it inevitable that we will all go down together? We don’t know. It may turn out that, a while ago, we went past the no-return point. But we don’t know that, there’s no way to prove it. As I say in The Doomsday Machine: “I act as if we have a chance to find our way out of this. I don’t know what that path is yet, but that doesn’t tell me there is no way.”

    So, I urge others, I encourage them. And if they give up hope, or even devote themselves entirely to pleasure, like a life on the Titanic drinking champagne after hitting the iceberg… I can’t say that’s crazy. But I don’t join in that. And should someone stop trying to save the world as a whole, and instead just works to ease the suffering of other people – I think that’s very reasonable, very good. I just think that it’s definitely not wasted for some of us to keep trying to explore and see if there’s a way out of the precarious situation in which humanity finds itself.”

    Daniel Ellsberg on The 80,000 Hours Podcast with Rob Wiblin episode #43 (2018)

  • “We live during the hinge of history. Given the scientific and technological discoveries of the last two centuries, the world has never changed as fast. We shall soon have even greater powers to transform, not only our surroundings, but ourselves and our successors. If we act wisely in the next few centuries, humanity will survive its most dangerous and decisive period. Our descendants could, if necessary, go elsewhere, spreading through this galaxy.”

    Derek Parfit, On What Matters: Volume 2 (2011), p. 616

  • “The Earth may remain habitable for at least a billion years. What has occurred so far is at most a tiny fraction of possible human history. Nor should we restrict this question to the lives of future human beings. just as we had ancestors who were not human, we may have descendants who will not be human… Our descendants might, I believe, make the further future very good.”

    Derek Parfit, On What Matters: Volume 2 (2011), p. 616

  • “What now matters most is that we avoid ending human history. If there are no rational beings elsewhere, it may depend on us and our successors whether it will all be worth it.”

    Derek Parfit, On What Matters: Volume 2 (2011), p. 620

  • “What now matters most is that we avoid ending human history. If there are no rational beings elsewhere, it may depend on us and our successors whether it will all be worth it, because the existence of the Universe will have been on the whole good.”

    Derek Parfit, On What Matters: Volume Two (2011), p. 620

  • “When Parfit thinks about the future, he wonders whether life for future people will be better or worse than it is now. He wants to be optimistic, but he cannot ignore the terrible suffering that people have endured in the past. Has it all been worth it? Has the sum of human happiness outweighed the sum of suffering?”

    Larissa MacFarquhar, How to be Good (The New Yorker, September 5, 2011)

  • “I am weakly inclined to believe that the past has been in itself worth it. But this may be wishful thinking.”

    Derek Parfit, On What Matters: Volume Two (2011), p. 612

  • “People often assume the long-term value thesis is especially about the possibility of there being lots of people in the future, and so only of interest to a narrow range of ethical views (especially utilitarian totalism), but as we can see in the list above, it’s actually much broader. It just rests on the idea that if something is of value, it’s better to have more of what’s valuable rather than less, and that it’s possible to have much more of it in the future. This might include non-welfare values, such as beauty or knowledge. The arguments are also not about humans; rather, they concern whatever agents in the future might have moral value, including other species.”

    80,000 Hours, If you want to do good, here’s why future generations should be your focus (2017)

  • “[Derek Parfit] sees that we have the ability to make the future much better than the past, or much worse, and he knows that he will not live to discover which turns out to be the case. He knows that the way we act toward future generations will be partly determined by our beliefs about what matters in life, and whether we believe that anything matters at all. This is why he continues to try so desperately to prove that there is such a thing as moral truth.”

    Larissa MacFarquhar, How to be Good (The New Yorker, September 5, 2011)

  • “A simple thought experiment suggests that humans are earth-life’s best bet. In this experiment there are three key factors: the probability that humans can avoid extinction and transcend oblivion; the probability that new intelligent life would re-evolve if humans became extinct; and the probability that a newly evolved intelligent species could avoid its own extinction and transcend oblivion, assuming there is enough time to do so. To favour extinction of humans, the product of the second and third probabilities must be greater than the first probability.”

    Bruce Tonn, Futures Sustainability (November, 2007)

  • “The stupendous time spans of the evolutionary past are not part of common culture–except among some creationists and fundamentalists. But most educated people, even if they are fully aware that our emergence took billions of years, somehow think we humans are the culmination of the evolutionary tree. That is not so. Our Sun is less than half way through its life. It is slowly brightening, but Earth will remain habitable for another billion years. However, even in that cosmic perspective – extending far into the future as well as into the past – the twenty-first century may be a defining moment. It is the first in our planet’s history where one species – ours – has Earth’s future in its hands and could jeopardise not only itself but also life’s immense potential.”

    Sir Martin Rees, Foreword (Global Catastrophic Risks, 2008)

  • “Without ‘ethical culture,’ there is no salvation for humanity.”

    Albert Einstein

  • “[O]nce space travel begins, there are, in principle, no further physical barriers to prevent Homo sapiens (or our descendants) from eventually expanding to colonize a substantial portion, if not all, of the visible Cosmos. Once this has occurred, it becomes quite reasonable to speculate that the operations of all these intelligent beings could begin to affect the large scale evolution of the Universe. If this is true, it would be in this era – in the far future near the Final State of the Universe – that the true significance of life and intelligence would manifest itself. Present-day life would then have cosmic significance because of what future life may someday accomplish.”

    John Barrow, Frank Tipler, The Anthropic Cosmological Principle, Oxford, 1986, p. 614

  • “A human being is part of the whole, called by us ‘Universe’; a part limited in time and space. He experiences himself, his thoughts and feelings as something separated from the rest — a kind of optical delusion of his consciousness. This delusion is a kind of prison for us, restricting us to our personal desires and affection for a few persons nearest us. Our task must be to free ourselves from this prison by widening our circle of compassion to embrace all living creatures and the whole of nature in its beauty. Nobody is able to achieve this completely but striving for such achievement is, in itself, a part of the liberation and a foundation for inner security.”

    — Albert Einstein

    “In the littered field of discredited self-congratulatory chauvinisms, there is only one that seems to hold up, one sense in which we are special: Due to our own actions or inactions, and the misuse of our technology, we live at an extraordinary moment, for the Earth at least-the first time that a species has become able to wipe itself out. But this is also, we may note, the first time that a species has become able to journey to the planets and the stars. The two times, brought about by the same technology, coincide – a few centuries in the history of a 4.5-billion-year-old planet. If you were somehow dropped down on the Earth randomly at any moment in the past (or future), the chance of arriving at this critical moment would be less than 1 in 10 million. Our leverage on the future is high just now.”

    Carl Sagan, Pale Blue Dot: A Vision of the Human Future in Space (1994), p. 305

  • “We love life here every instant. Every second is so good that it would blow your mind had its amperage not been previously increased. My contemporaries and I bear witness, and we are requesting your aid. Please, help us come into existence! Please, join us! Whether this tremendous possibility becomes a reality depends on your actions. If your empathy can perceive at least the outlines of the vision I am describing, then your ingenuity will find a way to make it real. Human life, at its best, is fantastic. I’m asking you to create something even greater. Life that is truly humane.”

    Nick Bostrom, Letter from Utopia (2010)

  • “The fact that no limits exist to the destructiveness of this weapon makes its very existence and the knowledge of its construction a danger to humanity… It is… an evil thing.”

    Enrico Fermi, I.I. Rabi, personal addendum to the Report of the General Advisory Committee

  • “A very large nuclear war would be a calamity of indescribable proportions and absolutely unpredictable consequences, with the uncertainties toward the worse… All-out nuclear war would mean the destruction of contemporary civilization, throw man back centuries, cause the deaths of hundreds of millions or billions of people, and, with a certain degree of probability, would cause man to be destroyed as a biological species…”

    Andrei Sakharov (Foreign Affairs, Summer 1983)

  • “[E]ven if average future periods were only about equally as good as the current period, the whole of the future would be about a trillion times more important, in itself, than everything that has happened in the last 100 years.”

    Nick Beckstead, On the Overwhelming Importance of Shaping the Far Future (2013), p. 67

  • ““Now” is only a fraction of possible humans that could leave pretty good lives compared to the future.”

    Toby Ord, Effective Altruism Global 2015: Oxford University, Panel Discussion “Who ought we help?”

  • “The theoretical blind spot of current philosophy of mind is the issue of conscious suffering: thousands of pages are being written about color qualia or the contents of thought, but almost no theoretical work is devoted to ubiquitous phenomenal states like physical pain or simple everyday sadness (“subclinical depression”). [T]he ethical/normative issue is of greater relevance. If one dares to take a closer look at the actual phenomenology of biological systems on our planet, the many different kinds of conscious suffering are at least as dominant a feature as are color vision or conscious thought, both of which appeared only very recently. Evolution is not something to be glorified. One way – out of countless others – to look at biological evolution on our planet is as a process that has created an expanding ocean of suffering and confusion where there previously was none. As not only the simple number of individual conscious subjects, but also the dimensionality of their phenomenal state-spaces is continuously increasing, this ocean is also deepening. For me, this is also a strong argument against creating artificial consciousness: we shouldn’t add to this terrible mess before we have truly understood what actually is going on here.”

    Thomas Metzinger, A Self Worth Having: A Talk with Nicholas Humphrey (Edge.org)

  • “If we should not discriminate over space, we shouldn’t discriminate through time. If every life is valued the same, why not also worry about future lives?”

    Andrew Snyder-Beattie, Effective Altruism Global 2015: Oxford University, Workshop “Existential risk”

  • “If we take for granted that consciousness evolved, consciousness would somehow have to promote survival and reproduction in order to be selected for. If consciousness did not promote survival and reproduction, it would not be selected for, and to the extent that it were biologically costly, it would be selected against. The only way consciousness could promote survival and reproduction, moreover, is by virtue of guiding an organism’s actions, prompting it to perform survival and reproduction enhancing actions – and the only way in which consciousness could prompt an organism towards survival and reproduction seems to be by imbuing experiences with a certain valence or a pro/con attitude. Without a valence or a pro/con attitude, it is unclear how an experience would be able to guide an organism’s actions. Evolution, moreover, cares for action, not for experiences as an end in itself. It therefore seems that if consciousness were to ever get going, valence would have to be present from the very start. Otherwise, consciousness would disappear as fast as it occurred. This suggests that hedonic valence phylogenetically is as old as consciousness itself, which in turn lends support to the view that hedonic valence lies at the heart of consciousness. This supports dimensionalism, moreover, since according to dimensionalism, pleasure and pain – rather than being two things out of the many things we can experience – imbues all [o]ur experiences. Indeed, one might, from a dimensionalist approach to consciousness, argue that the first experience any organism ever had was an experience of either pleasure or pain, and that consciousness of the kind our species has today is a more fine-grained version of something that is most fundamentally a pleasure/pain mechanism.”

    Ole Martin Moen, The Unity and Commensurability of Pleasures and Pains (2013)

  • “[I]f you suppose we can discount welfare, we can easily end up with conclusions that sound absurd. For instance, a 3% discount rate would imply that the suffering of one person today was equal to the suffering of 16 trillion people in 1000 years.”

    80,000 Hours, If you want to do good, here’s why future generations should be your focus (2017)

  • “Suppose we who are living now decide to ignore global warming, with the result that the lives of future people are much harder. It would seem that we have made things worse for those future people. But, in fact, as long as their lives are worth living this is not the case—because if we had acted differently, the world would have been different, and those particular people would never have existed (in the same way that if cars had not been invented most people alive today would never have been born). So, although we have made the world worse in the future, we have made life worse for no one. Parfit calls this conundrum the Non-Identity Problem. He believes that it makes no difference: we still have just as much reason to avoid making life worse for people in the future. But he worries—rightly, as it turns out—that other people may draw the opposite conclusion: since global warming will not make particular future people worse off, it may seem less bad.”

    Larissa MacFarquhar, How to be Good (The New Yorker, September 5, 2011)

  • “Happiness or misery are no better and no worse in the year 10,000 B. C. than in the year 10,000 A. D. If they are, then there is no reason why they are not better or worse on Wednesdays than on Thursdays.”

    James MacKaye, The Politics of Utility (1909), p. 39

  • “Parfit has always been preoccupied with how to think about our moral responsibilities toward future people. It seems to him the most important problem we have. Besides the issue of global warming, there is the issue of population. It would seem that if the earth were teeming with many billions of people, making everyone’s life worse, that would be bad. But what if the total sum of human happiness would be higher with many billions of people whose lives were barely worth living—higher, that is, than with a smaller population of well-off people? Wouldn’t the first situation be, in some moral sense, better? Parfit calls this the Repugnant Conclusion. It seems absurd, but, at least for a consequentialist, its logic is difficult to counter.”

    Larissa MacFarquhar, How to be Good (The New Yorker, September 5, 2011)

  • “Why should costs and benefits receive less weight, simply because they are further in the future? When the future comes, these benefits and costs will be no less real. Imagine finding out that you, having just reached your twenty-first birthday, must soon die of cancer because one evening Cleopatra wanted an extra helping of dessert. How could this be justified?”

    Tyler Cowen, Derek Parfit, Against the Social Discount Rate (1992), p. 145

  • “Numbers can feel “cold” because of the collapse of compassion we experience when we think of “165 billion animals” — the approximate number of animals suffering in factory farms right now. The limits of our imaginations make it impossible for us to see 165,000,000,000 individuals who are each as complete in our minds as individuals we know personally. We are instead compelled to just see “a number” — a completely reductive representation of those 165,000,000,000 individuals. But each and every number in that sum represents an entire individual with a personality and unique experiences, and most of them are suffering tremendously. We care about all 165,000,000,000 because we care about each one. Ultimately, a full application of our empathy requires us to consider the number of individuals affected. Far from “cold and calculating”, comparing suffering is better described as warm and calculating. It would be much colder to risk abandoning many individuals who we could help.”

    Sentience Politics, Altruism, Numbers, and Factory Farms

  • “[Several theories of ethics or rationality claim that], in deciding what would be best for someone, we should give equal weight to all the parts of this person’s future. Later events may be less predictable; and a predictable event should count for less if it is less likely to happen. But it should not count for less merely because, if it happens, it will happen later.”

    Derek Parfit, Reasons and Persons (1984), p. 4

  • “Scope neglect is the hazard of being a biological human, running on an analog brain; the brain cannot multiply by six billion. And the stakes of existential risk extend beyond even the six billion humans alive today, to all the stars in all the galaxies that humanity and humanity’s descendants may some day touch. All that vast potential hinges on our survival here, now, in the days when the realm of humankind is a single planet orbiting a single star. I can’t feel our future. All I can do is try to defend it.”

    Eliezer Yudkowsky, Cognitive Biases Potentially Affecting Judgement of Global Risks (2008)

  • “[T]he human mind is used to thinking in terms of decades or perhaps generations, not the hundreds of millions of years that is the time frame for life on Earth. Coming to grips with humanity in this context reveals at once our significance in Earth history, and our insignificance. There is a certainty about the future of humanity that cheats our mind’s comprehension: one day our species will be no more.”

    Richard Leakey, Roger Lewin, The Sixth Extinction: Patterns of Life and the Future of Humankind (1995), p. 224

  • “The Spanish flu of 1918 killed 25-50 million people. World War II killed 60 million people; 10^{7} is the order of the largest catastrophes in humanity’s written history. Substantially larger numbers, such as 500 million deaths, and especially qualitatively different scenarios such as the extinction of the entire human species, seem to trigger a different mode of thinking – enter into a ‘separate magisterium’. People who would never dream of hurting a child hear of an existential risk, and say, ‘Well, maybe the human species doesn’t really deserve to survive.’”

    Eliezer Yudkowsky, Cognitive Biases Potentially Affecting Judgement of Global Risks (2008)

  • “I believe that if we destroy mankind, as we now can, this outcome will be much worse than most people think. Compare three outcomes:

    (1) Peace. (2) A nuclear war that kills 99% of the world’s existing population. (3) A nuclear war that kills 100%.

    (2) would be worse than (1), and (3) would be worse than (2). Which is the greater of these two differences? Most people believe that the greater difference is between (1) and (2). I believe that the difference between (2) and (3) is very much greater. [T]he Earth will remain habitable for at least another billion years. Civilization began only a few thousand years ago. If we do not destroy mankind, these few thousand years may be only a tiny fraction of the whole of civilized human history. The difference between (2) and (3) may thus be the difference between this tiny fraction and all of the rest of this history. If we compare this possible history to a day, what has occurred so far is only a fraction of a second.”

    Derek Parfit, Reasons and Persons (1984), p. 453

  • “The line between cynicism and misanthropy—between thinking ill of human motives and thinking ill of humans—is often blurry.”

    Kevin Simler & Robin Hanson, The Elephant in the Brain: Hidden Motives in Everyday Life (2018)

  • “Some have argued that the difference between the deaths of several hundred million people in a nuclear war (as has been thought until recently to be a reasonable upper limit) and the death of every person on Earth (as now seems possible) is only a matter of one order of magnitude. For me, the difference is considerably greater. Restricting our attention only to those who die as a consequence of the war conceals its full impact.

    If we are required to calibrate extinction in numerical terms, I would be sure to include the number of people in future generations who would not be born. A nuclear war imperils all of our descendants, for as long as there will be humans. Even if the population remains static, with an average lifetime of the order of 100 years, over a typical time period for the biological evolution of a successful species (roughly ten million years), we are talking about some 500 trillion people yet to come. By this criterion, the stakes are one million times greater for extinction that for the more modest nuclear wars that kill “only” hundreds of millions of people.

    There are many other possible measures of the potential loss – including culture and science, the evolutionary history of the planet, and the significance of the lives of all of our ancestors who contributed to the future of their descendants. Extinction is the undoing of the human enterprise.”

    Carl Sagan, Nuclear War and Climatic Catastrophe: Some Policy Implications (1984), p. 27

  • “[P]erhaps surprisingly, [geoengineering expert David Keith] says in an interview in The Atlantic that “the most disastrous thing that could happen would be for Barack Obama to stand up tomorrow and announce the creation of a geo-engineering task force with hundreds of millions in funds”. Why, then, does he expressed this modest or even negative view towards funding his own research area? The answers to be found in the concept of moral hazard. This is the phenomenon where, e.g., someone who buys insurance may feel tempted towards riskier behavior than otherwise. In the present case, the moral hazard is that the more attention geoengineering gets (as, for instance, in a high-profile announcement from the American president), the greater will the temptation be for decision-makers and voters to think that there is no big hurry to cut down on greenhouse gas emissions, because if worst comes to worst we will surely be able to work things out with in one geoengineering scheme or another.”

    Olle Häggström, Here Be Dragons: Science, Technology and the Future of Humanity, Chapter “Our planet and its biosphere”, pp. 32-33

  • “Due to the psychological phenomenon known as hyperbolic discounting, we are not always in agreement with our future selves regarding what these should do. This is why we lock away our money in pension funds and install time locks on our refrigerators. Similarly, I may well be in a position where I, Olle Häggström_today, cannot resist the beef for tonight’s dinner, but feel that Olle Häggström_next week really ought to switch to vegetarianism, while realizing that he (just like me) will most likely be too weak to resist the beef offered on the menu.”
    Olle Häggström, Here Be Dragons: Science, Technology and the Future of Humanity, Chapter “Our planet and its biosphere”, p. 36

  • “It is not even impossible to imagine that the effects of an atomic war fought with greatly perfected weapons and pushed by the utmost determination will endanger the survival of man.”

    Edward Teller (Bulletin of the Atomic Scientists, February 1947)

  • “The extreme danger to mankind inherent in the proposal by [Edward Teller and others to develop thermonuclear weapons] wholly outweighs any military advantage.”

    Robert J. Oppenheimer et al, Report of the General Advisory Committee (October 1949)

  • “We are all animals of this planet. We are all creatures. And nonhuman animals experience pain sensations just like we do. They too are strong, intelligent, industrious, mobile, and evolutional. They too are capable of growth and adaptation. Like us, firsthand foremost, they are earthlings. And like us, they are surviving. Like us they also seek their own comfort rather than discomfort. And like us they express degrees of emotion. In short like us, they are alive.”

    Joaquin Phoenix

  • “It is not the strongest of the species that survive, nor the most intelligent, but the one most responsive to change. “

    Charles Darwin

  • “”I have a dream,” said Harry’s voice, “that one day sentient beings will be judged by the patterns of their minds, and not their color or their shape or the stuff they’re made of, or who their parents were. Because if we can get along with crystal things someday, how silly would it be not to get along with Muggleborns, who are shaped like us, and think like us, as alike to us as peas in a pod? The crystal things wouldn’t even be able to tell the difference. How impossible is it to imagine that the hatred poisoning Slytherin House would be worth taking with us to the stars? Every life is precious, everything that thinks and knows itself and doesn’t want to die. Lily Potter’s life was precious, and Narcissa Malfoy’s life was precious, even though it’s too late for them now, it was sad when they died. But there are other lives that are still alive to be fought for. Your life, and my life, and Hermione Granger’s life, all the lives of Earth, and all the lives beyond, to be defended and protected, EXPECTO PATRONUM!

    And there was light.””

    Eliezer Yudkowsky, Harry Potter and the Methods of Rationality, Chapter 47: Personhood Theory

  • “In the distant future I see open fields for far more important researches. Psychology will be based on a new foundation, that of the necessary acquirement of each mental power and capacity by gradation. Light will be thrown on the origin of man and his history.”

    Charles Darwin, The Origin of Species (1859)

  • “Some moral philosophers try to thread a boundary across this treacherous landscape by equating personhood with cognitive traits that humans happen to possess. These include an ability to reflect upon oneself as a continuous locus of consciousness, to form and savor plans for the future, to dread death, and to express a choice not to die. At first glance the boundary is appealing because it puts humans on one side and animals and conceptuses on the other. But it also implies that nothing is wrong with killing unwanted newborns, the senile, and the mentally handicapped, who lack the qualifying traits. Almost no one is willing to accept a criterion with those implications. There is no solution to these dilemmas, because they arise out of a fundamental incommensurability: between our intuitive psychology, with its all-or-none concept of a person or soul, and the brute facts of biology, which tell us that the human brain evolved gradually, develops gradually, and can die gradually. And that means that moral conundrums such as abortion, euthanasia, and animal rights will never be resolved in a decisive and intuitively satisfying way. This does not mean that no policy is defensible and that the whole matter should be left to personal taste, political power, or religious dogma. As the bioethicist Ronald Green has pointed out, it just means we have to reconceptualize the problem: from finding a boundary in nature to choosing a boundary that best trades off the conflicting goods and evils for each policy dilemma. We should make decisions in each case that can be practically implemented, that maximize happiness, and that minimize current and future suffering.”

    Steven Pinker, The Blank Slate (2002): Chapter 13, Out of Our Depths, p. 228

  • “We have seen that the senses and intuitions, the various emotions and faculties, such as love, memory, attention and curiosity, imitation, reason, etc., of which man boasts, may be found in an incipient, or even sometimes a well-developed condition, in the lower animals.”

    Charles Darwin, The Descent of Man

  • “I think that wide exposure to thoughtful science fiction clearly reveals that the moral status of AIs should be guided entirely by the psychological and social properties of the AIs and not by facts about their material architecture, species membership, bodily shape, or manufactured origin, except insofar as the latter facts influence their psychological and social properties.”

    Eric Schwitzgebel, An interview with Eric Schwitzgebel and Mara Garza (People for the Ethical Treatment of Reinforcement Learners, Nov 13, 2015)

  • “The love for all living creatures is the most noble attribute of man.”

    Charles Darwin, The Descent of Man

  • “If we continue to develop sophisticated forms of artificial intelligence, we have a moral obligation to improve our understanding of the conditions under which artificial consciousness might genuinely emerge. Otherwise we risk moral catastrophe – either the catastrophe of sacrificing our interests for beings that don’t deserve moral consideration because they experience happiness and suffering only falsely, or the catastrophe of failing to recognise robot suffering, and so unintentionally committing atrocities tantamount to slavery and murder against beings to whom we have an almost parental obligation of care.

    We have, then, a direct moral obligation to treat our creations with an acknowledgement of our special responsibility for their joy, suffering, thoughtfulness and creative potential. But we also have an epistemic obligation to learn enough about the material and functional bases of joy, suffering, thoughtfulness and creativity to know when and whether our potential future creations deserve our moral concern.”

    Eric Schwitzgebel, We have greater moral obligations to robots than to humans (Aeon, November 12, 2015)

  • “If your neurons were to be replaced one-by-one, by functionally-equivalent silicon chips, is there some magical moment at which your consciousness would be extinguished? And if a computer passes the Turing test—well, one way to think about the Turing test is that it’s just a plea against discrimination. We all know it’s monstrous to say, ‘this person seems to have feelings, seems to be eloquently pleading for mercy even, but they have a different skin color, or their nose is a funny shape, so their feelings don’t count.’ So, if it turned out that their brain was made out of semiconductors rather than neurons, why isn’t that fundamentally similar?”

    Scott Aaronson, Can Computers Become Conscious?