“An instructive and oft-repeated example introduced by Bostrom (2003c) is the paperclip maximizer. The seed AI is given the goal of producing as many paperclips as it can. Once this results in a superintelligent AGI, the machine is likely to find ways to transform most of our planet into a monstrous heap of paperclips, followed by a similar transformation of the entire solar system, and probably the Milky Way, and most of the observable universe. Such a scenario will look very unappetizing to us humans, and as soon as we realize what the machine is up to we would try to do everything in our power to stop it. But we’re up against someone who is so much more intelligent than we are that our chances of succeeding are microscopic. Perhaps most likely, before we even have the time to think about how to organize our resistance, the machine will have realized what we might be up to, and exterminated us simply as a safety precaution.”

— Olle Häggström, Here Be Dragons: Science, Technology and the Future of Humanity, Chapter “Computer revolution”, p. 116