“One reason for focusing on intelligence, that is, on instrumental rationality, is that this is the most relevant concept if we are trying to figure out what different kinds of systems would do. Normative questions, such as whether their behavior would count as being prudentially rational or morally justifiable, can be important in various ways. However, such questions should not blind us to the possibility of cognitive systems that fail to satisfy substantial normative criteria but which are nevertheless very powerful and able to exert strong influence on the world.”

— Nick Bostrom, The Superintelligent Will: Motivation and Instrumental Rationality in Advanced Artificial Agents (2012)