"A genie is a command-executing system: it receives a high-level command, carries it out, then pauses to await the next command. A sovereign is a system that has an open-ended mandate to operate in the world in pursuit of broad and possibly very long-range objectives. Although these might seem like radically different templates for what a superintelligence should be and do, the difference is not as deep as it might at first glance appear." (Nick Bostrom, "Superintelligence", 2014)
"A question distinct from, but related to, the question of kinetics is whether there will be one superintelligent power or many? Might an intelligence explosion propel one project so far ahead of all others as to make it able to dictate the future? Or will progress be more uniform, unfurling across a wide front, with many projects participating but none securing an overwhelming and permanent lead?" (Nick Bostrom, "Superintelligence", 2014)
"A system that has the intelligence amplification superpower could use it to bootstrap itself to higher levels of intelligence and to acquire any of the other intellectual superpowers that it does not possess at the outset. But using an intelligence amplification superpower is not the only way for a system to become a full-fledged superintelligence. A system that has the strategizing superpower, for instance, might use it to devise a plan that will eventually bring an increase in intelligence (e.g. by positioning the system so as to become the focus for intelligence amplification work performed by human programmers and computer science researchers)." (Nick Bostrom, "Superintelligence", 2014)
"A system might thus greatly boost its effective intellectual capability by absorbing pre-produced content accumulated through centuries of human science and civilization: for instance, by reading through the internet. If an AI reaches human level without previously having had access to this material or without having been able to digest it, then the AI’s overall recalcitrance will be low even if it is hard to improve its algorithmic architecture." (Nick Bostrom, "Superintelligence", 2014)
"An agent’s ability to shape humanity’s future depends not only on the absolute magnitude of the agent’s own faculties and resources - how smart and energetic it is, how much capital it has, and so forth - but also on the relative magnitude of its capabilities compared with those of other agents with conflicting goals." (Nick Bostrom, "Superintelligence", 2014)
"If some day we build machine brains that surpass human brains in general intelligence, then this new superintelligence could become very powerful. And, as the fate of the gorillas now depends more on us humans than on the gorillas themselves, so the fate of our species would depend on the actions of the machine superintelligence." (Nick Bostrom, "Superintelligence: Paths, Dangers, Strategies", 2014)
"Improvements in rationality and intelligence will tend to improve an agent’s decision-making, rendering the agent more likely to achieve its final goals. One would therefore expect cognitive enhancement to emerge as an instrumental goal for a wide variety of intelligent agents. For similar reasons, agents will tend to instrumentally value many kinds of information." (Nick Bostrom, "Superintelligence: Paths, Dangers, Strategies", 2014)
"Once machines attain some form of human-equivalence in general reasoning ability, how long will it then be before they attain radical superintelligence? Will this be a slow, gradual, protracted transition? Or will it be sudden, explosive?" (Nick Bostrom, "Superintelligence: Paths, Dangers, Strategies", 2014)
"Some paths to superintelligence require great resources and are therefore likely to be the preserve of large well-funded projects. Whole brain emulation, for instance, requires many different kinds of expertise and lots of equipment. Biological intelligence enhancements and brain–computer interfaces would also have a large scale factor: while a small biotech firm might invent one or two drugs, achieving superintelligence along one of these paths (if doable at all) would likely require many inventions and many tests, and therefore the backing of an industrial sector or a well-funded national program. Achieving collective superintelligence by making organizations and networks more efficient requires even more extensive input, involving much of the world economy." (Nick Bostrom, "Superintelligence: Paths, Dangers, Strategies", 2014)
"Suppose that a digital superintelligent agent came into being, and that for some reason it wanted to take control of the world: would it be able to do so?" (Nick Bostrom, "Superintelligence: Paths, Dangers, Strategies", 2014)
"The fact that there are many paths that lead to superintelligence should increase our confidence that we will eventually get there. If one path turns out to be blocked, we can still progress." (Nick Bostrom, "Superintelligence: Paths, Dangers, Strategies", 2014)
"Various considerations thus point to an increased likelihood that a future power with superintelligence that obtained a sufficiently large strategic advantage would actually use it to form a singleton. The desirability of such an outcome depends, of course, on the nature of the singleton that would be created and also on what the future of intelligent life would look like in alternative multipolar scenarios." (Nick Bostrom, "Superintelligence: Paths, Dangers, Strategies", 2014)
"We can tentatively define a superintelligence as any intellect that greatly exceeds the cog‐ nitive performance of humans in virtually all domains of interest." (Nick Bostrom, "Superintelligence: Paths, Dangers, Strategies", 2014)
"We have seen that a superintelligence could have a great ability to shape the future according to its goals. But what will its goals be? What is the relation between intelligence and motivation in an artificial agent?" (Nick Bostrom, "Superintelligence: Paths, Dangers, Strategies", 2014)
"[...] we use the term 'superintelligence' to refer to intellects that greatly outperform the best current human minds across many very general cognitive domains. This is still quite vague. Different kinds of system with rather disparate performance attributes could qualify as superintelligences under this definition. To advance the analysis, it is helpful to disaggregate this simple notion of superintelligence by distinguishing different bundles of intellectual super-capabilities. There are many ways in which such decomposition could be done. Here we will differentiate between three forms: speed superintelligence, collective superintelligence, and quality superintelligence." (Nick Bostrom, "Superintelligence: Paths, Dangers, Strategies", 2014)
No comments:
Post a Comment