返回首页

NickBostrom_2015-_当电脑比人类聪明时会发生什么_
I work with a bunch of mathematicians , philosophers and computer scientists, and we sit around and think about the future of machine intelligence, among other things. 我和一群数学家、 哲学家、及电脑科学家一起工作。 我们坐在一起思考机器智慧的未来。 以及其他问题。
a bunch of:一群;一束;一堆; mathematicians:n.[数]数学家(mathematician的复数形式); philosophers:n.哲学家(philosopher的复数); sit around:无所事事;
Some people think that some of these things are sort of science fiction-y, far out there, crazy. 有些人可能认为这是科幻小说的范畴, 离我们很遥远,很疯狂。
But I like to say, okay, let's look at the modern human condition. 但是我要说, 好,我们来看看现代人类的状况....
(Laughter) (观众笑声)
This is the normal way for things to be. 这是人类的常态。
But if we think about it, we are actually recently arrived guests on this planet,the human species . 但如果我们仔细想想, 其实人类是刚刚才抵达地球的访客
recently:adv.最近;新近; species:n.[生物]物种;种类;
Think about if Earth was created one year ago, the human species, then, would be 10 minutes old. 假设地球在一年前诞生, 人类这个物种则仅存在了10分钟。
The industrial era started two seconds ago. 工業革命在2秒钟前开始。
industrial:adj.工业的,产业的; n.工业股票; era:n.新时期;纪元;年代;新时代;
Another way to look at this is to think of world GDP over the last 10,000 years, 另外一个角度是看看这一万年来的GDP增长
I've actually taken the trouble to plot this for you in a graph. 我花了时间作了张图表,
plot:n.情节;阴谋;布局;小块土地;v.密谋;暗中策划;(在地图上)标出;绘制(图表);
It looks like this. 它长这个样子
(Laughter) (观众笑声)
It's a curious shape for a normal condition. 对一个正常的状态来说这是个很有趣的形状。
curious:adj.好奇的,有求知欲的;古怪的;爱挑剔的;
I sure wouldn't want to sit on it. 我可不想要坐在上面。
(Laughter) (观众笑声)
Let's ask ourselves, what is the cause of this current anomaly ? 我们不禁问自己:「是什么造成了这种异态呢?」
anomaly:n.异常;不规则;反常事物;
Some people would say it's technology . 有些人会说是科技
technology:n.技术;工艺;术语;
Now it's true, technology has accumulated through human history, and right now, technology advances extremely rapidly -- that is the proximate cause, that's why we are currently so very productive . 这是对的,科技在人类历史上不断累积, 而现在科技正以飞快的速度进步。 这个是近因, 这也是为什么我们现在的生产力很高。
accumulated:v.积累;积聚;(数量)逐渐增加;(accumulate的过去式和过去分词) extremely:adv.非常,极其;极端地; proximate:adj.近似的;最近的; currently:adv.当前;一般地; productive:adj.能生产的;生产的,生产性的;多产的;富有成效的;
But I like to think back further to the ultimate cause. 但是我想要进一步回想到最终的原因
ultimate:adj.最终的;极限的;根本的;n.终极;根本;基本原则;
Look at these two highly distinguished gentlemen: 看看这两位非常杰出的绅士:
highly:adv.高度地;非常;非常赞许地; distinguished:adj.尊贵的; v.区分;
We have Kanzi -- he's mastered 200 lexical tokens , an incredible feat . 这位是坎兹先生 他掌握了200个词彙,这是一个难以置信的壮举。
lexical:adj.词汇的;[语]词典的;词典编纂的; tokens:n.令牌;代币;标识符;(token的复数) incredible:adj.难以置信的,惊人的; feat:n.功绩,壮举;技艺表演;adj.合适的;灵巧的;
And Ed Witten unleashed the second superstring revolution . 以及 爱德 维腾,他掀起了第二次超弦革命。
Witten:n.威腾(美国著名的数学家);威滕(德国中西部一个城市名); unleashed:v.发泄;突然释放;使爆发(unleash的过去分词和过去式) superstring:n.超弦; revolution:n.革命;旋转;运行;循环;
If we look under the hood, this is what we find: basically the same thing. 如果我们往脑袋瓜里面看,这是我们看到的: 基本上是一样的东西。
basically:adv.主要地,基本上;
One is a little larger, it maybe also has a few tricks in the exact way it's wired. 一个稍微大一点, 它可能有一些特别的连结方法。
These invisible differences cannot be too complicated , however, because there have only been 250,000 generations since our last common ancestor. 但是这些无形的差异不会太复杂, 因为从我们共同的祖先以来, 只经过了25万代。
invisible:adj.看不见的;n.看不见的人或物; complicated:adj.复杂的;难懂的;v.使复杂化;(complicate的过去分词和过去式)
We know that complicated mechanisms take a long time to evolve .
mechanisms:n.机制;[机]机构(mechanism的复数);机械;[机]机构学; evolve:v.(使)逐渐形成;进化;进化形成;
So a bunch of relatively minor changes take us from Kanzi to Witten, from broken-off tree branches to intercontinental ballistic missiles . 因此 一些相对微小的变化 将我们从坎兹先生变成了维腾, 从捡起掉落的树枝当武器到发射洲际弹道飞弹
relatively:adv.相当程度上;相当地;相对地; minor:adj.未成年的; n.未成年人; vi.副修; intercontinental ballistic missiles:洲际弹道导弹;
So this then seems pretty obvious that everything we've achieved, and everything we care about, depends crucially on some relatively minor changes that made the human mind. 因此,显而易见的是至今我们所实现的所有事 以及我们关心的所有事物, 都取决于人脑中相对微小的改变。
obvious:adj.明显的;显著的;平淡无奇的; crucially:adv.关键地;至关重要地;
And the corollary , of course, is that any further changes that could significantly change the substrate of thinking could have potentially enormous consequences . 由此而来的推论就是:在未来, 任何能显着地改变思想基体的变化 都有可能会带来巨大的后果。
corollary:n.推论;必然的结果; significantly:adv.意味深长地;值得注目地; substrate:n.基质;基片;底层(等于substratum);酶作用物; potentially:adv.可能地,潜在地; enormous:adj.庞大的,巨大的;凶暴的,极恶的; consequences:n.后果,结果;影响(consequence的复数);
Some of my colleagues think we're on the verge of something that could cause a profound change in that substrate, and that is machine superintelligence. 我的一些同事觉得我们即将发现 足以深刻的改变思想基体的科技 那就是超级机器智慧
colleagues:n.同事;同行(colleague的复数); on the verge of:濒临于;接近于; profound:adj.深厚的;意义深远的;渊博的;
Artificial intelligence used to be about putting commands in a box. 以前的人工智慧是将指令输入到一个箱子里。
Artificial intelligence:n.人工智能;
You would have human programmers that would painstakingly handcraft knowledge items . 你需要程式设计师 精心的将知识设计成指令。
painstakingly:adv.煞费苦心地;费力地; handcraft:n.手工艺,手艺;vt.以手工做; items:n.项目;一件商品(或物品);一则,一条(新闻)(item的复数)
You build up these expert systems, and they were kind of useful for some purposes, but they were very brittle , you couldn't scale them. 你建立这些专门系统, 这些系统在某些特定的领域中有点用, 但是它们很生硬,你无法延展这些系统。
brittle:adj.易碎的,脆弱的;易生气的; scale:n.规模;比例;鳞;刻度;天平;数值范围;v.衡量;攀登;剥落;生水垢;
Basically, you got out only what you put in. 基本上这些系统所输出的东西仅限于你事先输入的范围。
But since then, a paradigm shift has taken place in the field of artificial intelligence. 但是从那时起, 人工智慧的领域里发生了模式的转变。
paradigm shift:n.范式转移(指行事或思维方式的重大变化); in the field of:在…方面,在…领域;
Today, the action is really around machine learning. 现在主要的课题是机器的学习。
So rather than handcrafting knowledge representations and features, we create algorithms that learn, often from raw perceptual data. 因此,与其设计知识的表现及特点, 我们写出具有学习原始感官数据的能力的程式码。
handcrafting:n.手工;v.用手工制作;(handcrafting是handcraft的现在分词); representations:代表;表现(representation的复数); perceptual:adj.知觉的;感知的;有知觉的;
Basically the same thing that the human infant does. 基本上和婴儿所做的是一样的。
infant:n.婴儿;幼儿;未成年人;adj.婴儿的;幼稚的;初期的;未成年的;
The result is A.I. that is not limited to one domain -- the same system can learn to translate between any pairs of languages, or learn to play any computer game on the Atari console . 结果就是不局限于某个领域的人工智慧 -- 同一个系统可以学习在任何两种语言之间翻译 或者学着玩雅达利系统上的任何一款游戏。
limited:adj.有限的; n.高级快车; v.限制; (limit的过去分词和过去式) domain:n.领域;域名;产业;地产; translate:v.翻译;被翻译;被译成;(使)转变; computer game:n.电脑游戏; Atari:n.雅达利(美国一家电脑游戏机厂商); console:v.安慰;抚慰;慰藉;n.控制台,操纵台,仪表板;
Now of course, 现在当然
A.I. is still nowhere near having the same powerful, cross-domain ability to learn and plan as a human being has. 人工智慧到现在还未能达到像人类一样 具有强大的跨领域的学习能力。
nowhere:v.无处; n.无处; adj.不存在的;
The cortex still has some algorithmic tricks that we don't yet know how to match in machines. 人类大脑还具有一些运算技巧 我们不知道如何将这些技巧复制到机器中。
cortex:n.[解剖]皮质;树皮;果皮; algorithmic:adj.[数]算法的;规则系统的;
So the question is, how far are we from being able to match those tricks? 所以现在需要问的是: 我们还要多久才能在机器里面复制这些技巧?
A couple of years ago, we did a survey of some of the world's leading A.I. experts, to see what they think, and one of the questions we asked was, "By which year do you think there is a 50 percent probability that we will have achieved human-level machine intelligence?" 在几年前, 我们对世界顶尖的人工智慧专家做了一次问卷调查, 想要看看他们的想法,其中的一个题目是: 到哪一年你觉得人类会有50%的机率 能够达成人类级的人工智慧？
survey:n.调查;测量;审视;纵览;vt.调查;勘测;俯瞰;vi.测量土地; probability:n.可能性;机率;[数]或然率;
We defined human-level here as the ability to perform almost any job at least as well as an adult human, so real human-level, not just within some limited domain. 我们把人类级的人工智慧定义为有能力 将任何任务至少执行得像一名成年人一样好, 所以是真正的人类级别,而不是仅限于某些领域。
defined:adj.有定义的,确定的; v.使明确; perform:v.表演;执行;履行;演出;工作,运转(好/不好); as well as:也;和…一样;不但…而且;
And the median answer was 2040 or 2050, depending on precisely which group of experts we asked. 而答案的中位数是2040或2050年 取决于我们问的专家属于什么群体。
median:n.n.中值,中位数;三角形中线;梯形中位线;adj.中值的;中央的;
Now, it could happen much, much later, or sooner, the truth is nobody really knows. 当然,这个有可能过很久才实现,也有可能提早实现 没有人知道确切的时间。
What we do know is that the ultimate limit to information processing in a machine substrate lies far outside the limits in biological tissue . 我们知道的是,机器基体处理资讯能力的最终界限 比生物组织的界限要大的多。
processing:v.加工;处理;审核;数据处理;v.列队行进;缓缓前进;(process的现在分词) biological:adj.生物学的;生物的;与生命过程有关的;加酶的;n.[药]生物制品; tissue:n.纸巾,手巾纸;(人、动植物细胞的)组织;
This comes down to physics. 这取决于物理原理。
A biological neuron fires, maybe, at 200 hertz , 200 times a second. 一个生物神经元发出脉冲的频率可能在200赫兹,每秒200次。
neuron:n.[解剖]神经元,神经单位; hertz:n.[物]赫兹(频率单位);
But even a present-day transistor operates at the Gigahertz . 但就算是现在的电晶体都以千兆赫(GHz)的频率运转。
present-day:adj.现代的;当今的;现在的;现时的; transistor:n.晶体管(收音机); Gigahertz:n.千兆赫;
Neurons propagate slowly in axons , 100 meters per second, tops. 神经元在轴突中传输的速度比较慢,顶多是每秒100公尺。
propagate:vt.传播;传送;繁殖;宣传;vi.繁殖;增殖; axons:n.[解剖]轴突;轴索(axon的复数形式);
But in computers, signals can travel at the speed of light. 但在电脑里面,信号是以光速传播的。
There are also size limitations , like a human brain has to fit inside a cranium , but a computer can be the size of a warehouse or larger. 另外还有尺寸的限制 就像人类的大脑必需要放得进颅骨内。 但是一部电脑可以跟仓庫一样大,甚至更大。
limitations:n.局限性;(限制)因素;边界(limitation的复数形式); cranium:n.颅;[解剖]头盖; warehouse:n.仓库,货栈;货仓;
So the potential for superintelligence lies dormant in matter, much like the power of the atom lay dormant throughout human history, patiently waiting there until 1945. 因此超级智慧的潜能现在正潜伏在物质里面, 就像是原子能在人类的历史中一直潜伏着, 耐心的等着,一直到1945年。
dormant:adj.休眠的;静止的;睡眠状态的;隐匿的; atom:n.原子; throughout:adv.自始至终,到处;全部;prep.贯穿,遍及; patiently:adv.耐心地;有毅力地;
In this century, scientists may learn to awaken the power of artificial intelligence. 在这个世纪内, 科学家有可能会将人工智慧的力量唤醒。
awaken:v.唤醒;唤起;觉醒;醒来;意识到;
And I think we might then see an intelligence explosion . 届时我觉得我们会见证到智慧的大爆发。
explosion:n.爆炸;爆发;激增;
Now most people, when they think about what is smart and what is dumb , 大部分的人,当他们在想什么是聪明什么是愚笨的时候,
dumb:adj.哑的,无说话能力的;不说话的,无声音的;
I think have in mind a picture roughly like this. 我想他们脑中浮现出的画面会是这样的:
roughly:adv.粗糙地;概略地;
So at one end we have the village idiot , and then far over at the other side we have Ed Witten, or Albert Einstein, or whoever your favorite guru is. 在一边是村里的傻子, 然后在另外一边 是 爱德 维腾 或爱因斯坦,或你喜欢的某位大师。
village idiot:n.愚笨的村人;蠢人; whoever:任何人:无论是谁: guru:n.古鲁(指印度教等宗教的宗师或领袖);领袖;专家;
But I think that from the point of view of artificial intelligence, the true picture is actually probably more like this: 但是我觉得从人工智慧的观点来看, 真正的画面应该比较像这样子:
point of view:观点;见地;立场;
AI starts out at this point here, at zero intelligence, and then, after many, many years of really hard work, maybe eventually we get to mouse-level artificial intelligence, 人工智慧从这一点开始,零智慧 然后,在许多许多年辛苦的研究以后, 我们可能可以达到老鼠级的人工智慧,
eventually:adv.最后,终于;
something that can navigate cluttered environments as well as a mouse can. 它可以在凌乱的环境中找到路 就像一只老鼠一样。
navigate:vt.驾驶,操纵;使通过;航行于;vi.航行,航空; cluttered:adj.杂乱的; v.凌乱地塞满; (clutter的过去分词和过去式)
And then, after many, many more years of really hard work, lots of investment , maybe eventually we get to chimpanzee-level artificial intelligence. 然后,在更多年的辛苦研究及投资了很多资源之后, 我们可能可以达到黑猩猩级的人工智慧。
investment:n.投资;投入;封锁;
And then, after even more years of really, really hard work, we get to village idiot artificial intelligence. 然后,在更加多年的辛苦研究之后, 我们达到村庄傻子级别的人工智慧。
And a few moments later, we are beyond Ed Witten. 然后过一小会儿后,我们就超越了爱德维腾。
The train doesn't stop at Humanville Station. 这列火车并不会在人类村这一站就停车。
It's likely, rather, to swoosh right by. 它比较可能会直接呼啸而过。
swoosh:vi.哗哗响; n.嗖的一声(涡动); vt.嗖的一声(涡动);
Now this has profound implications , particularly when it comes to questions of power. 这个具有深远的寓意, 特别是在谈到权力的问题。
implications:n.蕴涵式;暗指,暗示;含蓄,含意;卷入(implication的复数); particularly:adv.特别地,独特地;详细地,具体地;明确地,细致地;
For example, chimpanzees are strong -- pound for pound, a chimpanzee is about twice as strong as a fit human male. 举例来说,黑猩猩很强壮 -- 以体重比例来说, 一只黑猩猩比一个健康的男性人类要强壮两倍。
chimpanzees:n.[脊椎]黑猩猩(chimpanzee的复数);
And yet, the fate of Kanzi and his pals depends a lot more on what we humans do than on what the chimpanzees do themselves. 然而,坎兹和他朋友们的命运则很大的部分取决于 人类的作为,而非黑猩猩们自己的作为。
pals:n.朋友;伙伴;哥们儿;v.(和某人)结伴,共度时光;(pal的第三人称单数和复数)
Once there is superintelligence, the fate of humanity may depend on what the superintelligence does. 当超级智慧出现后, 人类的命运可能会取决于超级智慧的作为。
humanity:n.人类;人道;仁慈;人文学科;
Think about it: 想想看:
Machine intelligence is the last invention that humanity will ever need to make. 机器智慧将会是人类所需要作出的最后一个发明。
Machines will then be better at inventing than we are, and they'll be doing so on digital timescales . 从那之后机器将会比人类更会发明, 他们也将会在 数位时间 里做出这些事。
digital:adj.数字的;手指的;n.数字;键; timescales:n.时标;时间量程(timescale的复数);
What this means is basically a telescoping of the future. 这意味着未来到来的时间将被缩短。
telescoping:n.叠生成矿酌; adj.可伸缩的; v.使叠进;
Think of all the crazy technologies that you could have imagined maybe humans could have developed in the fullness of time: cures for aging, space colonization , 想想那些我们曾经想像过的疯狂科技 人类可能在有足够的时间下可以发明出来: 防止衰老、殖民太空、
technologies:n.技术;科技(technology的复数); fullness:n.充满;丰富;完全;成熟; colonization:n.殖民;殖民地化;
self-replicating nanobots or uploading of minds into computers, all kinds of science fiction-y stuff that's nevertheless consistent with the laws of physics. 自行复制的奈米机器人,或将我们的头脑上载到电脑里, 这一些仅存在科幻小说范畴, 但同时还是符合物理法则的东西
self-replicating:adj.自我复制的;自我复制; nanobots:n.纳米机器人(nanobot的复数); uploading:v.上载;上传;(upload的现在分词) stuff:n.东西:物品:基本特征:v.填满:装满:标本: nevertheless:adv.然而,不过;虽然如此;conj.然而,不过; consistent:adj.始终如一的,一致的;坚持的;
All of this superintelligence could develop, and possibly quite rapidly. 超级智慧有办法开发出这些东西,而且速度可能很快。
Now, a superintelligence with such technological maturity would be extremely powerful, and at least in some scenarios , it would be able to get what it wants. 这么成熟的超级智慧 将会非常的强大, 最少在某些场景它将有办法得到它想要的东西。
technological:adj.技术[工程](上)的;因工艺技术高度发展而引起的; maturity:n.成熟;到期;完备; scenarios:n.情节;脚本;情景介绍(scenario的复数);
We would then have a future that would be shaped by the preferences of this A.I. 这样以来我们的未来就将会被这个超级智慧的偏好所影响。
preferences:n.偏爱;爱好;喜爱;偏爱的事物;(preference的复数)
Now a good question is, what are those preferences? 现在出现了一个好问题,这些偏好是什么呢？
Here it gets trickier . 这个问题更棘手。
trickier:adj.狡猾的;机警的(tricky的变形);
To make any headway with this, we must first of all avoid anthropomorphizing . 要在这个领域往前走, 我们必须避免将机器智慧拟人化(人格化)。
headway:n.前进;进步;航行速度;间隔时间; first of all:adv.首先; anthropomorphizing:v.赋与(…)人性[人形]; (anthropomorphize的现在分词);
And this is ironic because every newspaper article about the future of A.I. has a picture of this: 这一点很讽刺因为每一篇关于未来的人工智慧 的报导都会有这张照片:
ironic:adj.讽刺的;反话的;
So I think what we need to do is to conceive of the issue more abstractly, not in terms of vivid Hollywood scenarios. 所以我觉得我们必须要更抽象的来想像这个议题, 而非以好莱坞的鲜明场景来想像。
conceive:v.想出;想象;构想;设想;怀孕; vivid:adj.生动的;鲜明的;鲜艳的;
We need to think of intelligence as an optimization process, a process that steers the future into a particular set of configurations . 我们需要把智慧看做是一个优化的过程, 一个将未来指引到特定的组态的过程。
optimization:n.最佳化,最优化; steers:驾驶;引领;某物;小公牛;引导; configurations:n.配置(configuration的复数);立体基阵;
A superintelligence is a really strong optimization process. 一个超级智慧是一个很强大的优化过程。
It's extremely good at using available means to achieve a state in which its goal is realized. 它将很会利用现有资源 去达到达成目标的状态。
This means that there is no necessary conenction between being highly intelligent in this sense, and having an objective that we humans would find worthwhile or meaningful . 这意味着有着高智慧以及 拥有一个对人类来说是有意义的目标之间 并没有必然的联系。
intelligent:adj.有才智的;悟性强的;聪明的;有智力的 objective:n.目标; adj.客观的; worthwhile:adj.值得做的,值得花时间的; meaningful:adj.严肃的;重要的;重大的;意味深长的;
Suppose we give an A.I. the goal to make humans smile. 假设我们给予人工智慧的目标是让人类笑。
Suppose:v.推断:假定:假设:设想:
When the A.I. is weak, it performs useful or amusing actions that cause its user to smile. 当人工智慧比较弱时,它会做出有用的或是好笑的动作 以让使用者笑出来。
performs:v.执行;表演;演出;履行(perform的第三人称单数) amusing:adj.有趣的;逗人笑的;v.逗乐;(提供)消遣;(使)娱乐;(amuse的现在分词)
When the A.I. becomes superintelligent , it realizes that there is a more effective way to achieve this goal: take control of the world and stick electrodes into the facial muscles of humans to cause constant, beaming grins . 当人工智慧演化成超级智慧的时后, 它会体认到有更有效的方法可以达到这个目标: 控制这个世界 然后在人类的脸部肌肉上连接电级 以使这个人不断的微笑。
superintelligent:adj.有超常智慧的; effective:adj.有效的,起作用的;实际的,实在的;给人深刻印象; electrodes:n.[电]电极(electrode的复数);电焊条; facial:adj.面部的,表面的;脸的,面部用的;n.美容,美颜;脸部按摩; muscles:n.肌肉(muscle的复数); beaming:v.发射(电波);笑容满面;播送;射出光(或热);(beam的现在分词) grins:v.露齿而笑;咧着嘴笑;龇着牙笑;
Another example, suppose we give A.I. the goal to solve a difficult mathematical problem. 另外一个例子, 假设我们给人工智慧的目标是解出一个非常困难的数学问题。
mathematical:adj.数学的,数学上的;精确的;
When the A.I. becomes superintelligent, it realizes that the most effective way to get the solution to this problem is by transforming the planet into a giant computer, so as to increase its thinking capacity . 当人工智慧变成超级智慧时, 它会体认到最有效的方法是 把整个地球转化成一部超大号的电脑, 进而增加它自己的运算能力。
solution:n.解决方案;溶液;溶解;解答; transforming:v.使改变形态;使改变外观(或性质);使改观;(transform的现在分词) giant:n.巨人;伟人;巨兽;adj.巨大的;特大的 capacity:n.能力;容量;资格,地位;生产力;
And notice that this gives the A.I.s an instrumental reason to do things to us that we might not approve of. 注意到这个模式会给人工智慧理由去做 我们可能不认可的事情。
instrumental:n.工具格;器乐曲;工具词;adj.起重要作用;用乐器演奏的;为乐器谱写的; approve:v.批准;赞成;同意;核准;
Human beings in this model are threats, we could prevent the mathematical problem from being solved. 在这个模型里面人类是威胁, 我们可能会在解开数学问题的过程中成为阻碍。
Of course, perceivably things won't go wrong in these particular ways; these are cartoon examples. 当然,在我们可预见的范围内,事情不会以这种方式出错; 这些是誇大的例子。
But the general point here is important: if you create a really powerful optimization process to maximize for objective x, you better make sure that your definition of x incorporates everything you care about. 但是它指出的概念很重要: 如果你创造了一个非常强大的优化流程 要最大化目标X, 你最好确保你对目标X的定义 包含了所有你所在意的事情。
maximize:vt.取…最大值;对…极为重视;vi.尽可能广义地解释;达到最大值; definition:n.定义;清晰度;(尤指词典里的词或短语的)释义;解释; incorporates:组成公司;
This is a lesson that's also taught in many a myth . 这也是在很多神话故事中教导的寓意。
myth:n.神话;虚构的人,虚构的事;
King Midas wishes that everything he touches be turned into gold. 希腊神话中的米达斯国王希望他碰到的所有东西都可以变成金子。
He touches his daughter, she turns into gold. 他碰到了他的女儿,她变成了黄金。
He touches his food, it turns into gold. 他碰到了他的食物,他的食物也变成了黄金。
This could become practically relevant , not just as a metaphor for greed , but as an illustration of what happens if you create a powerful optimization process and give it misconceived or poorly specified goals. 这实际上跟我们的题目有关, 不仅仅是对贪婪的隐喻, 但也指出了如果你创造了一个强大的优化流程 但同时给了它不正确或不精确的目标后 会发生什么事。
practically:adv.实际地;几乎;事实上; relevant:adj.相关的;切题的;中肯的;有重大关系的;有意义的,目的明确的; metaphor:n.暗喻,隐喻;比喻说法; greed:n.贪婪,贪心; illustration:n.插图;图解;示例;图示; misconceived:adj.设想错误的;v.误解(misconceive的过去式和过去分词形式); specified:v.指定;具体说明;逐一登记;列入清单;(specified是specify的过去分词)
Now you might say, if a computer starts sticking electrodes into people's faces, we'd just shut it off. 你可能会说,如果电脑系统开始在人脸上安装电极, 我们可以直接把他关掉就好了。
A, this is not necessarily so easy to do if we've grown dependent on the system -- like, where is the off switch to the Internet? 一、这并不一定容易做到,如果我们已经对这个系统产生依赖性 —— 比如:你知道网际网路的开关在哪里吗？
necessarily:adv.必要地;必定地,必然地; dependent on:依赖于;依靠;
B, why haven't the chimpanzees flicked the off switch to humanity, or the Neanderthals ? 二、为什么黑猩猩当初没有把人类的开关关掉？ 或是尼安德特人？
flicked:v.轻击;(使)突然快速移动;向…笑了一下;(flick的过去分词和过去式) Neanderthals:n.穴居人(Neanderthal的复数);尼安德塔人;
They certainly had reasons. 他们有很明显的理由要这么做,
We have an off switch, for example, right here. 而我们的开关就在这里:
(Choking) (窒息声)
The reason is that we are an intelligent adversary; we can anticipate threats and plan around them. 原因是人类是很聪明的敌人; 我们可以预见威胁并为其做出准备。
anticipate:v.预期,期望;占先,抢先;提前使用;
But so could a superintelligent agent , and it would be much better at that than we are. 但一个超级智慧也会, 而且它的能力将比我们强大的多。
agent:n.代理人,代理商;药剂;特工;v由…作中介;由…代理;adj.代理的;
The point is, we should not be confident that we have this under control here. 我想要说的一点是,我们不应该觉得一切都在我们的掌握之中。
confident:adj.自信的;确信的;
And we could try to make our job a little bit easier by, say, putting the A.I. in a box, like a secure software environment, a virtual reality simulation from which it cannot escape. 我们可能可以藉由把AI放到一个盒子里面 来给我们更多的掌握, 就像是一个独立的软体环境, 一个AI无法逃脱的虚拟实境。
virtual reality:n.(计算机创造的)虚拟现实; simulation:n.仿真;模拟;模仿;假装;
But how confident can we be that the A.I. couldn't find a bug . 但是我们有多大的信心这个AI不会找到漏洞？
bug:n.臭虫,小虫;故障;窃听器;v.烦扰,打扰;装窃听器;
Given that merely human hackers find bugs all the time, 就算只是人类骇客,他们还经常找出漏洞。
merely:adv.仅仅,只不过;只是; hackers:n.黑客,骇客;电脑黑客(hacker的复数); bugs:n.缺陷;虫子;窃听器;(bug的复数)v.窃听;使烦恼;(bug的第三人称单数)
I'd say, probably not very confident. 我想我们不是很有信心。
So we disconnect the ethernet cable to create an air gap, but again, like merely human hackers routinely transgress air gaps using social engineering . 那所以我们把网路线拔掉,制造一个物理间隙, 但同样的,就算只是人类骇客 也经常可以利用社交工程陷阱来突破物理间隙。
disconnect:vt.拆开,使分离;vi.断开; ethernet:n.[计]以太网; cable:n.电缆; v.打电报; (用锚链,缆索等)系住; routinely:adv.例行公事地;老一套地; transgress:vt.违反;侵犯;犯罪;vi.违反;违法;越界; gaps:n.差异,缺口;缝隙(gap的复数形式);v.裂开;使豁裂(gap的第三人称单数形式); social engineering:n.社会工程(根据政治信念改造社会);
Right now, as I speak, 现在,在我在台上说话的同时
I'm sure there is some employee out there somewhere who has been talked into handing out her account details by somebody claiming to be from the I.T. department. 我确定在世界的某一个角落里有一名公司职员 才刚刚被自称来自IT部门的人士说服(诈骗) 并交出了她的帐户信息。
claiming:v.声称;要求给予;理应获得;(claim的现在分词)
More creative scenarios are also possible, like if you're the A.I., you can imagine wiggling electrodes around in your internal circuitry to create radio waves that you can use to communicate. 更天马行空的状况也可能会发生, 就像是如果你是AI, 你可以想像藉由摆动你体内的电路 然后创造出无线电波,用以与外界沟通。
creative:adj.创造性的; wiggling:v.(使)扭动,摆动,起伏;(wiggle的现在分词) internal:n.内脏;本质;adj.内部的;里面的;体内的;(机构)内部的; circuitry:n.电路;电路系统;电路学;一环路;
Or maybe you could pretend to malfunction , and then when the programmers open you up to see what went wrong with you, they look at the source code -- Bam! -- the manipulation can take place . 或这你可以假装有故障, 然后当程式设计师把你打开檢查哪里出错时, 他们找出了原始码 --梆-- 你可以在此做出操控。
malfunction:n.失灵;故障,功能障碍; source code:n.源(代)码; manipulation:n.操作;管理措施;处理;操纵证券市场;变换; take place:发生;举行;
Or it could output the blueprint to a really nifty technology, and when we implement it, it has some surreptitious side effect that the A.I. had planned. 或这它可以做出一个很巧妙的科技蓝图, 当我们实施这个蓝图后, 它会产生一些AI计划好的秘密副作用。
output:n.(人、机器、机构的)产量;输出;输出功率;输出量;v.输出; blueprint:vt.计划;制成蓝图;n.蓝图,设计图;计划; nifty:adj.俏皮的;漂亮的;n.俏皮话; implement:v.实施;执行;贯彻;使生效;n.工具; surreptitious:adj.秘密的;鬼鬼祟祟的;暗中的; side effect:n.副作用;附带后果;
The point here is that we should not be confident in our ability to keep a superintelligent genie locked up in its bottle forever. 寓意是我们不能对我们控制人工智慧的能力 具有太大的信心
genie:n.妖怪;鬼;
Sooner or later , it will out. 它终究会逃脱出来,只是时间问题而已。
Sooner or later:迟早,早晚;
I believe that the answer here is to figure out how to create superintelligent A.I. such that even if -- when -- it escapes, it is still safe because it is fundamentally on our side because it shares our values. 我觉得解方是我们需要弄清楚 如何创造出一个超级智慧,哪怕是它逃出来了, 它还是安全的,因为它是站在我们这一边的 因为它拥有了我们的价值观。
fundamentally:adv.从根本上;基础地;重要地
I see no way around this difficult problem. 我们没有办法避免这个艰难的问题。
Now, I'm actually fairly optimistic that this problem can be solved. 但是我觉得我们可以解决这个问题。
fairly:adv.相当地;公平地;简直; optimistic:adj.乐观的;乐观主义的;
We wouldn't have to write down a long list of everything we care about, or worse yet, spell it out in some computer language like C++ or Python , that would be a task beyond hopeless. 我们并不需要把我们在乎的所有事物写下来, 或更麻烦的把这些事物写成电脑程式语言 这是个不可能完成的任务。
spell it out:亲口说出; Python:n.巨蟒;大蟒;
Instead, we would create an A.I. that uses its intelligence to learn what we value, and its motivation system is constructed in such a way that it is motivated to pursue our values or to perform actions that it predicts we would approve of. 与其,我们可以创造出一个人工智慧,它用它自己的智慧 来学习我们的价值观, 它的激励机制要设计成会让它想要 来追求我们的价值观或者去做它认为我们会赞成的事情。
motivation:n.动机;积极性;推动; constructed:v.修建;建造;组成;编制,绘制;(construct的过去分词和过去式) motivated:adj.有动机的; v.使产生动机; pursue:v.继续;从事;追赶;纠缠; predicts:vt.预报,预言;预知;vi.作出预言;作预料,作预报;
We would thus leverage its intelligence as much as possible to solve the problem of value-loading. 藉此我们可以最大化地利用到它们的智慧 来解决这个价值观的问题。
leverage:n.影响力;杠杆作用;杠杆效力;v.举债经营;借贷收购;
This can happen, and the outcome could be very good for humanity. 这个是有可能的, 而且这个的结果可对人类是非常有益的。
outcome:n.结果,结局;成果;
But it doesn't happen automatically . 但是它不会自动发生。
automatically:adv.自动地;机械地;无意识地;adj.不经思索的;
The initial conditions for the intelligence explosion might need to be set up in just the right way if we are to have a controlled detonation . 如果我们需要控制这个智慧的大爆炸, 那智慧大爆炸的初始条件 需要被正确的建立起来。
initial:adj.最初的; n.(名字的)首字母; v.用姓名的首字母作标记(或签名)于; detonation:n.爆炸;爆炸声;爆发;
The values that the A.I. has need to match ours, not just in the familiar context, like where we can easily check how the A.I. behaves , but also in all novel contexts that the A.I. might encounter in the indefinite future. 人工智慧的价值观要和我们的一致, 并不只是在常见的状况下, 比如我们可以很简单低檢查它的行为, 但也要在未来所有人工智慧可能会遇到的情况下 保持价值观的一致。
familiar:adj.熟悉的;常见的;亲近的;n.常客;密友; behaves:v.表现;表现得体;有礼貌;(behave的第三人称单数) novel:adj.新奇的;异常的;n.小说; contexts:n.环境,[计]上下文(context复数); encounter:v.遭遇,邂逅;遇到;n.遭遇,偶然碰见; indefinite:adj.不确定的;无限的;模糊的;
And there are also some esoteric issues that would need to be solved, sorted out: the exact details of its decision theory , how to deal with logical uncertainty and so forth. 还有很多深奥的问题需要被解决: 它们决策概念的所有细节, 它如何面对解决逻辑不确定性的情况等问题。
esoteric:adj.秘传的;限于圈内人的;难懂的; issues:n.重要议题;争论的问题;v.宣布;公布;发出;(issue的第三人称单数和复数) decision theory:un.定下决心的理论;判定理论;决策论; uncertainty:n.不确定,不可靠;
So the technical problems that need to be solved to make this work look quite difficult -- not as difficult as making a superintelligent A.I., but fairly difficult. 所以技术上待解决的问题 让这个任务看起来蛮难的 -- 还没有像做出一个超级智慧那样的难, 但还是挺难的。
technical:adj.工艺的,科技的;技术上的;专门的;
Here is the worry: 我们所担心的是:
Making superintelligent A.I. is a really hard challenge. 创造出一个超级智慧是一个很难的挑战。
Making superintelligent A.I. that is safe involves some additional challenge on top of that. 创造出一个安全的超级智慧 是一个更大的挑战。
involves:v.包含;需要;牵涉;牵连;影响;(使)参加,加入(involve的第三人称单数) additional:adj.附加的,额外的;
The risk is that if somebody figures out how to crack the first challenge without also having cracked the additional challenge of ensuring perfect safety. 最大的风险在于有人想出了如何解决第一个难题 但是没有解决第二个问题 来确保安全性万无一失。
cracked:adj.疯狂;破裂的;有裂纹的;v.破裂;裂开;断裂;(crack的过去分词和过去式) ensuring:v.保证;确保;担保(ensure的现在分词);
So I think that we should work out a solution to the control problem in advance , so that we have it available by the time it is needed. 所以我觉得我们应该先想出 如何 控制 的方法。 这样当我们需要的时候我们可以用的到它。
in advance:adv.预先,提前;
Now it might be that we cannot solve the entire control problem in advance because maybe some elements can only be put in place once you know the details of the architecture where it will be implemented . 现在也许我们无法完全解决「控制」的问题 因为有时候你要了解你所想要控制的架构后 你才能知道如何实施。
elements:n.要素;基本部分;少量;一群;(element的复数) architecture:n.建筑学;建筑风格;建筑式样;架构; implemented:v.使生效;贯彻;执行;实施;(implement的过去式和过去分词)
But the more of the control problem that we solve in advance, the better the odds that the transition to the machine intelligence era will go well. 但是如果我们可以事先解决更多的难题 我们顺利的进入到机器智能时代的机率 就会更高。
odds:n.几率;胜算;不平等;差别; transition:n.过渡;转变;变革;变迁;v.经历转变过程;过渡;
This to me looks like a thing that is well worth doing and I can imagine that if things turn out okay, that people a million years from now look back at this century 这对我来说是一个值得挑战的事情 而且我能想像到如果一切顺利的话, 我们的后代,几百万年以后的人类回顾我们这个时代的时候
and it might well be that they say that the one thing we did that really mattered was to get this thing right. 他们可能会说我们所做的最重要的事就是 把这个事情弄对了。
Thank you. 谢谢
(Applause) (观众掌声)