All of this superintelligence could develop, and possibly quite rapidly.

超级智慧有办法开发出这些东西，也许更快。

Whatever the creator of the superintelligence wants? Let the AI decide?

或者照着超级智能创造者的意愿来？还是让AI自己决定？

Many of them believe blue eyes are a sign of natural superintelligence.

他们中的许多人认为，蓝眼睛是天生超级智慧的标志。

I am not saying that the problem of aligning superintelligence is unsolvable in principle.

我没有说超级智能的对齐问题是根本无法解决的。

It could kill us because it doesn't want us making other superintelligences to compete with it.

它把我们杀光是因为它不想让我们做出与其竞争的其他超级智能。

And when exactly is superintelligence going to arrive? When do we need to start panicking?

究竟什么时候超级智能才会出现呢？我们什么时候开始恐慌？

If we build superintelligence, we'd be better off in the position of cats and dogs than ants.

如果我们建立超级智能，我们最好站在猫狗一样的角度，而不是蚂蚁。

First of all, Henry, superintelligence doesn't have to be something negative.

首先，超级智能并不是负面的东西。

Once there is superintelligence, the fate of humanity may depend on what that superintelligence does.

一旦出现超级智能，人类的命运可能取决于超级智能的所作所为。

First, it wants us dead before we build any more superintelligences that might compete with it.

首先，它希望我们在我们建立更多可能与它竞争的超级智能之前就死掉。

So it would have to be with a superintelligence.

所以它必须与超级智能一起使用。

Let's just pause the reckless race to superintelligence.

让我们暂停这场不计后果的超级智能竞赛吧。

Some people claim computers will eventually gain superintelligence, be able to outperform humans on any task, and destroy humanity.

一些人认为电脑最终将获得超级智能，能够取代人类做任何任务并摧毁人类。

So what's going to happen if we get AGI and superintelligence?

那么，如果我们拥有通用人工智能和超级智能，会发生什么呢？

But if your adversary is superintelligenceor a human using superintelligence against you, right, trying is just not enough.

但如果你的对手是超级智能，或者是一个使用超级智能来对抗你的人，那么尝试是不够的。

He meant superintelligence, the kind of thing that worries Bill Gates and Elon Musk.

他指的是超级智能，这是比尔盖茨和埃隆马斯克担心的事情。

Three years ago, people were saying it's inevitable, superintelligence, it'll be fine, it's decades away.

三年前，人们说这是不可避免的，超级智能，一切都会好起来的，这还需要几十年的时间。

And these companies are also trying to build superintelligence, leaving human intelligence far behind.

而这些公司还试图打造超级智能，将人类智能远远抛在后面。

And many think it'll only be a few years, maybe, from AGI to superintelligence.

许多人认为从通用人工智能到超级智能可能只需要几年的时间。

And obviously, if you've got a superintelligence of immense capability, far more capable than a human, that is a concern, that is a risk.

未来，“超级智能”将比人类更有能力，也同时存在着风险。