Kokotajlo on balancing China competition with safety slowdown

You said we need to increase transparency, and you said we should slow down frontier AI.

In terms of the latter, slowing down frontier AI, how do we balance the need of what is happening in China?

And ensure that the United States is keeping a pace, but we are at the same time putting in place these responsible safeguards.

What is the best way in all three of your opinions that we do that?

Great question.

A couple of things.

First of all, we need to not let our own companies do something that risks our lives, even if as a result, you know, China might then later do something that risks our lives.

Like this is this is not very complicated.

We have to first solve the immediate pressing threat here domestically and then.

Try to make sure that China doesn't get us killed either, to put it sort of bluntly and simply.

But another thing I can add to that is that right now, most Chinese AI progress actually comes from the United States in a couple different ways.

So one is distillation.

So some of these Chinese AI companies are just directly training.

Their AI.

systems using the outputs from Anthropic and OpenAI, for example.

And so if Anthropic and OpenAI stopped making their AIs better at coding, for example, and stopped making their AIs better at research, then that would sort of directly slow down the rate at which the Chinese AIs

progress towards recursive self-improvement.

In other manners as well.

So I think a lot of the ideas, a lot of the algorithmic secrets and special sauce for how to train AIs are invented and discovered in the United States and then one way or another make their way over to China.

Finally, I think that right now the U.S.

AI companies in the lead do not have adequate security to protect against model weight theft or certainly algorithmic secret theft and the code theft from Chinese military.

So if it really came down to it and the competition between the US and China turned into something

more of a crisis and some sort of conflict, for example, then I think that we should assume that the CCP would just get our best AIs.

And so it's almost a kind of a no-brainer that like, if we not only

The best way to slow down Chinese progress towards doing this extremely dangerous thing of recursive self-improvement is to just slow down the U.S.

Companies right now.

I think that's the sort of immediate thing to say right in the moment.

And then finally...

Yeah, yeah.

That'd be what I would say.