Coxon testifies on lack of AI control and accelerating capability

Good morning.

Thanks for having me here.

Yeah, I'm Jacob Coxon.

So until a few weeks ago, I was a capabilities researcher at Anthropic.

And before that, I also worked at OpenAI.

I guess the two things I saw that made me scared, the first is the fact that we

We do not know how to control any AI system yet.

So any AI system that we build, or perhaps more accurately that we grow, we don't fully control it.

We don't understand its drives or why it does the things it does.

That was the first thing that became apparent.

The second thing is that the pace of capability improvement is accelerating and the capabilities of the models next year will be very high.

In particular, we're approaching the point at which the AI systems will be capable of improving themselves.

So doing the research that humans were doing previously.

So these two things, the fact that the AI systems next year are going to be very powerful, and the second...

that we don't know yet scientifically how to fully control them.

And if you have a prepared statement that you want to read, please go ahead and do so, and then I'll actually turn it over to the other two panelists to do so.

Of course.