Kokotajlo corroborates reckless racing culture at AI labs
Daniel Kokotajlo begins his statement, saying he will avoid repeating Coxon. He corroborates the racing-to-superintelligence and move-fast-break-things culture seen at Anthropic and OpenAI, and quotes OpenAI's Dan Selsam about researchers' dependence on models.
With additional questions, I'm now going to have an opening statement from Daniel Kokotago.
Hello, everybody.
Members of the Committee of the Whole, thank you for having me here today.
A lot of what I say, I guess, is going to be stolen thunder by Jacob there, but I'll try to focus on the things that he didn't already say.
First of all, yeah, it's true.
They're racing towards superintelligence.
There's this move fast and break things culture there.
That's also what I saw while I was there and what I continue to hear from talking to friends still there at Anthropic and at OpenAI and at some of these other companies.
You know, I've also heard there's a statement by Dan Selsom, who currently works at OpenAI.
He confirms what Jacob was saying.
Most of the code is written by AIs.
Quote, researchers and engineers in all parts of the stack are rapidly increasing their dependence on the models even to perceive the world.
I myself barely look at raw code anymore and struggle to maintain the discipline to engage deeply with the models, explanations, and proposals, end quote.
Basically, right now at these companies, many of the researchers involved are kind of managing AIs.
Like the AIs do the actual coding and the humans...
Talk to them and tell them what to do and hear their explanation.
of what's going on in the near future.
And by near future, I mean maybe in the next year, two years, three years, something like that.
They will have trained AIs that can do the entire thing autonomously.
That's the recursive self-improvement that Jacob was talking about.
And it's unclear how long it's going to take them to succeed, but it really does seem like it could happen any year now.
If they do succeed at that, then
compared to today.
The actual work will be done by the AIs, the thinking will be done by the AIs, the planning, the designing will be done by the AIs, and the humans will be in a relationship to the AIs that's more like the relationship between, say, a board of directors and
and the actual company that the board of directors is supposed to be supervising, where you're reliant on AI-generated explanations, even to understand what's going on inside the company and inside the data centers.