Graham points to Anthropic risk reports but no clear number

I'm going to move on to the other companies.

Same question.

The first question that I ask, which is we want to hear how each of you quantify the risks imposed by AI in the worst catastrophic scenario.

So I'm going to go to Mr.

Graham.

Are you able to quantify the risk?

From the beginning of our safety work at Anthropic, our fundamental belief is that if not properly safeguarded, the risk might be too high.

And so I built and have led the team at Anthropic that did some of the first thinking around how we quantify risks like this.

For the past four years, we focused on issues like biological security and cybersecurity.

And the actual process of quantifying those risks is intricate and nuanced, and we learn more every year.

And we detail quite a lot of it in documents like our responsible scaling policy, how we think about the threat models of this,

all the way up to notions of loss of control and misalignment, which from the beginning of the company, we have tried to lead the world in the research of.

So what I do know today is I feel relatively sort of more optimistic.

The industry has moved quite fast to take cybersecurity seriously and biological security.

I think documents and policies like responsible scaling policies have so far been fairly effective at taking a cautious approach.

However, now we have bigger and bigger threats that might come if we don't properly safeguard them as the models get more and more capable.

And so what we do to mitigate that is, number one, put in tremendous amounts of effort across a number of different teams, our own safeguards teams, our research teams.

team to try to quantify and mitigate this.

And it's increasingly important for us to be dramatically more transparent about what we think about these risks.