Kim outlines minimum testing, evaluation, and red-team standards

And based on your experience inside the company, what minimum testing, evaluation, and red team procedures should frontier AI developers be required to complete before releasing any model to the public?

There are a number of very critical safety gaps that are present in XAI's model releases that are not present in other companies.

One gap is critical thresholds.

Companies typically commit to not releasing models above what they consider to be critical thresholds, whether that is in bioweapon capabilities,

cyber capabilities, and other dangerous capabilities that the model have.

A second...

A second part of these releases that is very critical is a documentation justification for deployment.

Typically what that looks like is explaining what sort of critical thresholds were reached and what work was done to mitigate those thresholds and their justification for why they believe this model is safe before releasing it.

And then especially, and then a third critical piece that is becoming increasingly relevant is

Companies should be testing for loss of control and manipulation capabilities of these models before releasing.

And specifically, these are standards that many companies have today, but are notably absent in the XCI safety releases.