Menin presses companies to commit not to release failing models

Will you all commit right now under oath publicly that if your models fail an internal test or a third party

validation test that's not selected by your company, will you commit to not release that model?

I'm going to start with Ms.

Dwyer.

Yeah, I can commit that OpenAI is not going to release models that we don't believe that are safe.

As part of that assessment process, we work with multiple third parties.

And as I mentioned before, with the U.S.

As well as the U.K.

Federal governments to assess whether our models are safe.

So if it fails either an internal test or a third-party test, you will commit to not release it.

OpenAI has historically delayed the release of models to ensure that we can build up the right safeguards.

We've done it before and we'll do it again.

Again, I think a simple yes or no would instill more confidence in the public on a matter as serious as this.

I'm going to move to Mr.

Graham.

We take that one particularly seriously.

The way we approach it is we try to detail an even more nuanced and lengthy approach to how we release models, what safeguards they require in order to be released, what tests need to be done.

And to your point about independent evaluators, this is one of the most important components, we think.

We have tried and long supported creating a burgeoning sort of group of independent evaluators, including working with multiple governments to do exactly this.

And we collectively work together to try to refine and improve the exact decision process by which you release a model.

And so we think what we should do is develop the right and best and improve on these decision processes.

We detail ours in our responsible scaling policy.

We should refine and improve it every so often.

So we release updates based on what we learn every six to 12 months or so at this rate, and that this should involve independent evaluators.

Okay, that also is not a clear yes or no.

I'm going to move on to Mr.

Cahill.

Speaker, thank you very much for that question.

What I can commit to is that we do not deploy models which are not safe pursuant to our scaling framework.

And I just wanted to mention also that, and I think I mentioned this a little earlier, but we do believe that every lab has a responsibility to go at the pace to ensure the safety of their models and the ability to take action in relation to that pacing.

For example, at Meta, we delayed the release of Muse for several months to focus on safety and security.

In relation to independent evaluators, we recently signed into court with a commitment to partner with independent evaluators and assessors.

And this is also detailed in our framework.

Okay, Ms.

Friend.

Ma'am, as part of our multi-layered approach to responsible AI governance, we conduct rigorous launch reviews ahead of launch of models to ensure that anything we put into general availability is safe.

Okay.

I mean, again, I think it would be far clearer if we could get clear yeses or noes from everyone, but in the absence of that, I'm going to take what you've each said as sort of an equivocation.