Maloney questions companies on how they define AI safety incidents

Thank you, Chair.

I worked in tech for nearly a decade, and one of the core principles is that if you can't measure it, you can't improve upon it.

So first, you mentioned you believe in incident reporting and disclosures, but in the context of this conversation, how do you define a safety incident?

Inside your companies.

Councilmember Lee touched on privacy.

Do you track privacy breaches?

What about a successful jailbreak?

Would that be considered an incident?

And is a containment failure that occurs during testing an incident or doesn't only count once a real person is harmed?

I would love more details on how exactly you're tracking some

of the concerns that the council has raised today.

And second, when an incident happens after disclosure, can you walk me through what changes?

Who owns the fix?

How do you verify that it worked?

And who decides whether or not the model continues to run in the meantime?

And then lastly, what are the most important safeguards that you know how to implement today?

What's stopping it from being implemented across all products?

And what do you believe should be the industry standard?

Thank you.

There were a lot of parts to that question, so I'm going to do my best to answer them.

If I fall short, please let me know.

So first, the question of how you define an incident.

I think this has been an area of a lot of confusion publicly, so thank you for asking the question.

When we talk about incidents at OpenAI, we're talking about misalignment incidents.

The best way to understand that is...

You know, it is not unusual, not suspicious for an agent to look at a website for information.

There's nothing abnormal about that.

But if the agent accesses non-public information on the website, that's an instance of misalignment.

And so what OpenAI is doing is making instances like that, even though there might not have been a harm to the third party, we are making instances like that.

public on our website, again, because we believe it's important to share what we've learned to advance policymakers as well as researchers' understanding.

On the specifics of incidents, my understanding as a policy person is that this is actually a robust area of policy debate.

And so that's why we're happy to make our thinking on this public through our reporting framework.

I'm going to pause there because I can't remember the rest of your questions.

Why don't we focus on the first question, which was on measurement, and hear from all four companies?

Council member, specifically in relation to incident, I think that's what, I think that was the, correct me if I'm wrong, I think that was the first piece in relation to tracking and how we define it.

We set this out in our scaling framework, which we've made publicly available.

I don't want to be imprecise as I don't have it in front of me, so I don't have it to quote to you.

So please let me follow up with you afterwards and connect you with the right people to talk to you about.

Yes, ma'am.

I think this is a critically important question as we're debating with multiple regulators around the country and around the world right now what constitutes an incident, particularly when you think about the possibility that

we could over-report incidents, some of which may not be of a serious enough nature for public concern.

We want to ensure that we narrow to the class of incidents that are truly important for us to use to learn to improve safety and also for us to share with you all to collectively improve public safety.

So a few things come to mind as we've been having these debates.

One, of course, is cybersecurity incidents, incidents covering the security of our most powerful AI models themselves.

That's been under discussion for several years now, protecting model weights in particular.

We also...

Talk a lot about focusing on incidents that cause material harms in one way or another.

And I think part of the discussion will be, you know, how we define those with precision.

We also consider incidents to be those that violate our policies.

You asked us about how we respond to incidents, and primarily our trust and safety teams are responsible for responding to what we call escalations, which usually involve violation of those policies.

First, we're

viewing the alleged violation, which are often reported to us by external parties.

And then if we determine that, in fact, our policy has been violated, we have a variety of different actions we might take.

And then, of course, a violation of the law.

As I've said a couple of times in the hearing, if it's illegal without AI, it's still illegal with AI.

And so I would think that incidents that need to be flagged would, of course, involve violation of the law.

As to most important safeguards, it's very hard to answer that question purely because there's a range of different safety considerations.

We've talked about many of them today during the hearing.

And again, several of us have referred to this layered approach.

My own background is in national security.

We talk about layered security in that context as well.

So you wouldn't really want to talk about a single safeguard.

You would want to talk about layers of safeguards that together create much more assurance and confidence in the safety of the whole system.

Thank you very much.