Narcisse questions companies on AI in healthcare and clinician oversight

We have Council Members Narcisse, Joseph, P.

Sanchez, and then Council Member Phil Wong.

So, Narcisse?

Good afternoon.

Thank you, Chair, and thank you, Madam Speaker.

I want to say thank you to you, too, because you guys stay here with us, and I'm happy to see at least two women is on the panel to talk to us.

Hollywood give us clues all the time, but we kind of slow in taking, you know, hints on clues.

But no companies in this world, I feel, that should be self-regulated.

What I'm sitting here for about four hours, and like I said, thank you to you, privacy, transparency, and safety.

So now, do any of those companies that you can say there is a standard across the board for all the companies to follow?

Because apparently not.

Now, do you regulate hospital?

I mean, do you have AI, sell AI, sorry.

Do you sell AI products to New York City Hospital?

And if so, do you think that a licensed clinician should sign on all the diagnostic tests?

Because I heard if Ms.

Friend stated that you're diagnosed with...

You went to a process of cancer, if I'm correct.

So you understand that people need to be diagnosed by a human.

And I can give you an example, like for colonoscopy.

Sometimes the bubble, according to AI, will tell you it's a polyps.

In the meantime, it's a polyps.

It's not polyps.

It's not polyps.

It's a bubble.

But AI cannot differentiate that.

So therefore, you need a human to see through.

And right now, I don't know if you read New York Times.

I'm sorry.

Can you conclude the question?

New York Times already telling us that there's a report that AI is getting confusion between the hospital and the insurance company.

So I don't know how you see that happening, how we can make sure that AI is being monitored.

Yes, so thank you for your question.

ChatGPT for healthcare is deployed, for example, in Memorial Sloan Kettering Cancer Center.

But to your point, we believe that doctors should remain in charge.

We do not think that AI should represent itself as a licensed professional.

And we also think, though, as we've talked about earlier, there's tremendous potential to use AI in healthcare settings, particularly for administrative uses, documentation.

I mean, doctors are already overworked, and giving them an opportunity to free up their time so that they can spend more time with their patients is an outcome I think we can all agree would lead to better care.

Council Member, thank you very much for these questions.

In relation to your first question on self-regulation, we believe that the accord

is a really important step in relation to the concerns that you mentioned.

In relation to healthcare, meta AI is prohibited from acting as a licensed professional, including a doctor or any other medical professional who's licensed.

There are, as my colleagues have mentioned, phenomenal benefits in relation to the use of AI in healthcare.

I know that there's phenomenal work happening at NYU Langone, who actually, I think, has a specialist division that's actually investing and working specifically around AI in healthcare settings.

And similarly, Columbia Irving, which is really doing phenomenal work in that space also.

Ma'am, thank you so much for those questions.

On whether or not a company should be self-regulated, I again want to emphasize at Google, we believe that we are regulated by existing regulations, sectoral regulations,

state, federal level regulations all apply to AI, and we have supported further regulation, particularly of frontier AI at the federal level, but also other AI-directed laws.

all levels of government.

So, no, we should not be self-regulated.

We should be within the regulatory frameworks we already are.

On the questions about health care, so we certainly agree that doctors should stay in the lead and that AI is a tool for clinicians.

It's not a replacement for them.

In my particular case, I was diagnosed by a person.

But often these days when I go in for checkups and screenings, my clinician will ask if they can use an AI tool to take notes while we're doing the checkup.

And I always say yes, because I know that our clinicians are incredibly overtaxed by a range of administrative requirements.

And if AI can take some of that burden off and allow my doctor to look me in the eye when we're having an appointment, I'm going to say yes to that.

But similar to our colleagues, we also have guardrails on our Gemini app so that users will never think that they are talking to a doctor, that they are talking to a licensed physician.

And the Gemini app will also flag for them that if they're seeking medical attention, they should get it from a professional.

Thank you.

Likewise, we've been very clear that we support the role of government here.

At the federal level, we've advanced a proposal for meaningful involvement to set standards and hold companies to account for safety processes, evaluations, security, and the like.

And we welcome the accords that were signed very recently.

To the point of medical usage, if I can use a personal example here, about six months ago, I experienced some symptoms and a longstanding eye disease that might make me go blind at any moment.

And that night, actually, at about 1 a.m., it was Claude that helped me identify that it was serious enough that I needed to be treated very, very quickly.

Claude triaged a very large part of it for me, but what was most important was I could use that to get to the hospital very fast.

It's very clear that both can be transformative and powerful together, certainly in my case and I think countless others, but this is also one that's closest to my heart that we need to be very careful about.

Thank you.

Thank you all so much.