Won questions Milstein on shadow AI use and city contract riders

Followed by Brewer.

Thank you so much, Chair De La Rosa.

So for the panel, what I have questions about is building off of what Councilmember Maloney just said.

The agencies are allowed to currently use Microsoft products and also Adobe.

But it's clear to me and the public that they are using other AI tools.

So not just the passive tools like using Zoom to transcribe their notes and for follow-ups, but in addition to that, they're using ChatGPT, they're using Claude, they're using NanoBanana Pro, because we're seeing it for agencies like HPD, DOE, DEP.

They're creating flyers for public consumption.

They're using it for their public emails.

They're using it for public messaging.

So what are you actually doing?

Because I want to know what the public policy is from the mayor on how the agencies are using AI and how you are.

Regulating it internally as well.

And also I would like to build off of that for city contracts.

What are you doing currently for data privacy writers in city contracts for all procurement of AI companies for making sure that our constituent data is not being used to train their AI models?

And is there a clause at all or a rider in the city contract, especially with co-pilot, since that's the only real one that you have?

And people are just using their own personal accounts or free accounts for everything else that they're currently using for their day-to-day jobs.

Thank you for the question, Council Member.

So let me first address the question about the AI rider.

We shared that with...

general counsels across the city this summer with an updated AI writer.

And I want to just make sure I've got the right language here, so I'm going to pull that up.

Let's see.

Yeah, because my privacy concerns are not just for the contracts that exist, but it is for the free-willing use of agency staff that are using non-city enterprise-licensed AI.

Yeah.

Yeah.

So I'll first address the writer, and then we'll talk about the shadow use.

So the writer, as I said, was distributed to agencies citywide this summer, and it's intended to mitigate risks and apply the city's policies and standards to AI products and features.

It's available for all city software contracts, and our legal team recommends that agencies use it even where no AI component is currently contemplated.

The writer includes terms relating to intellectual property, security, how the city's data may be used, and privacy.

And so that is now active and in use, and I think that with also a proactive component of recommending that it's used for all manner of technology.

In terms of the shadow use, which I think is probably both at the individual and at the agency level, we are not in a...

Technical position to monitor that.

But I certainly, we

as an agency share concerns about that.

And part of what we're interested in doing is making sure that agencies have the tools that they need that are properly regulated.

Microsoft tools, for example, where data can't be shared or used to train their models, that agencies have the tools that they need to do the jobs that they have so that they are not turning to outside tools.

So who is responsible if you are unable to regulate shadow use of AI tools for day-to-day use, especially with constituent data?

Who is going to regulate that?

I'd like to follow up with you on that question.

Okay.

Thank you, Council Member.

So I know the admin has a hard stop at 530.

We have about six more member questions.

So if you all could just hang on tight, we're going to try to get through this.

Members, please consolidate the questions.