Chair De La Rosa opening remarks on AI privacy and safety risks

Thank you so much.

Good afternoon, everyone, and welcome to our hearing.

I'm Councilmember Carmen De La Rosa, and I'm the chair of the Committee on Technology.

I'm happy to join my neighbor to the north.

Chair Dinowitz today to discuss an issue that I care about deeply, not only as a chair of this committee, but also as a public school parent who is concerned about the future we are building for the next generation.

Technology has become an unavoidable part of our lives, with artificial intelligence changing the way we work, learn, and make decisions.

This is why we have an obligation to ensure that innovation serves people and that technological progress is guided by accountability, transparency, safety, and public interest.

Artificial intelligence tools are used by the Department of Education in its daily operations, as well as in classrooms.

The use of AI in education has raised concerns that it may affect students'critical thinking.

and compromise the traditional learning process.

Another concern that people have is related to digital privacy, safety, and inequality.

Some AI tools use personal data as training data.

That data could include students'information such as names, dates of birth, behavioral analytics, medical history, family income, ethnicity, and even facial geometry.

Without proper safeguards, this information could be used to profile students, predict or surveil their behavior, or make assumptions about their abilities and future opportunities.

This information can also be breached.

In fact, based on a recent New York State Comptroller's report, between January 2023 and February 2025, DOE experienced 141 data breaches incidents.

Based on our conversations and public reporting, some tools used by the DOE may collect data without parents'knowledge or consent.