Maddie Dwyer of CDT on AI risks and governance recommendations
Dwyer from the Center for Democracy and Technology identifies four key AI risks from national research: data breaches, deepfakes, AI system failures, and troubling student-chatbot interactions. She recommends risk management, AI governance, transparency requirements for vendors, model procurement resources, and formal avenues to engage parents and students in AI decisions.
Maddie Dwyer, you may begin.
Chairs and members of the committees, my name is Maddie Dwyer, and I'm a state policy analyst at the Center for Democracy and Technology.
Thank you for the opportunity to testify on behalf of CDT regarding AI, student data, and privacy in New York City public schools.
CDT is a nonprofit, nonpartisan organization that works to advance civil rights and civil liberties in the digital age for everyone.
A key part of that mission is ensuring the use of data and technology in K-12 education is centered around privacy, security, civil rights, safety, and trust.
As has been acknowledged here today in the hearing, K-12 schools across the country, including in New York City, have rapidly adopted AI tools, which is reshaping how students, teachers, and school administrators learn, teach, and work.
This heightened acceleration of AI use in classrooms has happened even amid growing recognition of the risks this technology poses to students.
Our nationally representative polling research has uncovered four key risks of AI that all increase the more a school uses AI.
Data breaches and ransomware attacks, proliferation of deepfakes and non-consensual intimate imagery, AI systems failing to work as intended, and troubling interactions between students.
and chatbots.
The council plays a critical role in addressing these risks so that the potential beneficial uses of AI in schools can be realized.
In particular, the council can take action to address risk management, AI governance, and transparency, including encouraging New York City public schools to conduct more research on how this technology impacts students,
establish transparency requirements for ed tech AI vendors, and mandate creation of model procurement resources to guide AI purchasing decisions.
As the Council and New York City Public Schools evaluate the potential benefits and risks of AI use in schools, they should most importantly
provide formal avenues to engage parents, students, and community members in decisions about whether and how to use AI in the classroom and use this feedback to drive their policymaking.
The written testimony I've submitted for the council goes into more detail on our research, these policy priorities, and highlights further policy interventions.
Once again, I want to express my gratitude for the opportunity to testify, and CDT remains available and ready to offer expertise.
Thank you.
Thank you so much for your testimony.