Greg Pak testifies on AI pilot loopholes and need for opt-in consent
Greg Pak, a comic book writer and organizer, criticizes AI pilot programs for using chatbots to give writing feedback, questions whether pilots use databases from major AI corporations, and argues every student should be required to opt in rather than being enrolled automatically.
Hi, my name is Greg Pak.
I'm a comic book writer, and I organize with Comics Action and the AIM Moratorium.
And I hate consumer-facing AI.
So I very much appreciate both of the chairs for asking pointed questions about the pilot programs in particular that the DOE is opening up for high school students.
I've got some other questions, which I would hope you would continue to ask them.
There's a huge loophole with these.
Supposedly, teachers are not supposed to use AI for assessment, but these pilot programs, or grading, but these pilot programs absolutely seem to be using AI for feedback.
I took a look at the BRISC pilot program online.
And I saw promotional videos which showed kids tapping into computers, talking to chatbots, which were touted as giving them immediate feedback on their writing.
I'm a writer.
I think that's one of the worst things I can imagine, to have a chatbot trying to tell me how to write while I'm writing.
Writing is about process, about thinking.
A chatbot that writes horribly, because we've all seen AI writing, we know it's garbage, shouldn't be used for those kinds of purposes.
Ask about that.
Ask about if the databases of these chatbots are fueled by open AI.
Do they push, do they use the same databases that are clogged with racism and sexism and all the bad things?
Are they using those from the major AI corporations?
We don't know that.
Are they pushing teachers to use AI from chat GPT and all those kinds of things?
We've seen that some of the brisk supposedly touts its integration with chat GPT.
Can kids opt out more to the point every kid should be required to opt into these programs?
That's the consent you were talking about, Chair Krishnan.
I totally agree with that.
Then regarding the AI literacy programs, who's developing these AI literacy programs?
Every AI literacy program I've seen has been influenced by industry.
AI literacy should include AI quitteracy, which is information on how every kid, every person can opt out of AI.
Teach kids how to not use AI-inflected search.
Teach kids not how to de-
Remove AI from their devices.
Kids should not be guinea pigs used by the industry to test their products.
50,000 kids in New York City could be signed up for these programs.
We know how bad AI is for them.
They should not be used as guinea pigs.
And thank you.