October 19, 2023

Reps. Sara Jacobs, Lori Trahan Raise Civil Rights Concerns Around Use of AI in Schools

Reps. Sara Jacobs (CA-51) and Lori Trahan (MA-03) led lawmakers in writing to U.S. Secretary of Education Miguel Cardona requesting information on the U.S. Department of Education Office of Civil Rights’ understanding of the impacts of educational technology (EdTech) and artificial intelligence in the classroom. The letter follows a report by the Center for Democracy and Technology outlining serious concerns regarding Edtech and AI’s possible infringement on students’ civil rights in school educational and disciplinary systems.

 

“As recent research from the Center for Democracy and Technology (CDT) highlights, technologies that monitor student activity online, moderate and filter online content, and use predictive analytics raise serious concerns about the application of civil rights laws to schools’ use of these technologies. We are particularly concerned about the application of existing laws that protect students from discriminatory policies and outcomes related to sex, race, disability, and LGBTQ+ status,” the lawmakers wrote. “While the responsible use of AI in schools may enhance students’ educational experience, these examples show a consistent pattern of negative impacts to students in protected classes related to educational technology, including AI-driven technologies. As schools develop and refine policies related to their use of AI, we urge the Department, in coordination with the Office of Science and Technology Policy, to provide clarity on how educators can fulfill their civil rights obligations.”

 

The CDT report found that the misuse of EdTech and AI in the classroom can infringe on the civil rights of students in many ways, including:

 

  • Two-thirds of teachers recently reported that a student at their school was disciplined due to AI-powered monitoring software, with disproportionate impacts for marginalized communities. 
  • 19% percent of LGBTQ+ students reported they or someone they know was outed due to student monitoring software. 
  • Filtering and blocking software has recently been equated to a digital book ban, with one-third of teachers reporting that content associated with LGBTQ+ students and students of color is more likely to be restricted. 
  • 60% of teachers reported that their schools use algorithmic early warning systems that predict whether a student is dropping out of high school, some of which incorporate protected characteristics such as race, gender, and disability status.

 

This research is particularly concerning when considering the links between school disciplinary policies and incarceration rates of our nation’s youth.

 

Reps. Jacobs and Trahan were joined by Reps. Hank Johnson (GA-04), Bonnie Watson Coleman (NJ-12), and Adam Schiff (CA-30) in asking the following questions:

 

  1. Has the Department’s Office of Civil Rights (OCR) received any complaints alleging discrimination that involved, in any way, the use of an educational technology software, program, or service, including AI-driven technologies? If so, how many complaints has OCR received, and how many has OCR investigated?
  2. Has OCR initiated any enforcement actions against potential violations of civil rights law related to the use of education technology?

 

A copy of the letter sent today can be accessed HERE.

 

Rep. Sara Jacobs is a leader in digital privacy, championing the My Body, My Data Act – landmark legislation to protect reproductive and sexual health data. She also co-leads the Secure Access for Essential Reproductive (SAFER) Health Act to strengthen HIPAA in the post-Roe era by prohibiting medical providers from disclosing personal health information related to pregnancy termination or loss without patient consent.

 

###