Are There Title IX Implications When Universities Use AI for Recruitment or Curriculum Development?
| | Title IX
Title IX of the Education Amendments of 1972 prohibits sex-based discrimination in all aspects of academia. The law first gained attention for the effect on athletic programs, and over 50 years later, athletes are still using the Title IX complaint process and lawsuits to gain fair opportunities. Title IX has also captured a lot of attention because the complaint process is used to address accusations of sexual misconduct such as assault or stalking. But the guarantees of Title IX extend far beyond these two aspects of academic life.
Schools are generally not supposed to discriminate on the basis of sex when recruiting or accepting students or providing other academic opportunities. When Title IX was enacted, the majority of college students were males, and females made up less than 10% of graduate school enrollment in programs such as law and medicine. Following the enactment of Title IX, female enrollment climbed steadily. Schools also adapted curriculum and academic programming to avoid discrimination. But as the use of artificial intelligence increases in academia, it is important to consider whether Title IX rights could be at risk when AI is used for recruitment or curriculum development.
AI in Recruitment and Admissions
The obligations of Title IX start with recruitment of students and staff. The way a position is described, or a program is advertised, should not be biased on sex-based factors, including stereotypes. Recruitment of student athletes should be proportional and equitable.
However, when AI is used to create advertising and promotional material, it can reinforce gender stereotypes and biases. This is because artificial intelligence uses historical data, and in the academic world, the historical data often reflects the past inequities that Title IX was created to overcome. AI frequently delivers advertising based on historical engagement. Patterns of gender inequalities can be reproduced repeatedly and become worse when historical models form the framework for action.
The problems can intensify and seriously infringe on student and staff rights when AI is used in the admissions process or in hiring decisions. Decisions that rest in any way on historical data can perpetuate problematic patterns from the past, such as the underrepresentation of female students in certain engineering fields. AI programs operate on hidden algorithms, making it difficult to trace the decision-making process.
While the results can be discriminatory for potential students and staff, the reliance on AI could potentially make it easier to seek justice for a Title IX violation. When faced with an outcome that appears to be evidence of discrimination on the basis of sex, the burden would shift to the institution to prove the lack of discriminatory intent in admissions decisions. The schools may have difficulty overcoming this challenge if they are not entirely certain why their AI evaluation tools rejected certain candidates.
AI in Curriculum Development
Studies from Stanford and elsewhere have shown that when generative AI is used to create educational content, the results often display inappropriate gender biases, and there are currently limited methods for detecting and evaluating these biases. When students are required to work from a curriculum that reinforces gender stereotypes or discriminates in other ways, they are being deprived of equal educational opportunities in violation of Title IX.
Departments and individual instructors who are pressed for time or focused on research rather than instruction increasingly rely on artificial intelligence to develop curriculum for their classes. Unless the bias and violation are called to their attention by the filing of a Title IX complaint, these instructors may be unaware of the impact, and the cumulative effect can increase the detrimental effects.
Use of AI in Instructional Processes
When AI programs are used to provide feedback and instruction, the effects of programming biases can be very destructive. Learning opportunities become skewed. AI-powered “personalization” processes can create different educational opportunities based on a student’s sex, gender identity, or behavior.
But the effects may be most detrimental when artificial intelligence is used to evaluate student work. Automated grading systems have been known to favor certain traits, leading to disparate impacts on students and unfairly hindering opportunities for advancement.
Assessment tools based on AI can reward certain submissions based on factors that have nothing to do with mastery of the subject but that are based on gender traits. Certain styles of speech or a focus on particular issues can be interpreted as errors rather than preferences. It is important for professors using AI tools to evaluate student performance to consider the impacts and whether the effects reveal a bias.
This is particularly important when AI is being used to assess whether a student has inappropriately used AI to complete an assignment. When accusations of AI cheating are based solely on the results of flawed detection programs, experienced student defense lawyers understand how to expose the program’s deficiencies. This not only protects students who are falsely accused, but it also calls into question the reputation of the person or department making the accusation.
Understanding and Protecting Title IX Rights
In this day and age, when a college education requires years of intense preparation and an investment of income, it is important for students to understand their rights and the steps they can take to protect them. If a college infringes on the right to an equal education, it is important to speak up and assert those rights.
The federal Title IX scheme requires colleges and universities to have a designated Title IX Coordinator and a process for filing and adjudicating complaints of violations. When violations cause serious harm, or a student is falsely accused of violating Title IX or campus codes of conduct, it is wise to work with an attorney who understands how to protect student rights in campus proceedings and in court.
At Nesenoff & Miltenberg, we have championed students’ rights and defended students, faculty, and staff against unwarranted accusations for decades on campuses across the country. If you have a concern you’d like to discuss, we invite you to schedule a confidential consultation with our team.