CSCI 3358-- Foundations of algorithmic (un)fairness

Instructor: Jessie Finocchiaro

Tuesday, Thursdays 9a-10:15a, 245 Beacon St Room 125A

Email: {first name}.{last name}@bc.edu

Office hours: W 3-4, H 10:30-12

Class Calendar (office hours, due date reminders, etc.) Must be logged in to BC account.

This is a draft of the course plan. Schedule and grading schemes are subject to change

Course description

Computation is being increasingly used to support decision-making in our society: algorithmic predictions of repayment likelihood are given to banks to help them determine whether or not they should give someone a loan; in the COVID-19 pandemic, algorithms were used to allocate initially scarce vaccines; facial recognition algorithms allow us to use our faces as "keys" to unlock our phones and even houses. In these high-stakes settings, concerns of fairness and justice are salient. This course will equip students with the mathematical tools to understand and address some of these concerns. Topics will include: how to computationally define and diagnose (un)fairness, the role of uncertainty in fairness, disparate treatment vs disparate impact, and contextualization within US anti-discrimination law.

Prerequisites: CS1, comfort with probability (Randomness and Computation) required.

Course learning goals

By the end of this course, I hope you are able to…

• Identify bias in datasets and algorithms;
• Explain how bias is present, and what algorithmic solutions can supplement non-algorithmic solutions;
• Translate mathematical concepts about parity and fairness into their real-world applications, and be able to explain and acknowledge the shortcomings of algorithmic solutions;
• Communicate these finding to both technical and non-technical audiences.

Tentative schedule

Grading and late policies

Due dates will generally be at 5:00pm. If your submission is uploaded by the time I go to download everyone’s submissions, that is okay. Sometimes I might download submissions right away, but I might not always. This allows some flexibility so that submissions at 5:05pm are less stressful, but waiting on the order of a day enters the danger zone.

If you are not going to be able to closely make a given deadline, please feel welcome to email me and ask for an extension. I will grant two days of extension throughout the semester (for any assignment besides the case study), but in extenuating circumstances, will do my best to be more flexible.

The exception to this is the case study. Because this is an in-class presentation for a slated day, the deadline is firm. If an emergency arises, please notify me as soon as possible.

The first week of class, the students collectively agreed to proceed with a standards-based grading approach, as outlined here.

Participation

I want to make sure our class is a place where you are able to share your experiences and that your classmates have the opportunity to learn from you. In the era of large language models, I unfortunately have to make this as LLM-proof as possible. A classroom where discussion is lively is one that is the most fun and the most learning happens. If you buy in, others will too, and it will be a better experience for everyone.

Notes

Tuesday class structure will tend to be more of a lecture-style format, where Thursdays will dive into a case study going deeper into what we learned on Tuesday. Before we dive into case studies, I will spend the first 30 minutes of Thursday’s class going into more depth and answering questions that you still have after Tuesday’s lecture.

To better understand what you gleaned from class on Tuesday, I am going to ask you upload your notes on Tuesday’s lecture by Wednesday at noon. Since lectures will involve a decent amount of math, which is not easy to type in real time, I encourage you to take handwritten notes. Your upload of these notes can simply be a picture of your handwritten notes.

CS Experience

Computer Science is so much more than programming, and in this course, we are exploring one way CS interweaves with human experience. I ask that you attain 3 “CS Experiences”, which can take place in one of the following ways, but cannot repeat an experience more than twice:

• Attend a CS research seminar (and send me a pic of you at the seminar + a paragraph summarizing what you took away from it)
• Write a ~500 word biography of a computer scientist. If you don’t know who to write a biography of, let me know, and I’ll give you some recommendations related to what you’re interested in.
• Attend a department social event
• Go to a CS-related club (BCCSS, WICS, …)
• Visit the career center for a resume review or apply for a job!

Each experience is with 2% of your final grade, and falls under participation.

Case study presentation

Thursdays in this class will generally be dedicated to digging in deeper to the topic presented on Tuesday. It will be your responsibility to present a case study (approximately 30 minutes) in groups of 2-3. I will provide some references to get you started on the topic, but it is up to you to go deeper and explain the connection to the class material. Feel free to get creative!

Homework

Five homework assignments will be given throughout the semester. The assignments will be long, but material will be progressively covered as the due date draws nearer. Your life will be easier if you don’t procrastinate.

• Homework 1 due: Sept 4 (overview of probability, prereqs)

• Homework 2 due: Sept 20 (group fairness via parity)

• Homework 3 due: October 4 (calibration and individual fairness)

• Homework 4 due: November 1 (utilitarianism, welfare, and social choice)

• Homework 5 due: November 15 (fairness in resource allocation and fair division)

Final project

Project description

Throughout the course, you will be working on a larger-scale class project in groups of 2-3 on a topic of your choosing. There are so many important and fascinating challenges that we simply won’t have time to cover in class. This project is your opportunity to get some hands-on experience with something that piques your interest. Because this project is a large portion of your grade, it shouldn’t be all done between Thanksgiving and finals. To this end, we will have some regular check ins.

The AI Incident Database might be a great place to find inspiration for your project if you need ideas.

Sept 13: Project proposal due. (I will do my best to have feedback and suggested literature to you by Sept 20)

Sept 27: Update 1 due. At this point, you should understand what the challenges are that you’re going to face and how you plan to overcome them.

Oct 11: Update 2 due. At this point, you should have access to data that you will need for your project, if necessary. If applicable, experiments should be planned.

Nov 8: Update 3 due. You should have some preliminary results, at the very least.

I would recommend the work for the project is done before the Thanksgiving holiday so you can relax and enjoy some time off, and only have to work on your final presentation after the break.

Dec 12ish: During final exam period, project presentations. At the beginning of the exam period, technical reports due as well.

Required texts and materials

No textbooks are required for this course, though the following will be helpful resources:

• Fairness and Machine Learning, Limitations and Opportunities by Barocas, Hardt, and Narayanan
• Handbook of Computational Social Choice edited by Brandt, Conitzer, Endriss, Lang, and Procaccia

Communication expectations

I will do my best to be generally responsive, while still protecting my personal boundaries. In general, the best way to contact me is by email.

I will do my best to respond to emails within 24 hours (excluding weekends, which might delay responses), and I expect you to do the same. When grading, I will do my best to have grades uploaded to the portal within a week, but will let the class know if this isn’t attainable.

In writing your emails, please write in complete sentences and take the time to give more context on your problem than you think I need. It will help me get a better idea of your understanding and thinking, and I can provide better feedback with this in mind.

Accommodations and accessibility

Your experience in this class is important to me. If you have already established accommodations with Connors Family Learning Center (learning disabilities and ADHD) and/or Disability Services (all other disabilities), please communicate your approved accommodations to me at your earliest convenience so we can discuss your needs in this course.

If you have or suspect you have a disability but have not yet established services through the Connors Family Learning Center or Disability Services, you can see the relevant website for information about the registration process.

Each office provides resources and coordinates reasonable accommodations for students. Reasonable accommodations are established through an interactive process between you, your instructor, and the relevant office. It is the policy and practice of Boston College to create inclusive and accessible learning environments consistent with federal and state law.

Class recording policy

Boston College seeks to protect the integrity of what transpires in the classroom among students and professor, any course materials prepared by the professor, and the privacy of students and faculty. With this in mind, you are prohibited from recording (audio or video) any lectures, seminars, or other classroom activities without the express permission of the instructor. Authorized recordings (including any made in order to accommodate ADA considerations) and all other course materials (including any materials posted on Canvas) may only be used for the purposes of an individual’s (or group’s) study in the course, and may not be shared with any wider audience on or off campus unless the instructor has explicitly given such permission.

This class or portions of this class will be recorded by the instructor for educational purposes. These recordings will be shared only with students enrolled in the course and will be deleted at the end of the end of the course.

If you ask, I will probably give permission. However, I don’t want to be recorded without my knowledge. That already happens enough in the rest of our lives.

Religious accommodation

Both Massachusetts law and my humanity beg that I make reasonable accommodations to help students avoid negative academic consequences when their religious obligations conflict with academic requirements. I commit to making every reasonable effort to allow students to observe their religious holidays without jeopardizing your success in this course, and I will not ask you to choose between religious observance and academic work, though you will have to make up any work missed during your absence. Please let me know as early as possible if you notice any conflicts between requirements for this course and your religious observances, so that we can agree to a solution.

Inclusion and incident reporting

In a class about fairness and discrimination, the topics of discussion that emerge may be very personal to you, or to your classmates. I expect you to be respectful of everyone in the learning environment (including myself) who share their experiences and to not minimize or dismiss someone’s emotions or experience. We are all here to learn from each other.

Professional courtesy and sensitivity are especially important with respect to individuals and topics dealing with differences of race, culture, religion, politics, sexual orientation, gender, gender variance, and nationalities. Class rosters are provided to the instructor with the student’s legal name. I will gladly honor your request to address you by an alternate name or gender pronoun. Please advise me of your name and pronouns early in the semester so that I may make appropriate changes to my records. You can also update your name and pronouns in Canvas at any time. You can also review the Name Change resource from the Office of Student Services for more information about how you can update your name in various campus systems. If you are an LGBTQ+ student looking for additional support as you prepare to discuss your name or pronouns on campus, you are encouraged to contact Ira Kirschner, Associate Director, Thea Bowman AHANA and Intercultural Center.

In order to advance Boston College’s goal of creating a community in which everyone is treated with respect, dignity, and compassion and where acts of prejudice, hatred, and discrimination are not tolerated, the university created the bias-related incident reporting form. This form allows targets and witnesses to hate crimes or bias-related incidents to easily report the behavior and get connected to networks of care. In subsequent conversations with the Dean of Students Office, students can also get a better handle on the available institutional processes for addressing harm. You can learn more about the process at the Hate Crimes & Bias-Related Incidents Protocol website.

Classroom technology policy

A note on cell phones, texting, and checking one’s email during class: Research has shown us that even having our cell phones on the table in front of us diminishes our ability to learn well; further, taking notes via computer diminishes one’s ability to process information. Checking texts, emails, and messages is also unprofessional and disrespectful to our class community. Please turn off your phone, email, and computer during class; I will do so as well. I appreciate your cooperation with this important aspect of creating a class of which we all want to be a part. If there is a one-time exception (e.g., waiting on news for a family event), please let me know and I will generally be okay with making an exception.

On generative AI: Think of GenAI models like an electric bike. While they have their place and can help cyclists with active recovery, no one will become an elite road cyclist by solely riding an e-bike. In the same way, a LLM doing your homework for you will not help you become a better computer scientist. (Analogy courtesy of Bo Waggoner.)