import pandas as pd
import numpy as np
import os

import matplotlib.pyplot as plt
import plotly.express as px
import plotly.graph_objects as go
pd.options.plotting.backend = 'plotly'
TEMPLATE = 'seaborn'

import warnings
warnings.simplefilter('ignore')

Lecture 27 – Fairness, Conclusion

DSC 80, Winter 2023

Announcements

• Project 5 (prediction) is due on Thursday, March 23rd at 11:59PM (no slip days allowed)!
• There is no live lecture today; videos will be posted instead.
• The Final Exam is on Wednesday, March 22nd from 11:30AM-2:30PM, location TBD.
  • Lectures 1-26 (everything before this Friday) are in scope, as are all assignments.
  • You can bring 2 two-sided notes sheets. More details to come on Ed.
  • practice.dsc80.com now contains 3 past finals. Start reviewing!
• If at least 80% of the class fills out BOTH CAPEs and the End-of-Quarter Survey, then everyone will receive an extra 0.5% added to their overall course grade.
  • Deadline: Saturday, March 18th at 8AM.

This lecture will not be delivered live! Instead, a pre-recorded version of this lecture can be found here.

Agenda

• Fairness.
• Parity measures.
• Example: Loan approval.
• Parting thoughts.

Fairness

Fairness: why do we care?

• Sometimes, a model performs better for certain groups than others; in such cases we say the model is unfair.
• Since ML models are now used in processes that significantly affect human lives, it is important that they are fair!
  • Job applications and college admissions.
  • Criminal sentencing and parole grants.
  • Predictive policing.
  • Credit and loans.

Example: COMPAS and recidivism prediction

COMPAS (Correctional Offender Management Profiling for Alternative Sanctions) is a "black-box" model that estimates the likelihood that someone who has commited a crime will recidivate (commit another crime).

Propublica found that the model's false positive rate is higher for African-Americans than it is for White Americans, and that its false negative rate is lower for African-Americans than it is for White Americans.

Example: Facial recognition

• The table below comes from a paper that analyzes several "gender classifiers", and shows that popular classifiers perform much worse for women and those with darker skin colors.
• Police departments are beginning to use these models for surveillance.
• Self-driving cars use similar models to recognize pedestrians!

Note:

$$PPV = \text{precision} = \frac{TP}{TP+FP},\:\:\:\:\:\: TPR = \text{recall} = \frac{TP}{TP + FN}, \:\:\:\:\:\: FPR = \frac{FP}{FP+TN}$$

How does bias occur?

Remember, our models learn patterns from the training data. Various sources of bias may be present within training data:

• Training data may not be representative of the population.
  • There may be fewer data points for minority groups, leading to poorer model performance.
• The features chosen may be more useful in making predictions for certain groups than others.
• Training data may encode existing human biases.

Example: Gender associations

• English is not a gendered language – words like "teacher" and "scientist" are not inherently gendered (unlike in, say, French).
• However, English does have gendered pronouns (e.g. "he", "she").
• Humans subconsciously associate certain words with certain genders.
• What gender does English associate the following words with?

soldier, teacher, nurse, doctor, dog, cat, president, nanny

Example: Gender associations

• Unlike English, Turkish 🇹🇷 does not have gendered pronouns – there is only a single, gender-neutral pronoun ("o").
• Let's see what happens when we use Google Translate to translate Turkish sentences that should be gender-neutral back to English.
• Click this link to follow along.
• Why is this happening?
  • Answer: Google Translate is "trained" on a large corpus of English text, and these associations are present in those English texts.
  • Ideally, the results should contain a gender-neutral singular "they", rather than "he" or "she".

Example: Image searches

A 2015 study examined the image queries of vocations and the gender makeup in the search results. Since 2015, the behavior of Google Images has been improved.

In 2015, a Google Images search for "nurse" returned...

Search for "nurse" now, what do you see?

In 2015, a Google Images search for "doctor" returned...

Search for "doctor" now, what do you see?

Ethics: What gender ratio should we expect in the results?

• Should it be 50/50?
• Should it reflect the true gender distribution of those jobs?
• More generally, what do you expect from your search results?
  • This is a philosophical and ethical question, but one that we need to think about as data scientists.

Excerpts:

"male-dominated professions tend to have even more men in their results than would be expected if the proportions reflected real-world distributions.

"People’s existing perceptions of gender ratios in occupations are quite accurate, but that manipulated search results have an effect on perceptions."

How did this unequal representation occur?

• The training data that Google Images searches from encoded existing biases.
  • While 60% of doctors may be male, 80% of photos (including stock photos) of doctors on the internet may be of male doctors.
• Models (like PageRank) that "rank" images find the, say, 5 "most relevant" image, not the 5 "most typical" images.

Parity measures

Notation

• $C$ is a binary classifier.
  • $C \in \{0, 1\}$ is the prediction that the classifier makes.
  • For instance, $C$ may predict whether or not an assignment is plagiarized.
• $Y \in \{0,1\}$ is the "true" label.
• $A \in \{0, 1\}$ is a binary attribute of interest.
  • For instance, $A = 1$ may mean that you are a data science major, and $A = 0$ may mean that you are not a data science major.

• Key idea: A classifier $C$ is "fair" if it performs the same for individuals in group $A$ and individuals outside of group $A$.
  • But what do we mean by "the same"?

Demographic parity

• A classifier $C$ achieves demographic parity if the proportion of the population for which $C = 1$ is the same both within A and outside A. $$\mathbb{P}(C=1|A=1) = \mathbb{P}(C=1|A\neq 1)$$

• The assumption of demographic parity: the proportion of times the classifier predicts 1 is independent of $A$.

• Example 1: $C$ is a binary classifier that predicts whether or not an essay is plagiarized.
  • Suppose $A$ is "class is a science class".
  • If $C$ achieves demographic parity, then the proportion of the population for which an assignment is predicted to be plagiarized should be equal for science and non-science classes.

• Example 2: $C$ is a binary classifier that predicts whether an image is of a doctor.
  • Suppose $A$ is "image is of a woman".
  • If $C$ achieves demographic parity, then the proportion of the population for which the classification is "doctor" should be the same for women and non-women.

Accuracy parity

• Demographic parity is not the only notion of "fairness!"
  • You might expect more instances of plagiarism in non-science classes than you would in science classes; demographic parity says this is unfair but it may not be.

• A classifier $C$ achieves accuracy parity if the proportion of predictions that are classified correctly is the same both within $A$ and outside of $A$.

$$\mathbb{P}(C=Y|A=1) = \mathbb{P}(C=Y|A\neq 1)$$

• The assumption of accuracy parity: the classifier's accuracy should be independent of $A$.

• Example: $C$ is a binary classifier that determines whether someone receives a loan.
  • Suppose $A$ is "age is less than 25".
  • If the classifier is correct, i.e. if $C = Y$, then either $C$ approves the loan and it is paid off, or $C$ denies the loan and it would have defaulted.
  • If $C$ achieves accuracy parity, then the proportion of correctly classified loans should be the same for those under 25 and those over 25.

True positive parity

• A classifier $C$ achieves true positive parity if the proportion of actually positive individuals that are correctly classified is the same both within $A$ and outside of $A$.

$$\mathbb{P}(C=1|Y=1, A=1) = \mathbb{P}(C=1|Y=1, A\neq 1)$$

• A more natural way to think of true positive parity is as recall parity – if $C$ achieves true positive parity, its recall should be independent of $A$.

Other measures of parity

• We've just scratched the surface with measures of parity.
• Any evaluation metric for a binary classifier can lead to a parity measure – a parity measure requires "similar outcomes" across groups.
  • Precision parity.
  • False positive parity.
• Note: Many of these parity conditions are impossible to satisfy simultaneously!
  • See DSC 167 for more.

Example: Loan approval

LendingClub

LendingClub is a "peer-to-peer lending company"; they used to publish a dataset describing the loans that they approved (fortunately, we downloaded it while it was available).

• 'tag': whether loan was repaid in full (1.0) or defaulted (0.0)
• 'loan_amnt': amount of the loan in dollars
• 'emp_length': number of years employed
• 'home_ownership': whether borrower owns (1.0) or rents (0.0)
• 'inq_last_6mths': number of credit inquiries in last six months
• 'revol_bal': revolving balance on borrows accounts
• 'age': age in years of the borrower (protected attribute)

loans = pd.read_csv('data/loan_vars1.csv', index_col=0)
loans.head()

	loan_amnt	emp_length	home_ownership	inq_last_6mths	revol_bal	age	tag
268309	6400.0	0.0	1.0	1.0	899.0	22.0	0.0
301093	10700.0	10.0	1.0	0.0	29411.0	19.0	0.0
1379211	15000.0	10.0	1.0	2.0	9911.0	48.0	0.0
486795	15000.0	10.0	1.0	2.0	15883.0	35.0	0.0
1481134	22775.0	3.0	1.0	0.0	17008.0	39.0	0.0

The total amount of money loaned was over 5 billion dollars!

loans['loan_amnt'].sum()

5706507225.0

loans.shape[0]

386772

Predicting 'tag'

Let's build a classifier that predicts whether or not a loan was paid in full. If we were a bank, we could use our trained classifier to determine whether to approve someone for a loan!

from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier

X = loans.drop('tag', axis=1)
y = loans.tag
X_train, X_test, y_train, y_test = train_test_split(X, y)

clf = RandomForestClassifier(n_estimators=50)
clf.fit(X_train, y_train)

RandomForestClassifier(n_estimators=50)

Recall, a prediction of 1 means that we predict that the loan will be paid in full.

y_pred = clf.predict(X_test)
y_pred

array([1., 1., 1., ..., 0., 1., 0.])

clf.score(X_test, y_test)

0.7127196384433206

from sklearn import metrics

metrics.plot_confusion_matrix(clf, X_test, y_test);

Precision

$$\text{precision} = \frac{TP}{TP+FP}$$

Precision describes the proportion of loans that were approved that would have been paid back.

metrics.precision_score(y_test, y_pred)

0.7715067587422862

If we subtract the precision from 1, we get the proportion of loans that were approved that would not have been paid back. This is known as the false discovery rate.

$$\frac{FP}{TP + FP} = 1 - \text{precision}$$

1 - metrics.precision_score(y_test, y_pred)

0.22849324125771375

Recall

$$\text{recall} = \frac{TP}{TP + FN}$$

Recall describes the proportion of loans that would have been paid back that were actually approved.

metrics.recall_score(y_test, y_pred)

0.7325439453125

If we subtract the recall from 1, we get the proportion of loans that would have been paid back that were denied. This is known as the false negative rate.

$$\frac{FN}{TP + FN} = 1 - \text{recall}$$

1 - metrics.recall_score(y_test, y_pred)

0.2674560546875

From both the perspective of the bank and the lendee, a high false negative rate is bad!

• The bank left money on the table – the lendee would have paid back the loan, but they weren't approved for a loan.
• The lendee deserved the loan, but weren't given one.

False negative rate by age

results = X_test
results['age_bracket'] = results['age'].apply(lambda x: 5 * (x // 5 + 1))
results['prediction'] = y_pred
results['tag'] = y_test

(
    results
    .groupby('age_bracket')
    .apply(lambda x: 1 - metrics.recall_score(x['tag'], x['prediction']))
    .plot(kind='bar', title='False Negative Rate by Age Group')
)

Computing parity measures

• $C$: Our random forest classifier (1 if we approved the loan, 0 if we denied it).
• $Y$: Whether or not they truly paid off the loan (1) or defaulted (0).
• $A$: Whether or not they were under 25 (1 if under, 0 if above).

results['is_young'] = (results.age < 25).replace({True: 'young', False: 'old'})

First, let's compute the proportion of loans that were approved in each group. If these two numbers are the same, $C$ achieves demographic parity.

results.groupby('is_young')['prediction'].mean().to_frame()

	prediction
is_young
old	0.682932
young	0.301763

$C$ evidently does not achieve demographic parity – older people are approved for loans far more often! Note that this doesn't factor in whether they were correctly approved or incorrectly approved.

Now, let's compute the accuracy of $C$ in each group. If these two numbers are the same, $C$ achieves accuracy parity.

(
    results
    .groupby('is_young')
    .apply(lambda x: metrics.accuracy_score(x['tag'], x['prediction']))
    .rename('accuracy')
    .to_frame()
)

	accuracy
is_young
old	0.729195
young	0.676788

Hmm... These numbers look much more similar than before!

Is this difference in accuracy significant?

Let's run a permutation test to see if the difference in accuracy is significant.

• Null Hypothesis: The classifier's accuracy is the same for both young people and old people, and any differences are due to chance.
• Alternative Hypothesis: The classifier's accuracy is higher for old people.
• Test statistic: Difference in accuracy (young minus old).
• Significance level: 0.01.

obs = results.groupby('is_young').apply(lambda x: metrics.accuracy_score(x['tag'], x['prediction'])).diff().iloc[-1]
obs

-0.05240731701910695

diff_in_acc = []
for _ in range(100):
    s = (
        results[['is_young', 'prediction', 'tag']]
        .assign(is_young=results.is_young.sample(frac=1.0, replace=False).reset_index(drop=True))
        .groupby('is_young')
        .apply(lambda x: metrics.accuracy_score(x['tag'], x['prediction']))
        .diff()
        .iloc[-1]
    )
    
    diff_in_acc.append(s)

fig = pd.Series(diff_in_acc).plot(kind='hist', histnorm='probability', nbins=20,
                            title='Difference in Accuracy (Young - Old)')
fig.add_vline(x=obs, line_color='red')
fig.update_layout(xaxis_range=[-0.1, 0.05])
fig.add_annotation(text='<span style="color:red">Observed Difference in Accuracy</span>', x=-0.075,showarrow=False, y=0.17)

It seems like the difference in accuracy across the two groups is significant, despite being only ~6%. Thus, $C$ likely does not achieve accuracy parity.

Ethical questions of fairness

• Question: Is it "fair" to deny loans to younger people at a higher rate?

• One answer: yes!
  • Young people default more often.
  • To have same level of accuracy, we need to deny them loans more often.

• Other answer: no!
  • Accuracy isn't everything.
  • Younger people need loans to buy houses, pay for school, etc.
  • The bank should be required to take on higher risk; this is the cost of operating in a society.

• Federal law prevents age from being used as a determining factor in denying a loan.

Not only should we use 'age' to determine whether or not to approve a loan, but we also shouldn't use other features that are strongly correlated with 'age', like 'emp_length'.

loans

	loan_amnt	emp_length	home_ownership	inq_last_6mths	revol_bal	age	tag
268309	6400.0	0.0	1.0	1.0	899.0	22.0	0.0
301093	10700.0	10.0	1.0	0.0	29411.0	19.0	0.0
1379211	15000.0	10.0	1.0	2.0	9911.0	48.0	0.0
486795	15000.0	10.0	1.0	2.0	15883.0	35.0	0.0
1481134	22775.0	3.0	1.0	0.0	17008.0	39.0	0.0
...	...	...	...	...	...	...	...
466121	5000.0	1.0	0.0	0.0	8905.0	23.0	1.0
1354376	2100.0	10.0	1.0	0.0	14740.0	38.0	1.0
1150493	5000.0	1.0	1.0	0.0	3842.0	52.0	1.0
686485	6000.0	10.0	0.0	0.0	6529.0	36.0	1.0
342901	15000.0	8.0	1.0	1.0	16060.0	39.0	1.0

386772 rows × 7 columns

Parting thoughts

Course goals ✅

In this course, you...

• Practiced translating potentially vague questions into quantitative questions about measurable observations.
• Learned to reason about 'black-box' processes (e.g. complicated models).
• Understood computational and statistical implications of working with data.
• Learned to use real data tools (e.g. love the documentation!).
• Got a taste of the "life of a data scientist".

Course outcomes ✅

Now, you...

• Are prepared for internships and data science "take home" interviews!
• Are ready to create your own portfolio of personal projects.
• Have the background and maturity to succeed in the upper-division.

Topics covered ✅

We learnt a lot this quarter.

• Week 1: From BabyPandas to Pandas
• Week 2: DataFrames
• Week 3: Messy Data
• Week 4: Hypothesis and Permutation Testing
• Week 5: Missingness Mechanisms and Imputation
• Week 6: Web Scraping, Midterm Exam
• Week 7: Text Data
• Week 8: Feature Engineering, Modeling in scikit-learn
• Week 9: Pipelines and Generalization
• Week 10: Classifier Evaluation, Fairness Criteria
• Week 11: Final Exam

Thank you!

• This course would not have been possible without our 11 tutors: Nicole Brye, John Driscoll, Doris Gao, Yuxin Guo, Daniel Li, Weiyue Li, Karthikeya Manchala, Yash Potdar, Ethan Shapiro, Costin Smilovici, and Tiffany Yu.

• Don't be a stranger – our contact information is at dsc80.com/staff!

  • This quarter's course website will remain online permanently at dsc-courses.github.io.

• Apply to be a tutor in the future! Learn more here.

Good luck on the Final Exam, and enjoy your spring break! 🎉