W3 Lec Notes

Lecture 1

Bias - where do they come from:

• input
• process
• output

Case Analysis: Tay AI - chatbot that was very unhinged :skull: - got shut down in 16 hrs

Cogniitive bias:

• systematic error in judgement and decision-making common to all human beings which can be due to cognitivie limitations, motivational factors, and/or adaptations to natural environemtns

Case study 2: COMPAS - just a system that rates how risky some people are by just looking at offences → uhh it just ended up being very racist towards spanish and black ppl :skull: d

Historical Bias: state of the worrld in which the data was generated is flawe Measurement bias: happens from the way we choose/utilise/measurement

• bias in the quality of the proxy varies in different groups → the variable we choose to measure by is not great

Aggregation bias: arised when a ‘one-size-fit-all’ model is used for groups with different conditional distributions p(Y|X)

• False conclusions are drawn for a subgroup based on observing other different subgroups or generally when false assumptions about a population affect the model’s outcome and definition

Simpson’s paradox: A hypothetical nutrition study which measured how the outcome, body mass index changes as a function of daily pasta calorie intake

Evaluation bias:

Deployment bias: occurs when the problem the model is intended to solve is different from the way it is actually used. If the end users don’t use the model in the way it is intended, not guarantee the model performs well

Population bias: statistics, demographics, representatives and characteistics different in the user population represented in the datasset/platfrom from original target

Sampling bias: non-random sampling of subgroups. As a result of sampling bias, the trends estimated for one population may not generalise to data collected from a new population

Temporal bias: temporal bias arises from differences in populations and behaviours over time

Social bais: social bias happens when other people’s actions or content coming from them affects our judgement

Reporting bias: occurs when the frequency of events, properties and or otucomes captured in data set does not accurately reflect their real world frequency → ppl only record unusual or memorable things.

L2

Bias → focuses more on the representation Fairness → focusses more on the decisions outcome

Fairness 2 - differnet worldviews:

• what you see is what you get worldview
• we are all equal worldview

Impossibility of fairness:

• “what you see is what you get”

Looking for fairness in the world:

• What is fair in the eyes of a model

Algorithmic Fiarness:

• categories of farieness:
  • individual fairness
    • when you have two similar individuals, you should expect the two people to receive similar results
    • motivated by the observed pitfalls of group fairness
  • group fairness
    • fair between groups → all demographic groups should be treated equally
    • fits well with non-discrimination policies and is easy to evaluatie
  • subgroup fairness

Types of discrimination:

• direct vs indirect discrimination
• systemic discrimination
• sxplainable vs unexplainable discrimination

Definitions of fairness:

• equalised odds
• equal opportunity
  • ensure proportion of ppl who should be selected by the model (‘positives’) that are correctly selected by the model is the same for each group. We refer to this proportion as the true positive rate or sensitivity of the model
• demographic parity
• fairness through awareness
• fairness through unawareness
• treatment equality
• test fairness

Demographic parity: model is fiir if the composition of people who are selected by the model matches the group membership percentage of the total.

Equal accuracy but not equal opportunity Group unaware fairness removes all group membership information from the data set

Question: Is there a tradeoff between privacy and fairnes What is fair to one group might be unfair to another gorup

Point of interest and LBSN data:

• a POI can be deifned as an entity that has a somewhat fixed physical extension like a landmark, a building or a city