Fairness Testing Across Population Groups

Overview
Fairness testing across population groups is a core practice within the PMI Cognitive Project Management for AI (CPMAI) domain of Supporting Responsible and Trustworthy AI Efforts. It means checking whether an AI system's outputs, errors and impacts are distributed equitably across different groups of people. These groups are often defined by protected or sensitive attributes such as age, gender, race, ethnicity, disability, geography, language or socioeconomic status.

Why Fairness Testing Is Important
1. Preventing harm: AI models learn from historical data, which often reflects past discrimination. Without testing, a model can deny loans, jobs, healthcare or services to certain groups, or flag them unfairly.
2. Regulatory and legal compliance: Several laws prohibit discriminatory outcomes and require bias assessments for high-risk systems. Examples include the EU AI Act, anti-discrimination laws, the GDPR, fair lending laws and NYC Local Law 144 on hiring algorithms.
3. Trust and adoption: Users, customers and regulators will not trust or adopt systems seen as biased. Fairness is a pillar of trustworthy AI, alongside transparency, accountability, privacy, safety and robustness.
4. Business risk: Biased AI causes reputational damage, lawsuits, product recalls and lost revenue.
5. Hidden failures: A model can show high overall accuracy yet perform poorly for a minority subgroup. Aggregate metrics hide these disparities. Only disaggregated testing reveals them.
6. Project success: In CPMAI, a model that fails fairness criteria does not meet its business and ethical requirements. It should not move to operationalization.

What Fairness Testing Is
Fairness testing is the systematic evaluation of an AI model's performance and outcomes, broken down by population group, to find unjustified disparities. Key concepts include:
- Sensitive / protected attributes: characteristics against which discrimination must be avoided.
- Privileged vs. unprivileged groups: the comparison groups used in testing.
- Disaggregated evaluation: calculating metrics separately for each group instead of only overall.
- Intersectionality: testing combinations of attributes (e.g., older women of a particular ethnicity), because bias may appear only at intersections.
- Proxy variables: features such as zip code, name or purchase history that correlate with protected attributes. They can reintroduce bias even when the protected attribute is removed. Removing the sensitive attribute (fairness through unawareness) is usually NOT sufficient.

Common Fairness Metrics
- Demographic (statistical) parity: each group receives positive outcomes at similar rates.
- Disparate impact ratio: the unprivileged group's positive rate divided by the privileged group's rate. The four-fifths (80%) rule flags ratios below 0.8.
- Equal opportunity: true positive rates are equal across groups.
- Equalized odds: both true positive and false positive rates are equal across groups.
- Predictive parity: precision (positive predictive value) is equal across groups.
- Calibration: predicted probabilities mean the same thing for every group.
- Individual fairness: similar individuals receive similar outcomes.
- Counterfactual fairness: the outcome would not change if only the sensitive attribute changed.

Important: Some fairness metrics are mathematically incompatible and cannot all be met at once (the impossibility theorem), especially when base rates differ between groups. Choosing the right metric is a context-driven decision. It involves stakeholders, domain experts, legal/compliance teams and the business owner.

How Fairness Testing Works (Aligned to CPMAI Phases)
CPMAI is an iterative, data-centric methodology with six phases. Fairness is addressed throughout, not only at the end.

Phase I - Business Understanding:
- Identify who is affected by the AI system and which groups may be at risk.
- Define fairness requirements, acceptable thresholds and the chosen fairness definitions.
- Engage diverse stakeholders and identify legal obligations.

Phase II - Data Understanding:
- Assess whether training data represents all relevant groups.
- Look for sampling bias, historical bias, measurement bias, labeling bias and missing data patterns.
- Identify proxy variables.

Phase III - Data Preparation (pre-processing mitigation):
- Rebalance, re-sample or re-weight data.
- Collect additional data for underrepresented groups.
- Correct or audit labels.
- Document datasets, for example with datasheets.

Phase IV - Model Development (in-processing mitigation):
- Use fairness-aware algorithms, constraints or adversarial debiasing.
- Compare candidate models on fairness as well as accuracy.

Phase V - Model Evaluation:
- Run disaggregated and intersectional tests.
- Compute the chosen fairness metrics and compare them with thresholds.
- Apply post-processing mitigation, such as group-specific thresholds where legally allowed.
- Document results in model cards and get go/no-go sign-off.

Phase VI - Model Operationalization:
- Monitor fairness continuously in production, since data drift and concept drift can create new bias.
- Set alerts, run periodic audits and provide feedback and redress channels for affected users.
- Re-test after retraining.

Practical Tools and Artifacts
- Toolkits: IBM AI Fairness 360, Microsoft Fairlearn, Google What-If Tool, Aequitas.
- Documentation: model cards, datasheets for datasets, algorithmic impact assessments, bias audit reports.
- Governance: ethics review boards, human-in-the-loop review for high-stakes decisions, and clear accountability (RACI).

Common Pitfalls
- Relying only on overall accuracy.
- Assuming that removing protected attributes removes bias.
- Testing once, before deployment, and never again.
- Having data scientists pick a fairness metric without stakeholder input.
- Ignoring small subgroups because of low sample sizes. The better approach is to gather more data or report uncertainty.
- Treating fairness as purely technical. It is also social, legal and ethical.

Exam Tips: Answering Questions on Fairness Testing Across Population Groups
1. Prefer the proactive, lifecycle answer. If an option says to address fairness from Business Understanding and data collection onward, it usually beats options that test only at the end.
2. Disaggregate. When a scenario reports high overall accuracy but complaints from one group, the best answer is to evaluate performance per subgroup and investigate the data.
3. Removing the sensitive attribute is a trap. Choose answers that mention proxy variables and actual outcome testing.
4. Stakeholder involvement wins. Defining fairness metrics and thresholds should involve business owners, legal/compliance, domain experts and representatives of affected communities, not the data science team alone.
5. Do not deploy a model that fails fairness criteria. The correct action is usually to pause, mitigate (data, model or post-processing) and re-test. Release-and-fix-later is not the answer.
6. Continuous monitoring. For post-deployment scenarios, choose ongoing fairness monitoring, drift detection and periodic audits.
7. Data-first thinking. CPMAI is data-centric. Many bias problems are best solved by improving data representativeness and label quality.
8. Know the metric definitions. Recognize demographic parity, equal opportunity, equalized odds and disparate impact (the 80% rule). Also know that metrics can conflict, so the choice depends on context.
9. Documentation and transparency. Answers mentioning model cards, bias audit reports or impact assessments signal responsible AI best practice.
10. Iterate. CPMAI is iterative. If fairness issues are found during evaluation, going back to earlier phases (Data Understanding or Data Preparation) is expected and correct.
11. Watch for distractor wording. Be cautious of absolute options such as 'guarantee zero bias' or 'fairness is achieved once tested'. Favor balanced, risk-based and accountable responses.
12. High-stakes means more rigor. For hiring, lending, healthcare, criminal justice or other high-risk domains, choose answers with stronger testing, human oversight and regulatory alignment.

Quick Summary
Fairness testing across population groups means measuring AI outcomes and errors by group, including intersections. It starts with defined fairness goals, a representative data assessment and context-appropriate metrics. Disparities are mitigated at the data, model or output stage, documented, and monitored continuously. On the exam, choose answers that are proactive, stakeholder-driven, data-centric, iterative and focused on preventing harm before deployment.