Transparent Reporting on Data Sources and Preprocessing

Overview
Transparent reporting on data sources and preprocessing is the practice of clearly documenting and communicating where an AI system's data came from and how it was collected, cleaned, transformed, labeled, filtered, augmented and prepared before being used to train, validate or operate a model. In the PMI-CPMAI (Cognitive Project Management for AI) framework, this practice sits within the domain of supporting responsible and trustworthy AI. It connects directly to CPMAI Phase II (Data Understanding) and Phase III (Data Preparation). Because AI is data-centric, the data is effectively the 'source code' of model behavior. Being transparent about that data is therefore central to trust, accountability and compliance.

Why It Is Important
1. Trust and accountability: Stakeholders, regulators, auditors and end users cannot judge whether an AI system is fair, safe or reliable without knowing what data shaped it. Transparency creates a traceable chain of responsibility.
2. Bias detection and fairness: Many harmful outcomes come from unrepresentative samples, historical bias, labeling bias or preprocessing choices. Examples include dropping records with missing values that disproportionately belong to one group. Documentation lets reviewers spot these issues.
3. Reproducibility: Documented preprocessing steps let teams recreate a model, debug it, retrain it and compare versions. Without this, results cannot be verified.
4. Regulatory and legal compliance: Several laws require data governance documentation, provenance records and disclosure of data handling practices. Examples include the EU AI Act, GDPR, CCPA and sector rules such as HIPAA.
5. Data rights, consent and licensing: Reporting sources confirms the organization has the legal right to use the data. It also shows whether consent was obtained and whether licenses or terms of use are respected.
6. Risk management: Transparency surfaces data quality risks, drift risks and security or privacy risks early. This is cheaper than finding them after deployment.
7. Informed decision-making: Business owners can decide go or no-go, and judge whether a model fits a new context, only if they understand the data's limits.
8. Explainability support: Explaining model outputs is much easier when the inputs and their transformations are documented.

What It Is
Transparent reporting covers two broad areas.

A. Data sources (provenance and lineage)

• Origin of each dataset: internal systems, third-party vendors, public datasets, web scraping, sensors, user-generated content or synthetic data.
• Collection methods and time periods.
• Ownership, licensing, consent basis and usage restrictions.
• Population represented and known gaps (who or what is missing).
• Data volume, format, granularity and update frequency.
• Known quality issues, biases and limitations.
• Presence of personal, sensitive or protected attributes.

B. Preprocessing (transformation and preparation)

• Cleaning: handling missing values, duplicates, outliers and errors.
• Filtering and sampling decisions, such as inclusion and exclusion criteria, resampling and balancing.
• Labeling and annotation: who labeled, guidelines used, inter-annotator agreement and quality checks.
• Feature engineering, encoding, normalization and scaling.
• Anonymization, pseudonymization, de-identification and aggregation.
• Data augmentation and synthetic data generation.
• Train, validation and test splits and how leakage was prevented.
• Versioning of datasets and pipelines.
• Rationale for each significant decision and its potential impact.

Common artifacts and tools

• Datasheets for Datasets (Gebru et al.): a structured questionnaire covering motivation, composition, collection, preprocessing, uses, distribution and maintenance.
• Data Cards and Data Statements (the latter common in NLP).
• Model Cards: these reference the training and evaluation data used.
• Data lineage and provenance records: often captured automatically by data catalogs or MLOps tools.
• Data dictionaries and metadata repositories.
• Data quality reports and exploratory data analysis summaries.
• Change logs, version control and dataset versioning tools.
• Data Protection Impact Assessments (DPIAs) and AI impact assessments.

How It Works in Practice (CPMAI Lifecycle View)
Phase I, Business Understanding: Define transparency requirements up front. Identify stakeholders who need data information, such as regulators, auditors, users and the ethics board. Determine applicable regulations and organizational policies.

Phase II, Data Understanding: Inventory and catalog all candidate data sources. Record provenance, ownership, consent, licensing, collection methods, representativeness and quality findings. Flag sensitive attributes and known biases. This is where datasheets usually begin.

Phase III, Data Preparation: Log every transformation step and the reason for it. Version the datasets and the pipeline code. Record labeling procedures and quality controls. Document anonymization methods. Assess whether each preprocessing step could introduce or amplify bias.

Phase IV, Model Development: Link each model version to the exact dataset version it was trained on. Note any data-related limitations in the model card.

Phase V, Model Evaluation: Report evaluation data sources and test set composition. Report performance across subgroups and any data-driven limits on generalization.

Phase VI, Model Operationalization: Keep documentation updated as data changes. Monitor for data drift, record retraining data, and preserve audit trails. Make appropriate summaries available to stakeholders.

Key principles

• Proactive, not reactive: document as you go, not after an audit request.
• Audience-appropriate: technical detail for data scientists and auditors, plain-language summaries for business users and the public.
• Living documentation: update with every data refresh, retraining or pipeline change.
• Honest about limitations: disclose gaps, biases and uncertainties rather than hiding them.
• Balanced with privacy and IP: transparency does not mean exposing personal data or trade secrets. Use summaries, aggregate statistics and controlled access.
• Governed: clear ownership, such as a data steward, plus review and approval workflows.

Example Scenario
A bank builds a credit risk model. Transparent reporting would cover several points:

• The data came from internal loan records (2015 to 2023) and a third-party credit bureau under license.
• Applicants rejected before 2018 were excluded because outcomes were unavailable. This is a known selection bias.
• Missing income values were imputed using the median by region.
• The protected attributes race and gender were removed from features but retained separately for fairness testing.
• Zip code was evaluated as a potential proxy variable.
• All steps are versioned and linked to model version 2.1.

This lets auditors and regulators evaluate fairness and compliance.

Common Pitfalls

• Treating documentation as a one-time deliverable.
• Documenting only the final dataset and not the transformations.
• Ignoring third-party or open-source data provenance and licensing.
• Failing to record why decisions were made, recording only what was done.
• Over-disclosing sensitive information in the name of transparency.
• Not linking data versions to model versions.
• Assuming removing protected attributes eliminates bias. Proxies remain.

Exam Tips: Answering Questions on Transparent Reporting on Data Sources and Preprocessing
1. Choose the proactive, documented, governed option. When a question asks what the project manager should do, prefer answers that establish documentation, lineage and governance early. Avoid answers that fix problems silently or defer documentation.

2. Map the question to the CPMAI phase. Questions about cataloging sources, assessing quality and checking consent point to Data Understanding (Phase II). Questions about logging cleaning, labeling and transformation steps point to Data Preparation (Phase III). Linking data to models and ongoing updates points to Evaluation and Operationalization.

3. Recognize the artifacts. Know that datasheets, data cards and data statements document datasets. Model cards document models and reference their data. Lineage and versioning tools provide traceability. If an option names one of these as the documentation solution, it is often correct.

4. Transparency and privacy are complementary. Watch for distractors that publish raw data to be transparent. The correct answer usually discloses metadata, summaries and methods while protecting personal data and IP.

5. Bias questions often hinge on preprocessing. If a scenario describes unfair outcomes, look for answers that review data sources and preprocessing decisions. Examples include sampling, exclusion rules, imputation, labeling and proxy variables. Look for documenting the finding and the mitigation.

6. Third-party data needs provenance verification. For vendor or scraped data, the best answer usually verifies licensing, consent and collection methods and documents them before use.

7. Reproducibility and auditability are key words. Answers mentioning version control, audit trails and traceability between data and model versions are typically strong.

8. Tailor communication to stakeholders. If a question involves executives, customers or regulators, choose the option that gives an audience-appropriate summary rather than raw technical logs.

9. Documentation is continuous. Eliminate options implying that documentation is finished at deployment. CPMAI is iterative, and data documentation must be updated with retraining and drift.

10. Watch for the 'why' over the 'what'. Superior answers capture the rationale behind decisions, because that supports accountability and later review.

11. Beware of absolute or shortcut answers. Options like 'remove all sensitive attributes and no further documentation is needed' or 'the vendor is responsible for data transparency' are usually wrong. The project team retains accountability.

12. Link to trustworthy AI pillars. Transparency supports fairness, accountability, explainability, privacy and compliance. When an option connects data reporting to these outcomes, it is likely aligned with the exam's perspective.

Quick Recall Summary

• What: documenting and communicating data origin, rights, composition, quality and every preprocessing step with rationale.
• Why: trust, fairness, reproducibility, compliance, risk reduction and explainability.
• How: datasheets, data cards, lineage, versioning, data dictionaries and impact assessments, maintained across all CPMAI phases.
• Exam mindset: early, continuous, governed, audience-appropriate and privacy-respecting transparency.