Data Lineage and Usage Documentation for Audit

Overview
In the PMI Certified Professional in Managing AI (PMI-CPMAI) framework, which builds on the CPMAI methodology (Cognitive Project Management for AI), the Identify Data Needs work belongs to Phase II, Data Understanding. Here the team works out what data the AI solution requires, where that data comes from, and whether it can be used. One of the most important supporting practices in this work is Data Lineage and Usage Documentation for Audit. It is the disciplined recording of where data originated, how it moved and changed, who touched it, and under what rights and purposes it may be used. All of this is captured so that internal or external auditors, regulators and stakeholders can verify the AI system's data foundation.

Why It Is Important
1. AI is data-centric. In AI projects, model behavior is driven by data rather than explicit code. When a model makes a biased, wrong or harmful decision, the investigation almost always leads back to the data. Without lineage, root-cause analysis is guesswork.
2. Regulatory compliance. Many laws and regulations require organizations to demonstrate lawful basis, purpose limitation and traceability of personal and sensitive data. Examples include GDPR, CCPA, HIPAA, the EU AI Act and sector rules such as financial model risk management guidance like SR 11-7. Auditors ask for evidence, not assurances.
3. Trustworthy and responsible AI. Transparency, accountability and explainability, which are core pillars of trustworthy AI, depend on knowing exactly what data trained and fed the model.
4. Reproducibility. Retraining, validating or rolling back a model requires knowing which version of which dataset, with which transformations, produced a given model version.
5. Risk reduction. Documentation exposes licensing violations, consent gaps, data leakage, poisoning risks and unauthorized secondary use before they become legal or reputational crises.
6. Go/no-go decisions. CPMAI emphasizes iterating and asking whether the project is feasible. If data provenance or usage rights cannot be established, the project may need to pause or pivot. Lineage documentation provides the evidence for that decision.
7. Operational continuity. When team members leave, documented lineage preserves organizational knowledge. It also supports model monitoring and drift investigation in the operationalization phase.

What It Is
Data lineage is the end-to-end record of a dataset's life cycle. It covers the following stages:
- Origin/Provenance: the source system, vendor, sensor, public dataset, user-generated content or synthetic generator, with collection date and method.
- Movement: the pipelines, extracts, APIs and storage locations the data passed through.
- Transformations: cleaning, filtering, joins, aggregations, feature engineering, labeling/annotation, augmentation, anonymization and sampling.
- Consumption: which training runs, model versions, reports and downstream systems used the data.

Usage documentation records the permissions and context around the data. It includes:
- Ownership and data stewards
- Licensing terms, contracts and data-sharing agreements
- Consent scope and lawful basis for processing
- Permitted and prohibited purposes, including purpose limitation
- Sensitivity classification (PII, PHI, confidential)
- Retention and deletion requirements
- Access controls and the access log of who used the data and when
- Known limitations, biases and quality issues

Audit readiness means this information is complete, accurate, tamper-evident, versioned and retrievable on demand. An auditor should be able to trace any model output back to its source data and confirm it was used appropriately.

Common Artifacts
- Data inventory / data catalog entries with metadata
- Datasheets for Datasets covering motivation, composition, collection process, preprocessing, uses, distribution and maintenance
- Data cards and Model cards, which link models to training data
- Lineage graphs: directed acyclic graphs showing source-to-model flow
- Data dictionaries and schema documentation
- Labeling guidelines and annotator agreement records
- Data Protection Impact Assessments (DPIAs) and privacy reviews
- Version control records for datasets, using tools such as DVC, lakeFS or Delta Lake, plus experiment tracking such as MLflow
- Audit logs of access and changes
- Decision logs explaining why data was included, excluded or modified

How It Works (Process)
Step 1: Inventory data needs and sources. While identifying data needs, list every candidate source. Capture owner, location, format, volume, refresh frequency and access method.
Step 2: Establish provenance and rights. For each source, confirm how it was collected and whether consent or a license covers the intended AI use. Record any restrictions. Data collected for one purpose cannot automatically be repurposed for model training.
Step 3: Classify sensitivity. Tag PII, PHI, protected attributes and confidential data. Determine the required controls such as masking, anonymization or access restriction.
Step 4: Capture lineage as data flows. Automate metadata capture in pipelines wherever possible. Manual documentation is error-prone and drifts out of date. Record every transformation with its version, parameters and responsible party.
Step 5: Version datasets and link them to models. Each trained model version should reference the exact dataset snapshot, feature definitions, labeling version and code commit used.
Step 6: Document usage and access. Maintain access logs and approvals. Record which teams and systems consumed the data and for what purpose.
Step 7: Review and govern. Involve data governance, legal, privacy, security and compliance stakeholders. Obtain sign-offs and record them.
Step 8: Maintain throughout the life cycle. Update the documentation during data preparation, modeling, evaluation and operationalization. Continue updating it during monitoring and retraining, since new data and drift create new lineage.
Step 9: Enable audit retrieval. Store documentation centrally with immutable or tamper-evident logs, so auditors can independently trace and verify.

Roles
- AI project manager: ensures documentation is planned, resourced and part of the definition of done.
- Data owners and stewards: are accountable for source accuracy and rights.
- Data engineers: implement automated lineage capture.
- Data scientists: document feature engineering, sampling and training dataset versions.
- Legal, privacy and compliance: validate lawful use and retention.
- Auditors: independently verify.

Relationship to the CPMAI Phases
- Phase I, Business Understanding: defines the purpose, which sets the boundaries of permitted data use.
- Phase II, Data Understanding: identifies sources, provenance, rights and quality. This is where lineage documentation starts.
- Phase III, Data Preparation: records transformations, labeling and augmentation.
- Phase IV, Model Development: links model versions to datasets.
- Phase V, Model Evaluation: evaluation datasets are documented and kept separate to prevent leakage.
- Phase VI, Operationalization: production data, monitoring and retraining lineage are maintained.

Common Pitfalls
- Treating documentation as an after-the-fact task done just before an audit
- Relying on tribal knowledge or spreadsheets that are not version-controlled
- Assuming publicly available or scraped data is free to use
- Ignoring third-party or vendor data terms
- Failing to document labeling processes and annotator instructions
- Not linking model versions to the exact data snapshot
- Overlooking lineage for production inference data and retraining cycles
- Documenting technical flow but not consent, purpose and retention

Exam Tips: Answering Questions on Data Lineage and Usage Documentation for Audit
1. Choose proactive over reactive. The best answer usually establishes lineage and usage documentation early, during Identify Data Needs and Data Understanding. Reconstructing it later or when an auditor asks is rarely correct.
2. Rights before use. If a scenario involves data whose license, consent or provenance is unclear, the correct action is typically to verify rights and document them, or to escalate to legal and compliance, before using the data. Training anyway and fixing it later is wrong.
3. Purpose limitation is a frequent trap. Data collected for one purpose, such as billing, cannot simply be repurposed for AI training without a review of consent and lawful basis. Watch for answers that assume existing internal data is automatically usable.
4. Traceability equals reproducibility. When a question asks how to investigate a biased or failed model, look for answers that use lineage to trace back to dataset versions, transformations and labeling.
5. Prefer automation and versioning. Between manual spreadsheets and automated metadata capture with dataset version control, the exam favors automated, versioned, tamper-evident approaches.
6. Involve the right stakeholders. Governance, legal, privacy, security and data stewards should be consulted. Answers in which the data scientist decides alone on sensitive data use are usually wrong.
7. Remember the full life cycle. Lineage does not stop at deployment. Production data, drift monitoring and retraining all need documentation, so pick answers reflecting continuous maintenance.
8. Know the artifacts. Recognize datasheets for datasets, data cards, model cards, data catalogs, DPIAs, data dictionaries and audit logs, and what each provides.
9. Link data to models. If asked what an auditor needs to verify a model, the answer involves the mapping from model version to the specific training, validation and test data snapshots and their sources.
10. Go/no-go thinking. CPMAI is iterative. If data provenance or rights cannot be established, the appropriate PM response may be to return to an earlier phase, find alternative data or adjust scope, rather than proceed.
11. Read for keywords. Terms like auditor requests, regulator, cannot explain where data came from, third-party dataset, consent, retraining and reproduce results signal this topic.
12. Eliminate extreme answers. Discard options that ignore compliance for speed, delete data to avoid audits, or halt the entire project without assessment. The best answer balances governance with progress through documentation, verification and escalation.

Sample Question
An AI team plans to train a churn model using customer support transcripts collected years ago. An internal auditor asks whether the data may be used for this purpose. What should the project manager do first?
A) Begin training, since the data is internal.
B) Anonymize the data and proceed without further review.
C) Verify and document the provenance, consent scope and permitted uses of the transcripts with data owners and legal/privacy, and record the findings in the data lineage and usage documentation.
D) Discard the dataset entirely.
Answer: C. It establishes rights and traceability before use, involves the proper stakeholders and produces audit-ready evidence. Anonymization (B) may help, but it does not replace a purpose and consent review.

Key Takeaway
Data lineage and usage documentation answers four audit questions for every piece of data in an AI system. Where did it come from? What happened to it? Who used it, and for what? Were we allowed to? Start this documentation when data needs are identified, automate and version it, and maintain it across the whole AI life cycle.