Validating Completeness and Accuracy During Collection

Overview
Validating completeness and accuracy during collection is a core practice in the PMI Cognitive Project Management for AI (CPMAI) methodology. It belongs to the Data Understanding work of identifying data needs (CPMAI Phase II). The idea is simple: you check, as data is gathered, that it is complete (nothing important is missing) and accurate (it correctly represents reality). You do not wait until model training or deployment to find out the data was flawed.

Why It Is Important
AI projects are data projects first. CPMAI stresses that most AI project failures trace back to data problems, not algorithm problems. The principle of garbage in, garbage out applies even more strongly to machine learning, because models learn patterns directly from the data they are given.

Validating during collection matters for several reasons:
- Cost of late discovery: Defects found during modeling or after deployment force expensive rework. You may need to recollect data, relabel it, retrain the model and revalidate it.
- Model performance: Missing values, mislabeled records or unrepresentative samples produce biased, inaccurate or brittle models.
- Trustworthy AI: Incomplete data can underrepresent certain groups. This leads to unfair outcomes, compliance violations and reputational damage.
- Go/No-Go decisions: CPMAI is iterative. Early validation tells the team whether the data can support the AI pattern chosen in Business Understanding. If it cannot, the team iterates back rather than pressing forward.
- Scope and schedule control: Early data quality insight lets the project manager adjust timelines, budgets and scope realistically.

What It Is
Completeness asks: Do we have all the data we need?
- All required fields or attributes are populated, with no unexplained nulls.
- All required records are present, with no missing time periods, regions or categories.
- There is enough volume for the chosen algorithm and the AI pattern.
- The data covers the populations, edge cases and scenarios the model will see in production.
- Labels are present for supervised learning.

Accuracy asks: Does the data correctly reflect reality?
- Values are correct and within valid ranges.
- Labels and annotations are correct.
- Units, formats and encodings are consistent.
- There are no duplicates, transcription errors or sensor faults.
- Data matches authoritative sources or ground truth.

These sit alongside other data quality dimensions that CPMAI candidates should know: consistency, timeliness, validity, uniqueness and relevance. Completeness and accuracy are the two most frequently tested during collection.

How It Works
1. Define requirements first. Before collection, document the data requirements: needed attributes, volume, sources, formats, labeling standards and acceptable quality thresholds. For example, less than 2% missing values, or label agreement of at least 95%. These come from the business problem and the AI pattern identified in Phase I.
2. Establish acceptance criteria and metrics. Set measurable rules such as completeness ratio, error rate, inter-annotator agreement, schema conformance and range checks.
3. Validate at the point of capture. Use input validation, required fields, schema checks, dropdowns instead of free text, and automated ingestion rules that reject or flag bad records.
4. Profile data continuously. Run data profiling on incoming batches. Check distributions, null counts, outliers, duplicates and cardinality. Compare new batches against expected patterns.
5. Sample and spot-check. Have subject matter experts manually review samples, especially labeled data. Measure inter-rater reliability among annotators.
6. Cross-reference sources. Reconcile against systems of record, control totals or ground truth to confirm accuracy.
7. Check representativeness. Confirm the collected data reflects the real-world population and conditions, to avoid sampling bias and coverage gaps.
8. Log, track and remediate. Record issues in a data quality log. Find root causes such as faulty sensors, poor labeling guidelines or broken integrations. Fix them at the source rather than only patching downstream.
9. Document lineage and provenance. Keep metadata on where data came from, when and how. This supports audits, governance and reproducibility.
10. Feed back into the iteration. If thresholds are not met, decide whether to collect more data, augment it, relabel it, change sources or revisit the business objectives. This is the CPMAI iterative loop.

Key Roles
- Data engineers: build the validation pipelines.
- Data scientists: profile the data and assess whether it is fit for purpose.
- Subject matter experts: verify accuracy and labels.
- Data owners and stewards: provide governance.
- The project manager: ensures validation is planned, resourced, tracked and tied to go/no-go decisions.

Common Pitfalls
- Assuming data from internal systems is clean.
- Deferring all quality checks to the data preparation phase.
- Focusing on volume instead of quality and representativeness.
- Having no labeling guidelines, which leads to inconsistent annotations.
- Fixing symptoms downstream instead of correcting the collection process.
- Ignoring data drift between collection periods.

Exam Tips: Answering Questions on Validating Completeness and Accuracy During Collection
- Choose early and proactive. When asked when to validate, the best answer is usually during or immediately upon collection, not after modeling or deployment.
- Data before algorithms. If a scenario describes poor model performance, suspect data quality first. Answers that tune the algorithm or try a more complex model before checking data are usually distractors.
- Fix at the source. Prefer answers that correct the root cause in the collection process over repeated downstream cleansing.
- Defined, measurable criteria. Correct answers often involve setting quality thresholds or acceptance criteria before or during collection.
- Know the definitions. Missing records or fields point to completeness. Wrong values or labels point to accuracy. Underrepresented groups point to completeness and representativeness, and to bias.
- Iterate, do not force. If data cannot meet requirements, CPMAI favors going back to Business or Data Understanding to reassess feasibility. Proceeding to modeling anyway is the wrong choice.
- Involve SMEs for labels. For questions about labeled data accuracy, look for answers mentioning expert review, labeling guidelines and inter-annotator agreement.
- Watch for absolute words. Options claiming data must be 100% perfect, or that validation can be skipped for trusted internal sources, are typically wrong. Aim for fit-for-purpose quality.
- Think trustworthy AI. Link incomplete or unrepresentative data to fairness, bias and compliance risk when the scenario mentions demographics or regulated domains.
- Project manager perspective. The PM is not expected to write validation scripts. The PM ensures validation is planned, resourced, tracked, documented and used for go/no-go decisions.

Sample Question
A team has collected six months of customer transaction data. Profiling reveals 18% of records lack a region code and two months are missing entirely. What should the project manager do first?
A) Begin model training and impute missing values later
B) Assess the impact against data requirements and work with data owners to address the gaps at the source
C) Switch to a deep learning model that handles missing data
D) Remove all incomplete records and proceed

Answer: B. It validates completeness against the defined requirements and remediates at the source, which is consistent with CPMAI's iterative, data-first approach.

Summary
Validating completeness and accuracy during collection means checking early, measuring against defined criteria, fixing root causes and iterating when needed. On the exam, favor answers that are proactive, data-centric, measurable, SME-informed and aligned with CPMAI's iterative go/no-go philosophy.