Data Quality Standards and Acceptance Criteria

Overview
In the PMI Cognitive Project Management for AI (CPMAI) methodology, the Data Understanding phase (Phase II) is where the team works out what data an AI solution needs. A central part of that work is setting Data Quality Standards and Acceptance Criteria. These are the measurable rules that say when data is good enough to train, validate, test and run an AI model.

This guide covers why the topic matters, what it is, how it works in practice, and how to answer exam questions about it.

1. Why Data Quality Standards and Acceptance Criteria Matter
AI projects are data projects first. CPMAI stresses that most AI project failures come from data problems, not algorithm problems. The rule "garbage in, garbage out" is even stronger in machine learning, because models learn the patterns in the data, including its errors, gaps and biases.

Key reasons this topic is critical:

• It prevents wasted effort: If data quality is not defined up front, teams may spend weeks training models on data that can never support the business goal.
• It links data to business objectives: Acceptance criteria turn the business need from Phase I (Business Understanding) into concrete data requirements.
• It supports go/no-go decisions: CPMAI is iterative and uses decision points. Clear criteria let the team decide objectively whether to move to Data Preparation, look for more data, or rescope the project.
• It reduces risk: Poor quality leads to biased, inaccurate or unsafe models, which can cause reputational, legal and ethical harm.
• It aligns stakeholders: Agreed standards give data owners, data scientists, project managers and business sponsors a shared definition of "fit for purpose" data.
• It supports trustworthy AI: Standards for representativeness and bias help ensure the solution is fair, explainable and compliant.

2. What Data Quality Standards and Acceptance Criteria Are
The two terms are related but distinct:

• Data Quality Standards are the general expectations for data quality, organized into dimensions such as accuracy and completeness. They describe what good looks like.
• Acceptance Criteria are the specific, measurable thresholds data must meet before the project accepts it for use. They describe how we will know it is good enough. Examples:
  • No more than 2% missing values in critical fields.
  • Labels verified with 95% inter-annotator agreement.
  • Data covers at least 24 months of transactions.

Common Data Quality Dimensions

• Accuracy: Data correctly reflects real-world values.
• Completeness: Required fields and records are present, with acceptable levels of missing data.
• Consistency: Data is uniform across sources and formats, with no conflicting values.
• Timeliness / Currency: Data is recent enough to reflect current conditions.
• Validity: Data follows the defined formats, types, ranges and business rules.
• Uniqueness: There are no unwanted duplicate records.
• Integrity: Relationships between data elements are maintained.
• Relevance: Data actually relates to the problem being solved.
• Representativeness: Data reflects the real population and the conditions the model will face in production, including edge cases and minority classes.
• Label Quality: For supervised learning, labels are correct, consistent and sufficiently numerous.
• Volume / Sufficiency: There is enough data to train, validate and test the model reliably.
• Accessibility and Compliance: Data can be legally and ethically used and is available when needed. This covers privacy, consent, licensing and security.

3. How It Works in the CPMAI Process
Step 1: Start from the business problem. Review the Phase I outputs: the AI pattern (e.g., recognition, predictive analytics, conversational, anomaly detection), the success metrics and the expected ROI. Quality needs depend on the use case. A medical diagnosis model needs stricter accuracy than a movie recommendation engine.

Step 2: Identify data sources and owners. Catalog internal and external sources. Engage data stewards and subject matter experts (SMEs) who understand the data's meaning and limitations.

Step 3: Define quality dimensions relevant to the use case. Not every dimension carries equal weight. Choose the ones that most affect model performance and business risk.

Step 4: Set measurable acceptance thresholds. Make each criterion specific, measurable, achievable, relevant and time-bound. Avoid vague statements like "data should be clean."

Step 5: Document the criteria. Record them in a data requirements document, data quality plan or data dictionary. Get stakeholder sign-off.

Step 6: Profile and assess the data. Use data profiling to measure actual quality against the criteria. Typical activities include:

• Statistical summaries
• Missing value analysis
• Outlier detection
• Distribution and class balance checks
• Bias assessment

Step 7: Make a go/no-go decision.

• If the criteria are met, proceed to Data Preparation (Phase III).
• If not, choose among:
  • Remediating the data (cleansing, augmentation, additional labeling)
  • Acquiring new data
  • Adjusting scope
  • Iterating back to Business Understanding

Step 8: Monitor continuously. Quality is not a one-time check. Data drift and changes in source systems can degrade quality after deployment. CPMAI's iterative nature means the criteria are revisited in later iterations and during operationalization.

Typical Artifacts

• Data quality requirements and acceptance criteria checklist
• Data profiling reports
• Data quality scorecards or dashboards
• Data lineage and provenance records
• Risk and issue logs for data gaps
• Go/no-go decision records

Key Roles

• Project Manager: facilitates criteria definition, ensures sign-off and manages decisions.
• Data Scientists / ML Engineers: define technical thresholds needed for model performance.
• Data Engineers: assess feasibility, pipelines and data integrity.
• Data Owners / Stewards: provide context, governance and access rights.
• Business SMEs: confirm relevance and correct interpretation, and often validate labels.
• Compliance / Legal / Ethics: confirm privacy, consent and fairness requirements.

4. Common Pitfalls

• Starting model development before defining or assessing data quality.
• Using vague, non-measurable criteria.
• Ignoring representativeness and bias.
• Assuming that a large volume of data means good quality data.
• Leaving business SMEs out of label validation.
• Treating quality as a one-time activity instead of monitoring it.
• Overlooking legal and ethical restrictions on data use.
• Failing to tie criteria back to the business success metrics.

5. Exam Tips: Answering Questions on Data Quality Standards and Acceptance Criteria
Tip 1: Data first, model later. If a question describes a team eager to start modeling, the correct answer is usually to first define and verify data quality and acceptance criteria. CPMAI emphasizes that data understanding comes before modeling.

Tip 2: Prefer measurable criteria. When options contrast vague goals ("ensure high-quality data") with specific thresholds ("less than 3% missing values in key fields"), choose the measurable one.

Tip 3: Tie criteria to the business objective. The best answer usually connects quality thresholds to the use case and its success metrics. Quality is about being fit for purpose, not perfect.

Tip 4: Recognize go/no-go logic. If data fails to meet the criteria, the correct response is rarely "proceed anyway." Look for these answers instead:

• Remediate the data
• Acquire more data
• Revisit the scope
• Iterate back to Business Understanding

CPMAI is iterative, so going back a phase is acceptable and often correct.

Tip 5: Watch for bias and representativeness. Questions about poor model performance on certain groups or real-world conditions often point to training data that was not representative. Choose answers that address data coverage and fairness.

Tip 6: Involve the right stakeholders. Correct answers often involve collaborating with data owners, SMEs and compliance teams. Avoid answers where one role decides alone.

Tip 7: Volume does not equal quality. Be cautious of answers suggesting that more data automatically fixes problems. More bad data is still bad data.

Tip 8: Remember label quality for supervised learning. When labels are inconsistent, the fix usually involves clear labeling guidelines, SME review and agreement metrics.

Tip 9: Quality is continuous. For questions about deployed models degrading over time, think data drift. The answer involves ongoing monitoring against quality standards and possibly retraining.

Tip 10: Do not forget compliance. Data can be accurate and complete yet still unacceptable if it violates privacy, consent or licensing. Acceptance criteria include legal and ethical usability.

Tip 11: Know the vocabulary. Be ready to match each term to its meaning:

• Accuracy, completeness, consistency, timeliness, validity, uniqueness
• Representativeness, data profiling
• Data lineage, data provenance, data stewardship

Tip 12: Eliminate extreme answers. Options with words like "always," "never," or "skip," or that demand perfect data, are usually wrong. CPMAI favors pragmatic, risk-based and iterative approaches.

6. Sample Exam-Style Question
A team is building a predictive maintenance model. During Data Understanding, profiling shows that sensor data is missing for 30% of machines, while the agreed acceptance criterion allows no more than 5% missing data. What should the project manager do next?

A) Proceed to modeling and let the algorithm handle the missing values
B) Lower the acceptance criterion to 30% so the project can stay on schedule
C) Document the gap, assess remediation or additional data sources with stakeholders, and make a go/no-go decision
D) Cancel the project immediately

Correct answer: C. It follows CPMAI's evidence-based, iterative decision-making. A ignores the criteria. B changes standards arbitrarily without regard to the business need. D is too extreme before options have been explored.

7. Summary
Data quality standards and acceptance criteria define when data is fit for an AI project's purpose. They are set early in the Data Understanding phase, derived from business objectives, expressed as measurable thresholds, agreed with stakeholders, checked through profiling, and used for go/no-go decisions. They are then monitored throughout the AI lifecycle.

On the exam, favor answers that are:

• Data-first: verify data before modeling.
• Measurable: use specific thresholds, not vague goals.
• Business-aligned: tie quality to the use case.
• Collaborative: involve the right stakeholders.
• Ethical and compliant: respect privacy, consent and fairness.
• Iterative: remediate or go back a phase when criteria are not met.