Data Volume Requirements and Sampling Strategies

Overview
In the PMI Cognitive Project Management for AI (PMI-CPMAI) methodology, which is based on the CPMAI framework, Phase II: Data Understanding asks the team to confirm that the right data exists, in the right form and in the right quantity, to meet the AI objective. Data Volume Requirements and Sampling Strategies sit inside the task of Identifying Data Needs. They answer two linked questions:
1. How much data do we need?
2. How do we select a subset of data that faithfully represents the real-world problem?

Why It Is Important
AI projects fail far more often because of data problems than because of algorithm problems. CPMAI stresses that AI projects are data projects first. Getting volume and sampling wrong causes serious harm:
- Too little data leads to underfitting, high variance and overfitting to noise, unreliable validation and models that do not generalize.
- Too much unmanaged data inflates cost, labeling effort, storage and compute, and slows iteration. It can also delay the project without improving accuracy.
- Biased or unrepresentative samples produce models that perform well in testing but fail, or discriminate, in production. This creates ethical, legal and reputational risk.
- Underrepresented rare classes (fraud, defects, rare diseases) cause the model to ignore exactly the cases the business cares about most.
- Go/No-Go decisions depend on this analysis. If sufficient, representative data cannot be obtained, CPMAI expects the project manager to iterate back to Business Understanding, re-scope the problem, or stop.

What It Is
Data volume requirements are the estimated quantity of data (records, examples, labeled instances, time span, features) needed to train, validate and test a model so that it reaches the target performance defined in Business Understanding.

Sampling strategies are the methods used to select a subset of data from a population, or to rebalance a dataset, so that the training, validation and test sets are representative, unbiased and fit for the AI pattern being used.

Key Factors That Drive Data Volume
- Problem complexity and AI pattern: Recognition (image, speech) and conversational patterns usually need far more data than simple predictive analytics on tabular data.
- Algorithm type: Deep learning needs large volumes. Classical ML (decision trees, logistic regression) can work with smaller sets. Transfer learning and pre-trained or foundation models greatly reduce the need.
- Number of features (dimensionality): More features generally require more examples, because of the curse of dimensionality.
- Number of classes and class balance: Every class, especially rare ones, needs enough examples.
- Required accuracy and confidence: Higher performance targets need more data.
- Data variability and noise: Noisy or highly variable data needs more examples.
- Supervised vs. unsupervised learning: Supervised learning needs labeled data, which is often the real bottleneck.
- Temporal coverage: Seasonality and trends require data spanning enough cycles, for example several years for annual seasonality.
- Rules of thumb: One example is roughly 10 times as many examples as features. Treat these only as starting points. Learning curves, which plot performance against training size, give empirical evidence of whether more data will help.

Common Sampling Strategies
Probability sampling (preferred for representativeness):
- Simple random sampling: Every record has an equal chance of selection. Good when the population is homogeneous.
- Stratified sampling: The population is divided into strata (class, region, demographic) and each is sampled proportionally. This preserves class distribution and is ideal for imbalanced classification and for train/test splits.
- Systematic sampling: Every k-th record is selected. Beware of hidden periodic patterns.
- Cluster sampling: Whole groups (stores, hospitals) are selected. Cost-efficient, but carries a risk of within-cluster similarity.

Non-probability sampling (use with caution):
- Convenience, judgmental or quota sampling. These are fast but prone to selection bias.

Handling imbalance:
- Oversampling the minority class, including synthetic methods such as SMOTE.
- Undersampling the majority class.
- Class weighting and data augmentation, for example rotating or flipping images.

Data splitting and validation:
- Train/validation/test splits, for example 70/15/15 or 80/10/10.
- K-fold cross-validation for smaller datasets.
- Time-based splits for time series. Never randomly shuffle future data into training, because that causes data leakage.
- Holdout data must stay untouched until final evaluation.

Augmenting volume when data is insufficient:
- Acquire third-party data or use public datasets.
- Generate synthetic data.
- Use transfer learning or pre-trained models.
- Apply active learning to label the most informative samples.
- Narrow the project scope (CPMAI favors small, iterative wins).

How It Works in the CPMAI Process
1. Start from Business Understanding: Confirm the AI pattern, success criteria and target performance metrics.
2. Inventory data sources: Identify internal and external sources, ownership, access, cost and labeling status.
3. Estimate volume: Consider the pattern, algorithm, features, classes, accuracy target and temporal coverage. Consult data scientists and run small learning-curve experiments.
4. Assess representativeness: Compare the sample with the production population across key dimensions (demographics, geography, time, edge cases).
5. Select a sampling strategy: Use stratified sampling for imbalance, time-based splits for temporal data, and augmentation for gaps.
6. Assess quality alongside quantity: Volume without quality is useless. Check completeness, accuracy, consistency and timeliness.
7. Document and decide: Record assumptions, gaps, bias risks and costs. Then make a Go/No-Go call or iterate back to earlier phases.
8. Iterate: Revisit during Data Preparation, Model Development and Model Evaluation, and later monitor for data drift once the model is in operation.

Worked Example
A bank wants to detect fraudulent transactions, and fraud makes up 0.2% of records. Taking a random 10,000-record sample yields only about 20 fraud cases, which is far too few. The correct approach has four parts:
- Use the full historical population or a large time window.
- Apply stratified sampling so train and test sets keep the fraud ratio.
- Use oversampling or class weighting on the training set only.
- Evaluate with precision, recall or F1 rather than accuracy.

Exam Tips: Answering Questions on Data Volume Requirements and Sampling Strategies
1. Data first, algorithm second. When a scenario describes poor model performance, look for a data quantity, quality or representativeness cause before choosing "try a different algorithm."
2. Representative beats big. If an option offers more data that is biased and another offers less data that is representative, prefer representative data. Quality and representativeness trump raw volume.
3. Imbalanced classes mean stratified sampling, resampling or weighting. Watch for words such as rare, fraud, defect, minority, or 1% of cases.
4. Time series means time-based splits. Random shuffling of temporal data is a trap answer because it causes leakage.
5. Never resample or augment the test set. Apply oversampling and augmentation only to training data. Keep the holdout set pristine.
6. Insufficient data has CPMAI-aligned responses. These include acquiring or augmenting data, using transfer learning, reducing scope, or iterating back to Business Understanding. Be cautious of answers that proceed to modeling anyway.
7. Know the Go/No-Go logic. If data cannot meet requirements, the right PM action is often to pause, re-scope or stop. Do not push forward to meet the schedule.
8. Labeled data is the true constraint. For supervised learning, volume means labeled volume. Consider labeling cost, effort and quality.
9. Deep learning needs more data. If the scenario has small data, favor simpler models or pre-trained or transfer approaches.
10. Bias and ethics matter. Samples missing demographic groups or regions point to fairness risk. Choose answers that broaden coverage and document the limitation.
11. Use learning curves as evidence. If performance plateaus as data grows, more data will not help. Look at features, quality or the model instead.
12. Pick the BEST answer, not just a correct one. PMI questions often include several plausible actions. Choose the one that addresses root cause, follows the CPMAI phase order and involves the right stakeholders (data owners, data scientists, SMEs).
13. Watch phase placement. Estimating volume and assessing sources happens in Data Understanding. Actual cleaning, resampling and splitting execution happens in Data Preparation.
14. Do not trust accuracy alone on imbalanced data. Answers recommending precision, recall, F1 or AUC are usually stronger.

Quick Recap
Determine how much data the AI pattern and performance target require. Make sure the data, especially labeled data, is available and representative. Choose a sampling and splitting strategy that preserves real-world distributions and avoids leakage. Handle imbalance correctly. If the data falls short, augment it, re-scope the project, or make a disciplined Go/No-Go decision.