Data Extraction from Identified Sources

Overview
In the PMI Cognitive Project Management for AI (CPMAI) methodology, the Identify Data Needs phase (Phase II, Data Understanding) comes after Business Understanding. It sits before Data Preparation, Model Development, Model Evaluation and Model Operationalization.

Within this phase, Data Extraction from Identified Sources is the step where the team actually obtains data from the sources it has already located and qualified. It turns a list of potential data sources into usable datasets that can be inspected, profiled and later prepared for modeling. CPMAI stresses that AI projects are data projects first. If you cannot reliably extract the right data, no algorithm will save the project.

Why It Is Important
1. Data is the foundation of AI. Most AI project failures come from data problems, not algorithm problems. Extraction is where those problems first become real and visible.
2. It validates feasibility. A source may look perfect on paper but turn out to be unreachable, locked behind licensing, too slow to query, or legally restricted. Extraction confirms whether the data truly exists in the quantity, quality and format needed.
3. It supports go/no-go decisions. CPMAI is iterative. Extraction results may show the AI solution is not feasible with current data, prompting a return to Business Understanding to rescope the problem.
4. It manages risk and compliance. Extracting data touches privacy (GDPR, HIPAA, CCPA), security, intellectual property and contractual obligations. Doing it correctly prevents legal and ethical exposure.
5. It sets up the downstream phases. Poorly documented or inconsistent extraction creates rework in Data Preparation and leads to unreliable models. It also makes retraining in production impossible to reproduce.
6. It enables trustworthy AI. Data lineage and provenance start at extraction. Without them, auditability, explainability and bias analysis suffer.

What It Is
Data extraction is the controlled process of retrieving data from identified internal and external sources and bringing it into a project environment for analysis. Typical sources include:
- Internal: transactional databases, ERP/CRM systems, data warehouses, data lakes, log files, documents, emails, IoT/sensor streams, images, audio, video.
- External: public/open datasets, government data, third-party data vendors, partner data, APIs, web data (subject to terms of use), and synthetic data generators.

Data can be:
- Structured (tables, relational databases)
- Semi-structured (JSON, XML, logs)
- Unstructured (text, images, audio, video)

Each data type needs different extraction approaches and tooling.

How It Works: Key Steps
1. Confirm the source inventory. Revisit the data sources identified earlier and their mapping to the business problem and AI pattern. Examples of patterns include recognition, conversational, predictive analytics, patterns and anomalies, hyperpersonalization, goal-driven systems and autonomous systems. Ensure each source answers a defined data need.

2. Secure access and permissions. Identify data owners and stewards. Obtain approvals, credentials, data sharing agreements, licenses and contracts. Confirm consent and legal basis for use, especially for personal or sensitive data.

3. Assess governance, privacy and security requirements. Determine classification levels. Decide whether anonymization, pseudonymization or masking is needed at the point of extraction. Check data residency rules and retention policies. Involve legal, compliance and security teams early.

4. Select the extraction method. Common options are:
- Database queries (SQL)
- API calls
- Batch file exports (CSV, Parquet)
- ETL/ELT pipelines
- Streaming ingestion for real-time data
- Web scraping (only where legally permitted)
- Manual collection or new data capture when data does not yet exist

Choose based on volume, velocity, variety, frequency and whether the extraction must be repeatable for production retraining.

5. Extract a representative sample first. CPMAI favors small iterations. Pull a sample to verify format, completeness, and alignment with the data need before committing to full-scale extraction.

6. Perform the extraction and capture metadata. Record:
- Source and extraction date/time
- Query logic and filters applied
- Schema and versions
- Record counts

This establishes data lineage and provenance.

7. Validate the extracted data. Compare record counts to the source. Check for truncation, encoding errors and missing fields. Ensure timeframes and populations match what was requested. This is not full cleansing, which belongs to Data Preparation. It is a sanity check that the extraction worked.

8. Store securely in the project environment. Place data in a controlled repository such as a sandbox, data lake zone or feature store, with appropriate access controls and version control.

9. Evaluate sufficiency and quality. Assess the four Vs:
- Volume: is there enough data for training, validation and testing?
- Variety: what formats and types are present?
- Velocity: how fresh is the data, and how fast does it arrive?
- Veracity: is the data accurate and trustworthy?

Also check representativeness and potential bias, and whether labels exist for supervised learning. Identify gaps that require additional sources, labeling, data augmentation or synthetic data.

10. Document and communicate. Update the data inventory and risk register. Report findings to stakeholders. Feed decisions back into the go/no-go assessment.

Common Challenges
- Data silos: data spread across departments with different owners.
- Legacy systems: poor or no APIs.
- Inconsistent formats and undocumented schemas.
- Privacy restrictions and missing consent.
- Third-party licensing limits on use for AI training.
- Insufficient historical data or unlabeled data.
- Performance impact on production systems during extraction.
- Data drift: extracted historical data may not reflect current reality.

Key Roles
- Data owners and data stewards: grant access and explain meaning.
- Data engineers: build pipelines.
- Data scientists: confirm fitness for the model.
- Legal, compliance and privacy officers: approve usage.
- Security: controls handling.
- Project manager: coordinates, tracks risks and ensures alignment with business objectives.

How to Answer Exam Questions on This Topic
CPMAI exam questions are usually scenario-based. They describe a project situation and ask for the best, first or most appropriate action. Use this approach:
1. Locate the phase. Determine where the scenario sits in the CPMAI lifecycle. If the team is retrieving or obtaining data from known sources, it is Data Understanding/Identify Data Needs, not Data Preparation (cleansing, transforming, labeling, feature engineering) and not Model Development.
2. Look for the governance angle. Answers that secure permissions, confirm legal rights or protect sensitive data before pulling data are usually preferred over answers that rush to extract.
3. Prefer iterative, sample-first actions. CPMAI values small, quick iterations. Validating a sample before full extraction is often the best answer.
4. Tie back to the business problem. The right answer keeps extraction aligned with the defined data needs, not grabbing everything available.
5. Recognize feasibility signals. If extraction reveals that data is unavailable, insufficient or unusable, the correct response is often to reassess feasibility or return to Business Understanding. Pushing ahead with modeling is usually wrong.
6. Value documentation and lineage. Capturing metadata and provenance supports reproducibility, auditability and operationalization.

Exam Tips: Answering Questions on Data Extraction from Identified Sources
- Tip 1: When you see personal, health or financial data in a scenario, the first step is almost always to confirm legal authority, consent, and privacy/security controls before extracting.
- Tip 2: Distinguish extraction (getting data) from preparation (cleaning, transforming, labeling, augmenting). Distractor answers often place cleansing tasks in the extraction step.
- Tip 3: Answers involving collaboration with data owners or stewards are strong. They know the source systems, definitions and access processes.
- Tip 4: Be wary of options that extract all available data without filtering for relevance. CPMAI emphasizes data that is needed for the specific AI pattern and business objective.
- Tip 5: For third-party or external data, check licensing and terms of use. Data purchased for analytics may not be licensed for AI model training.
- Tip 6: Web scraping is not automatically acceptable. Look for answers that verify terms of service and legal permissibility.
- Tip 7: If extraction would burden production systems, prefer answers using replicas, off-peak windows, or data warehouse copies.
- Tip 8: Choose repeatable, automated pipelines over one-off manual pulls when the model will need retraining in production. Operationalization begins with repeatable extraction.
- Tip 9: If extracted data is insufficient (too little, unlabeled, biased or unrepresentative), valid next steps include:
  - finding additional sources
  - generating synthetic data
  - planning labeling
  - revisiting scope through a go/no-go decision
- Tip 10: Remember the CPMAI mantra that AI projects are data-centric. When in doubt, pick the answer that improves data understanding, quality assurance and governance over the one that jumps to algorithms.
- Tip 11: Watch for keywords such as first, best, most important and next. A sequencing question usually rewards access and compliance first, then sample extraction, then validation, then full extraction.
- Tip 12: Do not confuse data availability with data accessibility. Data may exist in the organization but be inaccessible due to policy, technology or ownership. Exam questions often test this distinction.

Sample Question
An AI team has identified customer support transcripts as a key data source for a sentiment model. What should the project manager do FIRST?
A) Begin labeling transcripts for sentiment.
B) Extract all transcripts into the data lake.
C) Confirm data ownership, consent and privacy requirements, and obtain access approval.
D) Select the NLP algorithm.

Answer: C. Access and compliance precede extraction (B). Labeling (A) belongs to Data Preparation, and algorithm selection (D) belongs to Model Development.

Summary
Data Extraction from Identified Sources converts identified data sources into accessible, documented, compliant datasets. It verifies that data truly supports the AI objective. Done well, it reduces project risk, confirms feasibility and enables smooth preparation and operationalization. On the exam, favor answers that are governance-first, iterative, business-aligned and well documented.