Mapping Internal Databases and Data Warehouses

Overview
Mapping internal databases and data warehouses is a core activity in the Identify Data Needs part of the PMI Cognitive Project Management for AI (CPMAI) methodology. It belongs to CPMAI Phase II, Data Understanding, which follows Phase I, Business Understanding. In this activity, the AI project team builds a clear inventory of the data the organization already holds. That inventory covers transactional databases, data warehouses, data lakes, data marts, operational systems, and departmental data stores. The team then judges how each source relates to the AI problem being solved.

Why It Is Important
CPMAI is a data-centric methodology. Its core principle is that AI projects are data projects. Most AI project failures come from data problems, not algorithm problems. Mapping internal data matters for several reasons:
- Feasibility check: It answers the question Do we have the data we need to solve this problem? This is part of the go/no-go decision. If the right data does not exist or cannot be reached, the project may need to be re-scoped, paused, or stopped.
- Cost and time savings: Internal data is usually cheaper and faster to obtain than external data. Knowing what you have avoids buying or collecting data you already own.
- Avoiding surprises later: Finding data quality, access, or compliance problems early prevents rework in Data Preparation (Phase III) and Model Development (Phase IV).
- Governance and compliance: Mapping shows where sensitive data such as PII, PHI, or financial records lives and who owns it. This supports privacy, security, and responsible AI requirements.
- Breaking down silos: Data is often scattered across departments. Mapping shows overlaps, gaps, and integration needs.
- Scoping the AI pattern: The data available affects which of the seven AI patterns is realistic. The patterns are recognition, conversational, predictive analytics, patterns and anomalies, hyperpersonalization, autonomous systems, and goal-driven systems.

What It Is
Mapping internal databases and data warehouses means systematically identifying, cataloging, and assessing the organization's internal data assets against the data requirements of the AI solution. Key concepts:
- Operational/transactional databases (OLTP): Systems such as ERP, CRM, HR, and point-of-sale that record day-to-day transactions. Their data is current but often normalized and spread across many tables.
- Data warehouses (OLAP): Central repositories of cleaned, integrated, historical, and structured data used for reporting and analytics. They are often a good starting point for AI training data, but they may be aggregated or missing granular detail.
- Data marts: Subject-specific subsets of a warehouse, such as a sales mart or a finance mart.
- Data lakes: Stores of raw structured, semi-structured, and unstructured data such as logs, images, documents, and audio. These are often valuable for AI.
- Metadata and data catalogs: Information about the data itself, including schemas, definitions, lineage, owners, refresh frequency, and quality indicators.
- Data owners and stewards: The people responsible for each data asset. Their approval is often needed for access.

How It Works (Typical Steps)
1. Start from business and AI requirements: Use the Phase I outputs (problem statement, AI pattern, success criteria) to define what data is needed. This includes features, labels or targets, volume, time span, and granularity.
2. Inventory data sources: Identify every internal system that may hold relevant data. Interview IT, data engineering, business analysts, and data stewards. Review existing data catalogs and architecture diagrams.
3. Document each source: For each one, record:
  - Location and system
  - Owner
  - Data types (structured or unstructured)
  - Schema and fields
  - Volume and history
  - Update frequency
  - Format
  - Access method (API, SQL, export)
  - Security classification
  - Known quality issues
4. Map requirements to sources: Build a mapping matrix that links each required data element to the system or systems where it exists. Note any transformations or joins needed.
5. Assess the data: Evaluate it using the data quality dimensions and the V's of big data:
  - Volume
  - Variety
  - Velocity
  - Veracity
  - Value
  - Completeness, accuracy, consistency, timeliness, and bias
6. Identify gaps: Note missing data, missing labels, insufficient history, or poor quality. Decide whether to collect new data, acquire external data, generate synthetic data, label data, or re-scope the project.
7. Check access, privacy, and compliance: Confirm the legal and ethical right to use the data for AI purposes. Relevant considerations include consent, GDPR/HIPAA, and contractual limits. Secure approvals from data owners.
8. Feed the go/no-go decision: Share the findings with stakeholders so they can decide whether to proceed to Data Preparation, iterate, or return to Business Understanding.

Common Challenges
- Data silos and inconsistent definitions across departments, such as different meanings of customer.
- Legacy systems with poor documentation.
- Aggregated warehouse data that lacks the transaction-level detail ML needs.
- Missing labels for supervised learning.
- Access restrictions and slow approval processes.
- Historical bias embedded in data.
- Data that technically exists but is not authorized for AI use.

Exam Tips: Answering Questions on Mapping Internal Databases and Data Warehouses
- Think data-first: When a scenario asks what to do before building a model, the answer is usually to identify and assess available data. Choosing algorithms or tools is rarely the first step.
- Know the phase order: Mapping internal data happens in Phase II, Data Understanding. It comes after Business Understanding and before Data Preparation. Eliminate answers that place cleansing, feature engineering, or model training before data sources are identified.
- Internal before external: When choosing a next step, prefer using and assessing existing internal data before buying third-party data or launching new collection. The exception is when the scenario explicitly says internal data is insufficient.
- Existence does not mean usability: Watch for traps where data exists but is low quality, unlabeled, biased, too aggregated, or not permitted for AI use. The correct answer often involves assessing quality and access, not assuming readiness.
- Involve data owners and stewards: Answers that engage stakeholders such as IT, data stewards, and compliance or legal are usually better than ones where the project manager acts alone.
- Compliance and ethics matter: If the scenario mentions personal or sensitive data, choose answers that address privacy, consent, security, and governance before use.
- Link to go/no-go: If mapping shows critical gaps, the CPMAI answer is to iterate. That means revisiting the business understanding, re-scoping, or finding alternate data, not forcing the project forward.
- Distinguish the terms: Warehouses hold structured, historical, integrated data for analytics. Lakes hold raw data of many types. OLTP databases hold current transactional data. Questions may test which source fits a need, for example unstructured images point to a data lake or document store.
- Iterative mindset: CPMAI is agile and iterative. Data mapping can be refined in later iterations, so prefer answers that support small, quick iterations over exhaustive, waterfall-style enterprise inventories.
- Watch for keywords: Terms such as data inventory, data catalog, data sources, data availability, data access, and data gaps signal this topic.

Quick Summary
Mapping internal databases and data warehouses means finding out what data the organization already has, where it lives, who owns it, how good it is, and whether it can legally be used. In CPMAI, this step is central to deciding whether an AI project is feasible. On the exam, favor answers that are data-centric, stakeholder-inclusive, compliance-aware, and iterative.