Working with Data Scientists on Model Architecture

Overview
In the PMI Cognitive Project Management for AI (CPMAI) methodology, the Model Development phase is where business requirements, prepared data and technical expertise come together to produce a working AI model. A central part of this phase is the collaboration between the AI project manager and the data science team on model architecture. Model architecture means the choice of algorithm family, model structure, learning approach and technical design. The CPMAI-certified project manager is not expected to design neural networks. They are expected to guide, constrain, question and align architectural decisions so that the model solves the right business problem in a feasible, trustworthy and maintainable way.

Why It Is Important
1. Architecture decisions are hard to reverse. Choosing a deep learning approach when a simple regression would do adds cost, compute needs, explainability problems and time. Choosing a model that is too simple can fail to meet performance targets.
2. Most AI projects fail for business and data reasons, not algorithm reasons. When data scientists pick architectures without business context, they often optimize the wrong metric, ignore deployment constraints or build something nobody can operationalize.
3. Trustworthy AI requirements depend on architecture. Explainability, transparency, bias mitigation, privacy and auditability are much easier with some model types than others. For example, decision trees and linear models are more interpretable than deep ensembles.
4. Cost and resources. Architecture drives compute cost, training time, inference latency, licensing and the skills needed. These are all core project constraints the PM must manage.
5. Operationalization readiness. A model that cannot be deployed, monitored or retrained in the target environment delivers no value. Architecture must account for the CPMAI Operationalization phase from the start.

What It Is
Working with data scientists on model architecture is the structured, collaborative process of translating the business problem, defined in Business Understanding, and the available data, examined in Data Understanding and Data Preparation, into an appropriate modeling approach. Key elements include:
• Identifying the AI pattern. CPMAI uses the Seven Patterns of AI: Hyperpersonalization, Recognition, Conversation and Human Interaction, Predictive Analytics and Decisions, Patterns and Anomalies, Goal-Driven Systems and Autonomous Systems. The pattern narrows down suitable architectures.
• Selecting the learning approach. Options include supervised, unsupervised, semi-supervised, reinforcement learning and generative or foundation-model approaches.
• Choosing the algorithm family. Examples are regression, decision trees and random forests, gradient boosting, clustering, neural networks (CNNs, RNNs, transformers) and large language models.
• Build vs. buy vs. reuse. The team decides between custom models, pre-trained models, transfer learning, fine-tuning, third-party APIs and AutoML.
• Defining constraints and success criteria. These cover accuracy, precision and recall targets, latency, explainability, fairness, cost, scalability and the deployment environment (cloud, edge, on-premises).
• Documenting decisions. The team records model cards, rationale for choices, assumptions, trade-offs and risks.

How It Works
Step 1: Anchor to the business problem. Before any architecture discussion, the PM confirms that the data science team understands the business objective, the AI pattern, the definition of success and the go/no-go criteria. The PM keeps asking: What decision or action will this model support?

Step 2: Share constraints early. The PM communicates the non-functional requirements:
• Regulatory and explainability needs, for example in credit decisions or healthcare
• Latency and throughput limits
• Budget and compute limits
• Data volume, quality and labeling limits
• Privacy and security requirements
• The target deployment platform and the team's ability to maintain the model

Step 3: Explore candidate approaches. Data scientists propose several candidate architectures. Good practice is to start with a simple baseline model, such as logistic regression or a heuristic, and add complexity only when it is justified. The PM encourages comparison against that baseline.

Step 4: Evaluate trade-offs together. Common trade-offs include:
• Accuracy vs. interpretability
• Model complexity vs. data availability, since complex models need more data
• Performance vs. cost and latency
• Custom build vs. time-to-value with pre-trained models
• Generalization vs. overfitting
The PM makes sure these trade-offs are discussed openly and that business stakeholders are involved when they affect business outcomes.

Step 5: Iterate in small cycles. CPMAI is iterative and agile. The motto is think big, start small, iterate often. Architectures are tested quickly, evaluated and refined. If the data cannot support the chosen approach, the team returns to Data Understanding or Data Preparation. Pushing ahead with a poorly supported approach wastes effort.

Step 6: Plan for evaluation and operationalization. The architecture must allow for:
• Meaningful evaluation metrics
• Monitoring for model drift and data drift
• Retraining pipelines
• Version control
• Rollback

Step 7: Document and govern. The PM ensures the following are recorded and reviewed in line with governance and trustworthy AI policies:
• Architecture rationale
• Hyperparameter choices
• Training data lineage
• Known limitations and bias assessments

The PM's Role vs. the Data Scientist's Role
• Data scientists own the technical design, algorithm selection, feature engineering, training and tuning.
• The PM owns alignment, scope, schedule, cost, risk, stakeholder communication, constraints and decision facilitation.
• The PM asks informed questions and does not dictate algorithms. They also prevent gold-plating, meaning complexity that adds no business value.

Common Pitfalls
• Letting technical curiosity drive architecture instead of business need
• Choosing deep learning by default without enough data
• Ignoring explainability until late in the project
• Not involving MLOps or IT teams in architecture decisions
• Treating model development as a one-time waterfall step
• Failing to define acceptance criteria before training begins

Exam Tips: Answering Questions on Working with Data Scientists on Model Architecture
1. Business first. When an answer choice ties the architecture decision back to the business objective or success criteria, it is usually correct. CPMAI always starts from Business Understanding.
2. The PM facilitates; the PM does not design. Reject answers where the PM selects the algorithm alone or overrides data scientists on technical details. Prefer answers where the PM clarifies requirements, communicates constraints and facilitates trade-off decisions.
3. Simplest effective solution wins. If a simpler, more interpretable model meets the requirements, it is typically the preferred answer. This is especially true in regulated or high-stakes contexts.
4. Explainability and trustworthy AI. In scenarios involving finance, healthcare, hiring or legal decisions, choose answers that prioritize transparency, fairness and auditability. Raw accuracy alone should not decide the question.
5. Data drives feasibility. If a scenario mentions limited, poor-quality or unlabeled data, the right answer often involves one of the following:
• Returning to the data phases
• Using transfer learning or pre-trained models
• Choosing a less data-hungry approach
Forcing a complex architecture is rarely the answer.
6. Iterate, don't over-plan. Answers that suggest a quick proof of concept, a baseline model or a small iteration are generally better than answers proposing a long, exhaustive design phase.
7. Go back when needed. CPMAI allows and encourages moving back to earlier phases. If model results reveal a problem with the business framing or the data, the correct action is often to revisit that phase.
8. Consider operationalization early. Prefer answers that account for deployment environment, latency, monitoring and retraining during architecture selection.
9. Build vs. buy. When time-to-value matters and a suitable pre-trained model or service exists, reuse is often the better answer. Check that it meets data privacy, cost and customization needs.
10. Stakeholder involvement. When a trade-off affects business outcomes, such as accuracy vs. speed or false positives vs. false negatives, the correct answer involves bringing business stakeholders into the decision.
11. Watch for keywords. Words like first, best, most appropriate and next matter. The first step is usually to clarify requirements or constraints. Building or tuning comes later.
12. Eliminate extremes. Answers using words like always use deep learning, skip evaluation or let the data scientists decide everything are usually wrong.

Quick Example
Scenario: A bank's data science team proposes a deep neural network for loan approval because it scored 2% higher accuracy than a gradient boosted tree. Regulators require applicants to receive reasons for denial. What should the PM do?
Best answer: Facilitate a trade-off discussion with stakeholders and compliance. Then favor the more interpretable model, or an approach with robust explainability, because the regulatory requirement for explainable decisions outweighs a small accuracy gain.

Summary
Working with data scientists on model architecture means bridging business needs and technical design. The CPMAI project manager ensures that architecture choices are:
• Driven by the business problem and the AI pattern
• Feasible given the data
• Constrained by cost, latency and trustworthy AI requirements
• Developed iteratively
• Ready for operationalization
On the exam, choose answers that reflect collaboration, business alignment, simplicity, explainability and iteration.