Pre- and Post-Implementation Reviews

1. Why Pre- and Post-Implementation Reviews Matter

Organizations constantly launch projects: new ERP systems, process redesigns, mergers, regulatory programs, outsourcing arrangements and new products. Every one of these initiatives changes the organization's risk profile. A mature risk management process does not simply assess risk once a year; it embeds risk thinking into change itself. Pre- and post-implementation reviews are the mechanism by which an organization (and its internal audit activity) confirms that risks created by change are identified early, that controls are designed in rather than bolted on afterwards, and that the promised benefits of the change are actually delivered.

For the CRMA candidate, this topic sits inside the domain Evaluating Risk Management Processes. The examiner wants to know that you understand how the internal audit activity provides assurance (and sometimes advice) on the way management governs change, and how to do so without compromising independence and objectivity. Questions frequently test the timing of the review, the objective of the review, the role of internal audit, and the evidence that would best support a conclusion.

The core logic is simple: it is far cheaper to prevent a control weakness at the design stage than to detect and correct it after go-live. Studies of system development consistently show that a defect caught in requirements costs a fraction of the same defect caught in production. Internal auditors who only arrive after implementation are, in effect, conducting an expensive autopsy. Internal auditors who are involved before implementation are performing preventive medicine.

2. What Is a Pre-Implementation Review?

A pre-implementation review is an evaluation performed before a new system, process, product, or organizational change goes live. It may occur at one or several points in the project life cycle: at business case approval, at design sign-off, at the end of testing, and immediately before cut-over (often called a go-live readiness review).

Typical objectives include assessing whether:
- A sound business case exists, with clearly stated objectives, expected benefits, costs, and risks that are aligned to organizational strategy and risk appetite.
- Project governance is adequate: a steering committee, a sponsor, defined roles, a project charter, and a change-control process.
- A risk assessment for the project has been performed and is being maintained in a risk register with owners and responses.
- Requirements have been documented and approved by the business, and that control, security, privacy, regulatory and audit-trail requirements are included.
- Controls are designed into the process or system, including segregation of duties, input validation, access controls, interfaces and reconciliations, and data conversion controls.
- Testing (unit, integration, user acceptance) has been planned, executed and signed off, with defects tracked to closure.
- Data migration is complete, accurate and reconciled.
- Training, documentation, and change management for affected staff are adequate.
- Contingency, rollback and business continuity arrangements exist if go-live fails.
- The organization is genuinely ready for cut-over and the decision to proceed has been made by the appropriate authority.

The deliverable is usually a readiness opinion or a set of observations that management can act on before the change is irreversible. Because the auditor is commenting on something that does not yet exist in operation, the work is design-focused rather than operating-effectiveness focused.

3. What Is a Post-Implementation Review?

A post-implementation review (often abbreviated PIR) is an evaluation performed after the change has gone live, typically once the system or process has stabilized (commonly three to twelve months after cut-over, depending on the cycle length of the process). It looks backward at the project and forward at the results.

Typical objectives include determining whether:
- The original objectives were achieved and the expected benefits were realized (cost savings, revenue growth, cycle-time reductions, error reductions, compliance outcomes). This is often called benefits realization.
- The project was delivered within budget, scope and schedule, and whether variances were approved through change control.
- The controls that were designed are actually operating effectively in the live environment.
- Residual risks remain within appetite and whether new, unanticipated risks have emerged.
- Open issues, defects and workarounds are being tracked and resolved, and manual workarounds have not quietly become permanent.
- Users have adopted the system or process and are using it as intended.
- Data integrity was preserved through conversion and subsequent processing.
- Lessons learned have been captured and fed back into the organization's project methodology, risk management process and future business cases.

The post-implementation review therefore has two dimensions: a product dimension (did we get what we paid for, and is it controlled?) and a process dimension (how well did we manage the change, and what do we learn for next time?).

4. How the Reviews Work in Practice

Step 1 - Risk-based selection. Not every project warrants internal audit involvement. The chief audit executive should use the organization's project portfolio, strategic importance, size, complexity, regulatory exposure and history of project failure to decide which initiatives receive pre- and/or post-implementation reviews. This selection is itself part of risk-based audit planning.

Step 2 - Define the role and protect independence. Before any involvement, the internal audit activity must be explicit about whether it is providing assurance (independent evaluation and opinion) or consulting (advice to management). Internal auditors may advise on control design, but they must not design, build, approve or own the controls, make management decisions, or sign off as the owner of requirements. If they do, their objectivity is impaired, and they should not subsequently provide assurance on that same area (the Standards require disclosure of such impairments and generally a cooling-off before assuming assurance responsibility). A useful exam phrase: internal audit advises and evaluates; management decides and owns.

Step 3 - Plan the engagement. Identify the objectives, criteria and scope. Criteria often come from the organization's own project management methodology, system development life cycle, change-management policy, and frameworks such as COSO, COBIT or PMBOK. Map the project risk register to the review scope so the audit effort follows the significant risks.

Step 4 - Gather evidence. For a pre-implementation review, evidence includes the business case, charter, steering committee minutes, risk register, requirements documents and sign-offs, design specifications, test plans and results, defect logs, data conversion reconciliations, training records, and the go/no-go decision record. For a post-implementation review, evidence includes benefits tracking against the business case, actual versus budgeted cost and schedule, operational performance metrics, incident and help-desk logs, control testing in the live environment, user interviews and surveys, and the lessons-learned register.

Step 5 - Evaluate and conclude. Compare condition to criteria, identify root causes, and assess the risk impact. In a pre-implementation review the conclusion is often stated as readiness or as risks that must be mitigated before go-live. In a post-implementation review the conclusion addresses whether objectives were achieved, whether controls operate effectively and whether residual risk is acceptable.

Step 6 - Communicate and follow up. Report to the project sponsor, steering committee and, for significant initiatives, senior management and the board or audit committee. Timeliness is critical in pre-implementation work: a finding delivered after go-live has lost most of its value. Track management's action plans to completion.

5. The Risk Management Assurance Perspective

In the CRMA context, there is a second layer. Beyond reviewing individual projects, the internal audit activity evaluates whether the organization's risk management process itself incorporates effective pre- and post-implementation review practices. Questions to ask include:
- Does the organization require a risk assessment for every significant change, and is it actually performed?
- Are project risks integrated into the enterprise risk register, or do they live in a silo?
- Do business cases state the risk appetite considerations and the expected effect on the risk profile?
- Does management itself perform post-implementation reviews and benefits tracking, or is this left entirely to internal audit?
- Are lessons learned from completed projects used to improve the risk management framework?
- Is there a formal gate process where risk owners must confirm readiness before go-live?

Where management already performs robust reviews, internal audit can rely on and test that work rather than duplicating it. Where management does not, the absence of such reviews is itself a reportable weakness in the risk management process.

6. Key Distinctions the Exam Likes to Test

Timing: Pre-implementation happens before go-live and focuses on design and readiness. Post-implementation happens after go-live and focuses on results, operating effectiveness and lessons learned.

Primary value: Pre-implementation review primarily prevents problems and reduces the cost of control remediation. Post-implementation review primarily confirms benefits and improves future projects.

Nature of control evaluation: Before implementation you can only evaluate design adequacy. After implementation you can evaluate operating effectiveness.

Benefits realization: This is the hallmark of the post-implementation review. If a question mentions whether projected savings materialized, the answer is post-implementation.

Independence: Advising on control design during development is acceptable consulting; building the controls, approving the design as the owner, or making the go-live decision is not. Serving on a steering committee in an advisory, non-voting capacity is generally acceptable; voting on project decisions is not.

Who performs: Management is responsible for performing post-implementation reviews of its own projects as part of good governance. Internal audit may perform or evaluate them, but it does not replace management's responsibility.

7. Common Findings

Pre-implementation reviews frequently identify: unapproved or vague requirements, missing control and audit-trail requirements, inadequate segregation of duties in new role designs, incomplete user acceptance testing, data conversion that has not been reconciled, go-live pressure overriding unresolved high-severity defects, and no rollback plan.

Post-implementation reviews frequently identify: benefits that were never measured because no baseline was captured, scope creep without change control, manual workarounds that have become permanent, excessive privileged access left over from the implementation team, controls that were designed but switched off for performance reasons, training gaps leading to data quality problems, and lessons-learned sessions that were skipped because the team moved to the next project.

8. Exam Tips: Answering Questions on Pre- and Post-Implementation Reviews

Tip 1 - Anchor on timing first. Read the question stem and decide immediately whether the scenario is before or after go-live. Many wrong answer choices are simply the right activity at the wrong time. Testing operating effectiveness before a system exists in production is impossible; redesigning requirements after go-live is too late.

Tip 2 - Look for the words benefits, objectives achieved, lessons learned. These point to post-implementation. Words such as readiness, design, requirements, business case, testing sign-off and cut-over point to pre-implementation.

Tip 3 - Choose the answer that preserves independence. If an option has the internal auditor designing controls, approving specifications, writing procedures for management, deciding whether to go live, or acting as project manager, eliminate it. The best answer will have the auditor evaluating, advising, recommending, or reporting, with management retaining decision and ownership.

Tip 4 - Prefer prevention and early involvement when asked about value. If a question asks where internal audit adds the most value or when involvement is most cost-effective, the answer is early in the life cycle, because control weaknesses are cheapest to fix at design.

Tip 5 - Prefer root cause and process improvement in post-implementation questions. When asked about the most important outcome of a post-implementation review, favor answers about confirming benefits realization and feeding lessons learned into future projects over answers about assigning blame or re-performing testing.

Tip 6 - Think in terms of the risk management process, not just the project. CRMA questions often ask what the absence of a review indicates. The best answer links it to a weakness in the organization's risk management or governance process, such as a failure to assess the risk impact of change or a failure to monitor whether risk responses worked.

Tip 7 - Match evidence to the conclusion. If asked for the best evidence that a system is ready for go-live, choose documented user acceptance testing results with business sign-off and defect closure, not a verbal assurance from the project manager. If asked for the best evidence that benefits were realized, choose measured results compared to the baseline in the approved business case, not the project team's closing presentation.

Tip 8 - Remember that management owns the reviews. If a question asks who is responsible for performing post-implementation reviews, the answer is management (the project sponsor or project management office), with internal audit providing assurance over that process. Internal audit may perform the review itself when risk warrants, but it does not relieve management of the responsibility.

Tip 9 - Watch for the stabilization period. A post-implementation review conducted the week after go-live is premature; the system has not operated through a full cycle and benefits cannot be measured. Answers that reference allowing the process to stabilize or complete a full processing cycle are usually correct.

Tip 10 - Eliminate extremes. Options stating that internal audit should review every project, or that pre-implementation involvement always impairs independence, or that post-implementation reviews are unnecessary if the project was on budget, are typically wrong. Risk-based selection, defined roles, and benefits-focused evaluation are the balanced positions the exam rewards.

9. Quick Reference Summary

Pre-implementation review: before go-live; evaluates business case, governance, project risk assessment, requirements, control design, testing, data conversion, training, contingency and readiness; prevents costly rework; auditor advises and evaluates but does not design or decide.

Post-implementation review: after stabilization; evaluates achievement of objectives, benefits realization, cost and schedule performance, operating effectiveness of controls, residual and emerging risk, user adoption, open issues and lessons learned; improves future projects and confirms the risk profile is within appetite.

Risk management assurance angle: assess whether the organization systematically assesses the risk of change, embeds controls at design, verifies results after implementation, and learns from each project. Gaps in any of these are weaknesses in the risk management process that the CRMA-qualified auditor should identify and report.

Master the timing, the objectives, the independence boundary and the evidence, and you will be well prepared for any question on pre- and post-implementation reviews.