Control Design and Operating Effectiveness

Why This Topic Matters for the CRMA Exam

The Certification in Risk Management Assurance (CRMA) tests your ability to provide assurance over an organization's risk management processes. At the heart of that assurance lies one fundamental question: do the controls that management relies on to manage risk actually work? Answering that question requires two separate judgments: whether a control is designed properly, and whether it is operating as designed. These are not the same thing, and the exam repeatedly tests whether you understand the difference.

Within the CRMA domain on Evaluating Risk Management Processes, control design and operating effectiveness is where theory meets fieldwork. Everything else in risk assurance - risk identification, risk appetite, governance structures, monitoring - ultimately funnels into whether control activities reduce risk to an acceptable level. If you cannot evaluate controls properly, you cannot conclude on the effectiveness of risk management. This is also where the Three Lines Model becomes concrete: the first line owns and operates controls, the second line designs frameworks and monitors, and the third line (internal audit) independently evaluates design and operation. The exam expects you to think like a third-line assurance provider while understanding the responsibilities of the other lines.

From a practical standpoint, this topic also underpins the IIA Global Internal Audit Standards. Standard 13.3 (Engagement Objectives and Scope), Standard 14 (Conducting Engagement Work), and the requirement to evaluate the adequacy and effectiveness of governance, risk management, and control processes all depend on the ability to assess controls at both the design and operating levels. Questions on this topic therefore blend conceptual knowledge with practical judgment.

What Is Control Design Effectiveness?

A control is effectively designed when, if it operates exactly as intended, it would prevent, detect, or correct the risk it is meant to address, reducing that risk to a level within the organization's risk appetite. Design effectiveness is evaluated on paper and through understanding, before you look at whether people actually follow it.

Key elements of a well-designed control include:

1. Clear linkage to a risk. The control must address a specific, identified risk or control objective. A control that does not map to a risk is either unnecessary or a sign the risk assessment is incomplete.
2. Appropriate type. Preventive controls stop errors or fraud before they occur (segregation of duties, authorization limits, system edit checks). Detective controls identify problems after the fact (reconciliations, exception reports, physical counts). Corrective controls fix identified issues. A high-impact, high-likelihood risk usually warrants preventive controls, with detective controls as a backstop.
3. Correct precision and frequency. A monthly review of a report cannot detect a daily error in time to prevent material loss. A reviewer who only checks totals cannot detect misclassifications within line items. Precision refers to the level of detail at which the control can detect a misstatement or failure; frequency refers to how often it runs relative to how often the risk arises.
4. Competent and appropriately authorized performer. The person performing the control must have the knowledge, authority, and independence to do it properly. A reconciliation reviewed by the same person who prepared it is poorly designed regardless of how diligently they do it.
5. Defined criteria for follow-up. A well-designed control specifies what happens when an exception is found: thresholds, escalation paths, and documentation of resolution.
6. Evidence of performance. If the control leaves no trace (no signature, log, ticket, or system record), it cannot be tested later and may not be reliably performed.

A design deficiency exists when a necessary control is missing entirely, or when an existing control is not properly designed so that even if it operates as prescribed, the control objective would not be met.

What Is Operating Effectiveness?

A control is operating effectively when it is performed consistently, by the right person, in the manner designed, throughout the period under review. Operating effectiveness is about execution. You can only meaningfully test operating effectiveness after concluding the design is adequate; there is no point testing whether people consistently perform a control that would not mitigate the risk even if performed perfectly.

An operating deficiency exists when a properly designed control does not operate as designed, or when the person performing it lacks the necessary authority or competence to perform it effectively.

Consider a simple example. Policy requires the finance manager to review and approve all vendor master file changes before they take effect. If the system allows changes to take effect before approval, that is a design deficiency - the control cannot prevent unauthorized changes. If the system correctly blocks changes until approval, but testing shows the manager approved 30 changes in a single minute without reviewing supporting documentation, that is an operating deficiency - the control exists but is not being performed with the intended rigor.

How the Evaluation Works: Step by Step

Step 1: Understand the risk and control objective. Start with what could go wrong. Use the risk register, risk and control matrices, process documentation, and interviews. Identify the key risks within scope and the controls management says mitigate them. Distinguish key controls (those whose failure would leave a significant risk unmitigated) from non-key controls. Assurance effort should focus on key controls.

Step 2: Evaluate design through walkthroughs and documentation review. A walkthrough traces a single transaction or event from initiation through recording, processing, and reporting, confirming your understanding of how the control actually works in practice. During the walkthrough, ask: Who performs it? What exactly do they do? When and how often? What inputs do they use? What evidence is produced? What happens when something goes wrong? Compare the control as it exists to the risk it should mitigate. Also identify compensating controls - other controls that may mitigate the same risk if the primary control is weak.

Step 3: Conclude on design. If the design is inadequate, you may stop testing that control and report the design deficiency, while assessing whether compensating controls reduce the residual risk. If the design is adequate, proceed to operating effectiveness testing.

Step 4: Plan operating effectiveness tests. Choose the nature, timing, and extent of testing:

- Nature (procedures): The four classic testing techniques, in increasing order of persuasiveness, are inquiry (asking people), observation (watching the control being performed), inspection (examining documents or system records that evidence performance), and reperformance (independently re-executing the control). Inquiry alone is never sufficient to conclude a control is operating effectively. Observation evidences performance only at the moment observed. Inspection and reperformance provide the strongest evidence.
- Timing: Testing should cover the period for which you intend to conclude. If you test only transactions from the last two months but conclude on a full year, your conclusion is unsupported unless you also perform roll-forward or roll-back procedures.
- Extent (sample size): Sample sizes are typically driven by control frequency and risk. Common professional benchmarks: annual control - test 1; quarterly - test 2; monthly - test 2 to 5; weekly - test 5 to 15; daily - test 20 to 40; multiple times per day or recurring transactional controls - test 25 to 60. Higher-risk controls, controls with prior deficiencies, and manual controls justify larger samples. Automated controls, once general IT controls (change management, access) are confirmed effective, may require testing a single instance plus confirmation that the configuration did not change during the period, sometimes called the benchmarking approach.

Step 5: Execute tests and evaluate results. Document each item tested, the attribute examined, and whether it passed. For each exception, determine: Was it an isolated error or systemic? What was the root cause? Did the control fail entirely or was documentation merely missing? Did management detect and correct the issue through another control? Even a single exception in a small sample often means you cannot conclude the control is operating effectively without expanding the sample or evaluating compensating controls.

Step 6: Classify and assess the severity of deficiencies. Severity depends on the magnitude of potential impact and the likelihood that the deficiency will result in a failure to achieve the objective. Consider the volume of transactions, the complexity, susceptibility to fraud, whether compensating controls exist, and the interaction with other deficiencies. Aggregate related deficiencies - several individually minor weaknesses in the same process can together be significant.

Step 7: Conclude and report. Your conclusion should state whether controls are adequately designed and operating effectively to manage the key risks within the scope, supported by sufficient, reliable, relevant, and useful evidence. Recommendations should address root causes, not symptoms. A design deficiency requires a redesign recommendation; an operating deficiency usually requires training, supervision, monitoring, or performance accountability.

Important Distinctions the Exam Expects You to Know

Manual versus automated controls. Manual controls are susceptible to human error and inconsistency, so they require larger samples. Automated controls are consistent but depend on the integrity of the underlying system, so their testing relies heavily on IT general controls. A hybrid control (IT-dependent manual control, such as a manager reviewing a system-generated exception report) requires testing both the completeness and accuracy of the system report and the manual review.

Entity-level versus transaction-level controls. Entity-level controls (tone at the top, risk assessment processes, monitoring, codes of conduct) create the environment in which transaction-level controls operate. Weak entity-level controls increase the risk that transaction-level controls will fail and may justify expanded testing.

Preventive versus detective. Detective controls are only effective if they operate frequently enough and precisely enough to catch failures before harm becomes unacceptable, and if detected issues are actually corrected.

Inherent versus residual risk. Controls bridge the gap between inherent risk (before controls) and residual risk (after controls). The purpose of evaluating design and operation is to validate that residual risk really is where management thinks it is. If controls are not effective, true residual risk is higher than reported, and this is a key finding for risk management assurance.

Management's self-assessment versus independent testing. Control self-assessment (CSA) and second-line monitoring are valuable inputs, but internal audit should not rely on them as the sole evidence of operating effectiveness without evaluating the competence, objectivity, and rigor of the work performed.

How to Answer Exam Questions on This Topic

CRMA questions on control design and operating effectiveness are typically scenario-based. You will be given a description of a control, a test result, or an auditor's situation and asked for the best conclusion, the best next step, or the most appropriate test. Use the following approach:

First, identify which dimension the question is about. Look for signal words. Design questions use language such as 'as described', 'if performed as intended', 'the policy requires', 'the control would not', 'missing', 'not sufficient to address'. Operating questions use language such as 'testing revealed', 'the sample showed', 'was not performed', 'without evidence of review', 'inconsistently', 'during the period'.

Second, apply the sequence rule. Design is always evaluated before operation. If a question describes a design flaw and asks what the auditor should do, the answer is rarely 'expand sample testing' - you do not test the operation of a control that cannot achieve its objective. Instead, report the design deficiency and consider compensating controls.

Third, apply the evidence hierarchy. When asked for the most persuasive or most appropriate test, prefer reperformance and inspection over observation and inquiry. When asked what is insufficient, inquiry alone is the classic answer. When the question asks about a control that leaves no documentary evidence, observation and reperformance become more important.

Fourth, think about the whole period. Any option that concludes on a full period based on a point-in-time test, or on a sample drawn from only part of the period, is a trap.

Fifth, follow exceptions to root cause. If testing finds an exception, the best answer usually involves understanding why it occurred and whether it is systemic before concluding - not immediately declaring the control ineffective, and not ignoring it because it is 'only one'.

Sixth, remember the risk management lens. CRMA is about assurance over risk management, so questions often ask about the implication of control failures for residual risk, risk appetite, and the accuracy of management's risk reporting. The correct answer frequently connects the control finding back to whether the risk is actually within appetite.

Exam Tips: Answering Questions on Control Design and Operating Effectiveness

Tip 1 - Design before operation, always. If an answer option suggests testing operating effectiveness of a control already identified as poorly designed, eliminate it.

Tip 2 - 'Performed as intended' equals design; 'performed consistently' equals operation. Memorize these two phrases. They decode most questions instantly.

Tip 3 - Inquiry alone is never enough. Any option that concludes on operating effectiveness based solely on interviews, management representations, or questionnaires is incorrect.

Tip 4 - Observation proves only the moment. If the question involves concluding on a period, observation alone is insufficient.

Tip 5 - Reperformance is the gold standard. When asked for the strongest evidence, choose reperformance (or inspection when reperformance is impractical).

Tip 6 - Frequency drives sample size. Know the typical ranges. A question describing a daily control tested with a sample of 2 is signaling insufficient extent.

Tip 7 - A missing signature is not automatically a failed control. Distinguish between lack of evidence of performance and lack of performance. The best answer often involves obtaining alternative evidence before concluding - but also recognize that a chronic lack of evidence is itself a design weakness because the control cannot be monitored.

Tip 8 - Segregation of duties and independence are design issues. If the preparer also approves, or the reviewer reports to the person being reviewed, that is a design deficiency regardless of how carefully the work is done.

Tip 9 - Automated controls need IT general controls. If a question involves relying on an automated control, the correct answer typically involves confirming change management and access controls are effective before reducing sample sizes.

Tip 10 - Compensating controls can mitigate but not erase a deficiency. The existence of a compensating control may reduce the severity of a finding; it does not mean the original control is effective. Be alert to options that overstate the effect of compensating controls.

Tip 11 - Root cause determines the recommendation. Design deficiencies call for redesign; operating deficiencies call for training, supervision, monitoring, or accountability. An option recommending more training for a design flaw is wrong.

Tip 12 - Aggregate deficiencies. If a scenario lists several small weaknesses in one process, consider whether collectively they rise to a significant deficiency.

Tip 13 - Connect findings to residual risk. The CRMA frequently asks about the implication for the organization's risk profile. An ineffective key control means residual risk is higher than management reports, and the risk may exceed appetite.

Tip 14 - Watch for the 'most' and 'best' qualifiers. Several options may be partially correct. Choose the one that is most complete, most persuasive, or most directly addresses the risk described.

Tip 15 - Do not overstep the third-line role. Options where the internal auditor designs the control, performs the control, or takes ownership of remediation conflict with independence. The auditor evaluates and recommends; management owns and implements.

Worked Example 1

Scenario: Policy requires that all journal entries over 10,000 be approved by the controller before posting. The auditor learns the accounting system permits posting before approval and the controller reviews entries at month end. What is the auditor's most appropriate conclusion?

Analysis: The control is intended to be preventive (approval before posting), but as it actually works it is detective and operates monthly. Entries could be posted and affect reporting before any review. This is a design deficiency - even if the controller reviews every entry at month end, the control does not achieve the objective of preventing unauthorized postings. The correct answer would identify a design deficiency and recommend a system configuration change or workflow approval, not expanded testing of the month-end review.

Worked Example 2

Scenario: A daily bank reconciliation is well designed. The auditor selects 25 days and finds that on 3 days the reconciliation was prepared but not reviewed by the independent reviewer, and on 1 day it was not prepared at all. What should the auditor do?

Analysis: The design is sound, so these are operating exceptions. Four exceptions in 25 is a high rate and cannot be dismissed. The auditor should investigate root cause (staff absence, workload, lack of monitoring), determine whether any reconciling items went undetected on those dates, consider whether a compensating control caught any issues, and conclude the control is not operating effectively for the period unless alternative evidence demonstrates otherwise. Recommendations should address supervision, backup coverage, and monitoring of completion.

Worked Example 3

Scenario: The auditor wants to conclude that a system access control prevented unauthorized users from approving purchase orders during the year. Which approach provides the most persuasive evidence?

Analysis: Inquiry of the IT manager is insufficient. Observing a user being denied access shows only the current moment. The most persuasive approach is to inspect the system configuration and user access listing, test that the configuration enforces the restriction (reperformance), and confirm through IT general controls testing that there were no unauthorized changes to the configuration during the period. This combination supports a full-period conclusion on an automated control.

Quick Reference Summary

- Design effectiveness: Would the control mitigate the risk if performed as intended? Evaluated via walkthroughs, documentation review, and judgment about type, precision, frequency, performer, and evidence.
- Operating effectiveness: Is the control actually performed consistently as designed throughout the period? Evaluated via inquiry, observation, inspection, and reperformance, using risk-based samples covering the full period.
- Sequence: Design first, then operation.
- Evidence strength: Inquiry (weakest) - Observation - Inspection - Reperformance (strongest).
- Deficiency types: Design deficiency (missing or inadequate control) versus operating deficiency (adequate control not performed properly).
- Severity: Magnitude times likelihood, considering compensating controls and aggregation.
- Outcome: Conclusion on whether key risks are actually mitigated to within risk appetite, with root-cause-based recommendations.

Master these principles, practice decoding scenario language, and apply the sequence and evidence rules consistently, and you will handle every control design and operating effectiveness question on the CRMA exam with confidence.