Above-the-Line and Below-the-Line Testing

Why This Topic Matters
Automated transaction monitoring systems (TMS) are the engine of a modern AML programme. They generate alerts by comparing customer activity against rules, scenarios, and thresholds (for example, cash deposits totalling more than $9,000 in a rolling seven-day period). The problem is that a threshold is only ever an assumption. If it is set too low, the system drowns investigators in false positives; if it is set too high, genuinely suspicious activity slips through undetected. Regulators (FinCEN, OCC, FCA, MAS, and others) expect institutions to prove that their thresholds are reasonable and risk-based, not arbitrary. Above-the-line (ATL) and below-the-line (BTL) testing is the primary technique used to provide that evidence. On the CAMS exam, this topic sits within the Monitoring, Investigation and AI/Technology domain and is tested through scenario questions about model validation, tuning, and system effectiveness.

What It Is
Imagine a threshold as a horizontal line. Transactions or aggregated activity above the line trigger an alert; activity below the line does not.

Above-the-Line (ATL) testing examines a sample of the alerts that were generated. The goal is to evaluate alert quality: how many led to a Suspicious Activity Report (SAR/STR), how many were quickly closed as false positives, and whether the rule is producing "productive" alerts. ATL testing answers the question: Is this rule too sensitive, creating noise and wasting investigator time?

Below-the-Line (BTL) testing examines a sample of activity that fell just below the threshold and therefore did not generate an alert. Analysts review this activity as if it had alerted. If a meaningful proportion would have warranted escalation or a SAR, the threshold is too high and the institution has a detection gap. BTL testing answers the question: Is this rule missing suspicious activity that we should be catching?

Together, ATL and BTL testing help an institution find the optimal threshold where false positives are manageable and false negatives are minimised.

How It Works in Practice
1. Select the scenario or rule to be tested (for example, structuring, rapid movement of funds, high-risk jurisdiction wires).
2. Define the testing bands. For BTL, this is usually a band immediately beneath the threshold (for example, 70% to 99% of the threshold value). For ATL, the sample is drawn from alerts generated during the review period, often stratified by alert score or amount.
3. Draw a statistically valid sample. Sample sizes should be documented and defensible; random or stratified sampling is common.
4. Investigators review each item using the same procedures applied to real alerts and classify outcomes: SAR-worthy, requires further review, or no concern.
5. Analyse the results.
  - A high false-positive rate in ATL suggests the threshold could be raised or the rule refined (adding segmentation, exclusions, or additional conditions).
  - Any SAR-worthy findings in BTL indicate the threshold should be lowered or the rule logic broadened.
  - If BTL finds nothing and ATL shows reasonable productivity, the threshold is supported.
6. Document and govern. Results, methodology, decisions, and approvals are recorded. Threshold changes go through change-management and model-risk governance, and the exercise is repeated periodically (often annually or after significant business, product, or risk changes).

Relationship to Model Validation and Tuning
ATL/BTL testing is one component of broader TMS validation, which also includes data integrity testing (is complete and accurate data flowing into the system?), scenario coverage assessment (do the rules map to the institution's risk assessment?), and conceptual soundness review. It is usually performed by compliance or a model validation function with appropriate independence, and results are often reviewed by internal audit. Regulators such as the OCC (Bulletin 2011-12 on model risk management) treat transaction monitoring systems as models that require validation. With the rise of machine-learning and AI-driven monitoring, the principle remains the same: institutions must test what the system catches and what it misses.

Key Terminology to Remember
- False positive: an alert that, on review, is not suspicious.
- False negative: suspicious activity the system failed to flag; BTL testing is designed to detect these.
- Threshold/parameter: the numeric trigger of a rule.
- Tuning: adjusting thresholds and logic based on testing results.
- Productivity rate: the proportion of alerts that lead to a SAR or escalation.
- Segmentation: applying different thresholds to different customer groups (retail vs. corporate, low vs. high risk).

Exam Tips: Answering Questions on Above-the-Line and Below-the-Line Testing

Tip 1: Anchor on the direction of the line. Above the line = alerts that fired = testing for false positives and efficiency. Below the line = activity that did not alert = testing for false negatives and effectiveness gaps. If a question describes reviewing "transactions that did not generate alerts," the answer involves BTL testing.

Tip 2: Match the finding to the action. If BTL testing uncovers SAR-worthy activity, the correct response is to lower the threshold or expand the rule, not to increase it. If ATL testing shows almost all alerts are false positives, the response is to raise or refine the threshold. Watch for distractors that reverse this logic.

Tip 3: Never choose "ignore" or "accept the gap" options. CAMS answers consistently favour documented, risk-based remediation. An option that says the institution should simply accept missed activity because alert volumes are already high is wrong.

Tip 4: Recognise that testing must be documented and periodic. Questions about a regulator asking "how do you know your thresholds are appropriate?" are best answered with reference to documented ATL/BTL testing, sampling methodology, and governance approval, not with "the vendor set them" or "they have not changed since implementation."

Tip 5: Distinguish ATL/BTL testing from other controls. Data integrity testing checks inputs; scenario coverage checks whether risks are addressed by any rule; ATL/BTL checks whether an existing rule's threshold is calibrated correctly. If a question asks about missing transaction feeds, the answer is data validation, not BTL testing.

Tip 6: Remember independence and model risk management. The best answer often involves an independent review or validation function, with internal audit providing a further line of assurance, consistent with the three-lines-of-defence model.

Tip 7: Think in terms of balance. The exam rewards the understanding that the objective is not zero alerts or maximum alerts, but an optimal threshold that reflects the institution's risk profile while using investigative resources effectively.

Sample Question Walk-Through
A bank's structuring rule alerts on cash deposits aggregating over $9,500 in seven days. During a periodic review, analysts examine a sample of customers whose aggregated deposits fell between $8,000 and $9,499 and find several patterns that would have merited a SAR. What should the bank do?
Reasoning: The review of non-alerting activity is BTL testing. SAR-worthy findings indicate false negatives. The correct action is to lower the threshold (or adjust the rule logic), document the analysis, and route the change through governance. Options suggesting the threshold be raised, left unchanged because alert volume is acceptable, or that the findings be dismissed because no alert was triggered are incorrect.

Summary
Above-the-line testing validates that alerts being generated are meaningful; below-the-line testing validates that suspicious activity is not escaping detection just beneath the threshold. Together they provide the documented, risk-based evidence regulators expect to see for transaction monitoring calibration. For the CAMS exam, know the direction of each test, the type of error each one detects, the corrective action each finding implies, and the governance expectations surrounding the process.