In the context of CompTIA Cybersecurity Analyst+ (CySA+) and Security Operations, **Confidence Levels** serve as a critical metric for evaluating the reliability and accuracy of Threat Intelligence (TI). They represent the provider's certainty that a specific threat, Indicator of Compromise (IOC), …In the context of CompTIA Cybersecurity Analyst+ (CySA+) and Security Operations, **Confidence Levels** serve as a critical metric for evaluating the reliability and accuracy of Threat Intelligence (TI). They represent the provider's certainty that a specific threat, Indicator of Compromise (IOC), or vulnerability is valid and applicable.
Confidence levels allow security analysts to weigh intelligence before making decisions, ensuring that responses are proportional to the certainty of the threat. This is often standardized using frameworks like STIX or the Admiralty Scale (evaluating Source Reliability and Information Credibility).
Generally, confidence is categorized into three tiers:
1. **High Confidence (80–100%):** The intelligence is confirmed, corroborated by multiple independent and reliable sources, or observed directly. In a Security Operations Center (SOC), high-confidence IOCs are often fed into SOAR (Security Orchestration, Automation, and Response) platforms to trigger **automated actions**, such as blocking an IP address or quarantining a file.
2. **Medium Confidence (60–79%):** The information is logical and consistent with recent trends but may lack independent corroboration. Analysts typically treat this as a signal to **investigate** or monitor rather than blocking immediately, as there is a moderate risk of false positives.
3. **Low Confidence (<60%):** The data comes from unknown sources, is uncorroborated, or seems illogical. This data is usually **logged** for future correlation but not acted upon actively to prevent disrupting business operations.
Understanding confidence levels is vital for **Triage**. It prevents alert fatigue by filtering out noise and ensures that automation is applied safely—automatically blocking high-confidence threats while reserving human analysis for ambiguous findings.
Mastering Confidence Levels in Threat Intelligence: A CompTIA CySA+ Guide
Introduction: Why Confidence Levels Matter In the context of Security Operations and the CompTIA CySA+ curriculum, managing threat intelligence is not just about collecting data; it is about evaluating its quality. Threat intelligence comes from various sources—open-source feeds (OSINT), commercial vendors, government agencies, and internal logs. Not all of these sources are equally trustworthy. Confidence levels allow security analysts to score intelligence based on validity and reliability. This is crucial because acting on low-confidence intelligence can lead to wasted resources chasing false positives, while ignoring high-confidence intelligence can result in a breach. Automated systems (SOAR) effectively rely on these scores to decide whether to block an IP address automatically or simply alert an analyst.
What are Confidence Levels? Confidence levels are a standardized method of grading the likelihood that a specific piece of threat intelligence is accurate and that the source providing it is trustworthy. While different platforms (like STIX/TAXII) may use percentages (0-100) or qualitative scales (High, Medium, Low), the most historically significant framework referenced in security operations is the Admiralty System (or NATO System). This system separates the evaluation into two distinct axes: Source Reliability and Information Credibility.
How it Works: The Admiralty/NATO Scale To properly assess intelligence, you must evaluate the source and the data independently. A trustworthy source can sometimes provide bad information, and an untrustworthy source can sometimes tell the truth.
1. Source Reliability (A-F) This measures the track record of the entity providing the data. A - Completely Reliable: No doubt of authenticity, trustworthiness, or competency; has a history of complete accuracy (e.g., a verified internal sensor). B - Usually Reliable: Minor doubt; history of mostly valid information. C - fairly Reliable: Doubts exist; provided valid information in the past. D - Not Usually Reliable: Significant doubt; history of invalid information. E - Unreliable: Lacks authenticity, trustworthiness, and competency. F - Reliability Cannot Be Judged: No basis exists for evaluation (common for new OSINT sources).
2. Information Credibility (1-6) This measures the likelihood that the specific statement is true, often based on corroboration. 1 - Confirmed by other sources: Logical, consistent with other info, confirmed by independent sources. 2 - Probably True: Consistent with other info, not confirmed. 3 - Possibly True: Reasonably logical, agrees with some info, no conflict. 4 - Doubtfully True: Not logical, contradicts other info. 5 - Improbable: Illogical, contradicted by other info. 6 - Truth Cannot Be Judged: The validity cannot be determined.
How to Answer Questions on the Exam When facing CySA+ questions regarding confidence levels, you will often be presented with a scenario where an analyst receives an alert or a feed update. You must decide the next best action.
Step 1: Identify the Source. Is it a well-known vendor (High Reliability) or a random post on a hacker forum (Low Reliability)? Step 2: Check for Corroboration. Do logs from your firewall match the threat report? If yes, the credibility increases. Step 3: Apply the Logic. If intelligence is marked 'Low Confidence,' the answer is rarely to 'Shut down the network.' The answer is usually to 'Verify' or 'Monitor.' If intelligence is 'High Confidence,' the correct answer is usually to 'Block' or 'Remediate.'
Exam Tips: Answering Questions on Confidence Levels Tip 1: Separate Source from Data. Do not confuse the messenger with the message. A question might trick you by saying, 'A source known for rumors (Reliability E) provides an IP address that is confirmed by three other vendors (Credibility 1).' Even though the source is bad, the data is confirmed. You should treat the data as actionable.
Tip 2: The 'F' and '6' Trap. Remember that 'F' (Reliability) and '6' (Credibility) do not mean 'Bad' or 'False.' They mean Unknown. If you see a new zero-day report with a score of F6, it means you need to investigate it manually because no history exists yet.
Tip 3: STIX/TAXII Confidence. While the Admiralty scale is foundational, the exam also covers STIX. Know that in STIX, confidence is often a score from 0 to 100. A score of 90-100 is usually required for automated blocking rules in a firewall implies 'High Confidence,' whereas a score of <50 implies 'Low Confidence' requiring manual review.
Tip 4: Contextualize Action. High confidence justifies immediate containment. Low confidence justifies passive monitoring. If a question asks for the most appropriate response to a 'low confidence' indicator, choose the option that involves gathering more information rather than disrupting business operations.