Log Analysis and Correlation

Log Analysis and Correlation – A Comprehensive Guide for GIAC GCIH Certification

Introduction to Log Analysis and Correlation

Log Analysis and Correlation is a foundational skill within Incident Response and Cyber Investigations. It involves the systematic examination, interpretation, and cross-referencing of log data generated by various systems, applications, and network devices to detect, investigate, and respond to security incidents. For GIAC GCIH candidates, mastering this topic is essential because it directly supports the identification of attacker techniques, the reconstruction of attack timelines, and the development of effective remediation strategies.

Why Log Analysis and Correlation Is Important

Logs are the digital breadcrumbs left behind by every action taken on a network, system, or application. Understanding their importance is critical for several reasons:

• Incident Detection: Logs often contain the earliest indicators of compromise (IOCs). Unusual login attempts, unexpected process executions, or anomalous network connections can all surface through careful log review.

• Attack Reconstruction: During an investigation, logs allow responders to piece together the sequence of events — from initial access through lateral movement to data exfiltration. This timeline reconstruction is essential for understanding the full scope of a breach.

• Evidence Preservation: Logs serve as forensic evidence. Properly collected and preserved logs can support legal proceedings, regulatory compliance, and internal disciplinary actions.

• Root Cause Analysis: Correlating logs from multiple sources helps identify the root cause of an incident, enabling organizations to close vulnerabilities and prevent recurrence.

• Compliance and Auditing: Many regulatory frameworks (PCI-DSS, HIPAA, SOX, GDPR) require organizations to maintain, review, and analyze logs. Failure to do so can result in significant penalties.

• Threat Hunting: Proactive log analysis enables security teams to hunt for threats that may have evaded automated detection systems.

What Is Log Analysis and Correlation?

Log Analysis is the process of reviewing log entries from individual sources to extract meaningful information. This includes parsing raw log data, filtering noise, identifying anomalies, and interpreting events in the context of normal operations.

Log Correlation takes analysis a step further by combining log data from multiple, disparate sources to identify patterns, relationships, and sequences of events that would not be apparent when examining a single log source in isolation. Correlation connects the dots across firewalls, IDS/IPS, operating systems, applications, authentication systems, DNS servers, web proxies, and more.

Key Log Sources in Incident Response

Understanding which logs to examine is critical:

• Firewall Logs: Show allowed and denied connections, source/destination IPs, ports, and protocols. Useful for identifying unauthorized access attempts and data exfiltration channels.

• IDS/IPS Logs: Contain alerts triggered by known attack signatures or anomalous behavior. Provide details about the nature of detected threats.

• Windows Event Logs: Include Security logs (logon events, privilege use, audit policy changes), System logs (service starts/stops, driver failures), and Application logs. Key Event IDs include:
- Event ID 4624: Successful logon
- Event ID 4625: Failed logon attempt
- Event ID 4648: Logon using explicit credentials (pass-the-hash indicator)
- Event ID 4720: User account created
- Event ID 4732: Member added to a security-enabled local group
- Event ID 7045: New service installed (potential persistence mechanism)
- Event ID 1102: Audit log was cleared (anti-forensics indicator)

• Syslog (Linux/Unix): Authentication logs (/var/log/auth.log or /var/log/secure), system logs (/var/log/syslog or /var/log/messages), and application-specific logs.

• Web Server Logs (Apache, IIS, Nginx): Access logs show HTTP requests including URLs, user agents, status codes, and source IPs. Error logs capture application-level failures and potential exploitation attempts (SQL injection, XSS, directory traversal).

• DNS Logs: Reveal domain resolution requests that may indicate C2 (Command and Control) communication, DNS tunneling, or domain generation algorithm (DGA) activity.

• Proxy Logs: Show web traffic patterns, URLs accessed, data volumes transferred, and can reveal beaconing behavior associated with malware C2.

• DHCP Logs: Map IP addresses to MAC addresses at specific times, essential for tying network activity to physical devices.

• Authentication Logs (Active Directory, RADIUS, LDAP): Track user authentication events, group membership changes, and privilege escalation.

• Endpoint Detection and Response (EDR) Logs: Provide detailed process execution, file system changes, registry modifications, and network connections at the endpoint level.

How Log Analysis and Correlation Works

Step 1: Collection and Centralization
Logs must be collected from all relevant sources and centralized in a log management platform or SIEM (Security Information and Event Management) system such as Splunk, ELK Stack (Elasticsearch, Logstash, Kibana), QRadar, ArcSight, or Graylog. Centralization ensures that analysts have a unified view and can correlate across sources.

Step 2: Normalization
Different log sources use different formats (syslog, JSON, CSV, Windows Event Log XML, CEF, LEEF). Normalization converts these into a common format so that fields like timestamp, source IP, destination IP, username, and action can be compared consistently across sources.

Step 3: Time Synchronization
All systems must use synchronized time (NTP — Network Time Protocol). Accurate timestamps are critical for correlation. Without time synchronization, constructing accurate timelines becomes extremely difficult or impossible. This is a frequently tested concept on the GCIH exam.

Step 4: Filtering and Reduction
Not all log entries are relevant. Analysts must filter out known-good activity (baseline noise) to focus on anomalies. This can be accomplished using search queries, regular expressions, and predefined filters.

Step 5: Pattern Recognition and Anomaly Detection
Analysts look for patterns that indicate malicious activity:
• Multiple failed logon attempts followed by a successful logon (brute force attack)
• Logon events from unusual geographic locations or at unusual times
• Large volumes of data transferred to external IPs
• Execution of known malicious tools (psexec, mimikatz, powershell encoded commands)
• DNS queries to known malicious domains or unusual TLDs
• Beaconing behavior (regular, periodic outbound connections)

Step 6: Correlation
This is where multiple log sources are combined to build a comprehensive picture. Correlation techniques include:

• Time-Based Correlation: Identifying events that occur within a specific time window across multiple systems. For example, a firewall log showing an inbound connection at 14:01, followed by a web server log showing exploitation at 14:02, followed by an authentication log showing a new account created at 14:05.

• IP/Host-Based Correlation: Tracking a specific IP address or hostname across multiple log sources to map an attacker's movements.

• User-Based Correlation: Following a specific user account across authentication logs, file access logs, and email logs to detect insider threats or compromised accounts.

• Rule-Based Correlation (SIEM Rules): Automated correlation rules that trigger alerts when specific conditions are met. For example, a SIEM rule that fires when more than 10 failed logons from the same source IP occur within 5 minutes, followed by a successful logon.

• Statistical Correlation: Identifying deviations from baseline behavior using statistical models. Unusual data volumes, connection frequencies, or process execution patterns may indicate compromise.

Step 7: Timeline Construction
Using correlated data, analysts build a chronological timeline of the incident. This timeline maps out:
• Initial compromise vector (how the attacker gained access)
• Exploitation details (what vulnerability or technique was used)
• Post-exploitation activity (privilege escalation, lateral movement, persistence)
• Data staging and exfiltration
• Attacker cleanup or anti-forensics activities

Step 8: Reporting and Documentation
Findings must be clearly documented with supporting evidence from the logs. Reports should include timestamps, source/destination details, affected systems, user accounts involved, and the analyst's interpretation of events.

Common Log Analysis Techniques and Tools

• grep, awk, sed (Linux command-line tools): Powerful for searching and filtering text-based logs. For example: grep '4625' Security.log | awk '{print $5}' | sort | uniq -c | sort -rn to find the most common sources of failed logons.

• Splunk Search Processing Language (SPL): Used in Splunk to query and correlate logs. Example: index=firewall sourcetype=cisco_asa action=denied | stats count by src_ip | sort -count

• Regular Expressions (Regex): Essential for parsing unstructured log data and extracting specific fields.

• Microsoft Log Parser: A versatile tool for querying Windows Event Logs, IIS logs, and other formats using SQL-like syntax.

• Event Log Explorer / Windows Event Viewer: Native tools for browsing and filtering Windows Event Logs.

• Wireshark / tcpdump: While primarily packet capture tools, they complement log analysis by providing network-level evidence that can be correlated with log entries.

Common Attack Indicators Found Through Log Correlation

• Brute Force / Password Spraying: Multiple Event ID 4625 (failed logon) entries from the same or multiple source IPs, targeting one or many accounts, followed by Event ID 4624 (successful logon).

• Pass-the-Hash / Pass-the-Ticket: Event ID 4624 with Logon Type 9 (NewCredentials) or Event ID 4648 (explicit credential usage), especially when combined with lateral movement patterns.

• Lateral Movement: Logon events (Type 3 — network logon, or Type 10 — RemoteInteractive/RDP) appearing across multiple systems in sequence, often using the same compromised account.

• Privilege Escalation: Event ID 4672 (special privileges assigned to new logon), Event ID 4728/4732/4756 (user added to privileged group).

• Data Exfiltration: Firewall or proxy logs showing large outbound data transfers, especially to unusual destinations or using unusual protocols (DNS tunneling, ICMP tunneling, HTTP/S to non-standard ports).

• Persistence Mechanisms: Event ID 7045 (new service installed), scheduled task creation logs, registry modification logs, and startup folder changes.

• Log Tampering / Anti-Forensics: Event ID 1102 (Security log cleared), gaps in log sequences, or missing logs from specific time periods.

• Web Application Attacks: Web server access logs containing SQL injection patterns (UNION SELECT, OR 1=1), directory traversal (../../), command injection (;ls, |whoami), or encoded payloads (%27, %3B).

• C2 Beaconing: Regular, periodic outbound connections (e.g., every 60 seconds) to the same external IP or domain, visible in firewall, proxy, or DNS logs.

Challenges in Log Analysis and Correlation

• Volume: Modern environments generate enormous quantities of log data. Effective filtering and prioritization are essential.
• Format Diversity: Different vendors and systems produce logs in different formats, making normalization critical.
• Time Zone Differences: Systems in different time zones must have their timestamps normalized to a common reference (usually UTC).
• Log Retention: Insufficient retention periods may mean that critical evidence has already been purged.
• Encryption: Encrypted traffic (TLS/SSL) limits visibility into payload content without decryption capabilities.
• Attacker Evasion: Sophisticated attackers may clear logs, use living-off-the-land techniques, or blend in with normal traffic to avoid detection.

Exam Tips: Answering Questions on Log Analysis and Correlation

The GCIH exam will test your ability to interpret log data, identify attack patterns, and understand the principles of effective log correlation. Here are targeted tips:

1. Know Your Windows Event IDs: This cannot be overstated. Be able to recognize key Event IDs (4624, 4625, 4648, 4672, 4720, 4732, 7045, 1102) and understand what they indicate. Know the difference between Logon Types (Type 2 = Interactive, Type 3 = Network, Type 10 = RemoteInteractive/RDP). Create a reference sheet or index tab for quick lookup during the exam.

2. Understand Logon Types: Questions may present a log entry and ask you to identify the type of access. Logon Type 3 is commonly associated with lateral movement via SMB/network shares. Logon Type 10 indicates RDP. Logon Type 9 (NewCredentials) is associated with pass-the-hash techniques.

3. Time Synchronization (NTP): Expect questions about why NTP is critical for incident response. The correct answer almost always relates to the need for accurate timestamps to enable effective log correlation and timeline reconstruction.

4. Read Log Entries Carefully: Exam questions may present raw log excerpts (firewall logs, IDS alerts, web server access logs). Read every field carefully. Pay attention to source and destination IPs, ports, timestamps, HTTP methods, status codes, and user agents. A single detail can change the correct answer.

5. Recognize Attack Patterns in Logs: Be prepared to identify:
- Brute force patterns (many failed logons followed by success)
- SQL injection in web logs (look for UNION, SELECT, OR 1=1, single quotes, semicolons)
- Directory traversal (../ sequences)
- Command injection (pipe characters, semicolons followed by system commands)
- DNS tunneling (unusually long DNS queries, high volume of DNS requests to a single domain)
- Port scanning (many connections to different ports on the same host in a short time)

6. Correlation Across Sources: Some questions will require you to correlate information from multiple log sources presented together. Practice linking a firewall allow entry to a corresponding IDS alert and a web server access log entry to identify the complete picture of an attack.

7. Understand SIEM Concepts: Know what SIEM systems do (collect, normalize, correlate, alert) and the value they provide. Understand the difference between log management (storage and retrieval) and SIEM (active correlation and alerting).

8. Know the Importance of Baselines: To identify anomalies, you must first understand what normal looks like. Questions may test your understanding of why establishing a baseline of normal activity is a prerequisite for effective log analysis.

9. UTC and Time Zones: If a question involves logs from systems in different time zones, convert all timestamps to UTC before attempting to correlate events. The exam may test whether you can identify the correct sequence of events when time zones differ.

10. Log Integrity and Chain of Custody: Understand that logs should be protected from tampering using write-once storage, centralized logging (forwarding logs to a secure, separate system), hashing, and digital signatures. Questions about evidence handling may reference these concepts.

11. Index Your Books: The GCIH exam is open-book. Index key topics such as Event IDs, log formats, SIEM correlation rules, common attack indicators, and specific tool commands. Having quick access to reference material will save you valuable time.

12. Practice with Real Logs: Before the exam, practice analyzing sample logs from various sources. Familiarity with real-world log formats will make it much easier to quickly interpret log entries presented in exam questions.

13. Elimination Strategy: When unsure, eliminate obviously incorrect answers first. For log analysis questions, look for answers that misidentify the source/destination, misinterpret the event type, or draw conclusions not supported by the log data presented.

14. Think Like an Incident Handler: Many questions are framed from the perspective of an incident handler investigating an alert. Ask yourself: What is the log telling me? What happened? What should I investigate next? What is the appropriate response?

Summary

Log Analysis and Correlation is a critical competency for the GCIH certification and for real-world incident handling. It requires knowledge of diverse log sources, the ability to normalize and correlate data across systems, and the analytical skills to recognize attack patterns within potentially massive volumes of data. By understanding key log sources, memorizing critical Event IDs, practicing with real log data, and indexing your reference materials effectively, you will be well-prepared to tackle log analysis questions on the GCIH exam with confidence.