The Linux Audit Daemon, or auditd, is a key component of the Linux Auditing System, operating in user space to gather and save audit log file records to the disk.

Employing the Linux Audit Daemon is vital for production systems as it records a log of system events.

These logs are essential for monitoring and can be critical for detecting security breaches or incidents, helping maintain system integrity and security.

Why malicious actors target the Linux Audit Daemon?

Attackers often target the auditd because it plays a critical role in security monitoring by logging detailed information about system activities.

Disabling or tampering with auditd logging can help attackers conceal unauthorized actions, evade detection, and maintain persistence within a compromised system.

By stopping or manipulating the daemon, attackers can significantly reduce the likelihood of their activities being recorded, thus weakening the system’s defenses against breaches.

This makes auditd a prime target in efforts to undermine security measures on Linux systems.

The problem we aim to solve

Attackers may employ various methods to disable auditd and tamper with audit logs, ranging from using system utilities to directly killing the service.

Each method of disruption complicates the detection process, as monitoring must consider multiple aspects of the system to ensure auditd’s continuous operation.

The solution

To address this challenge, my approach involves implementing a singular, efficient Splunk query that can detect whether the auditd daemon has been stopped, regardless of the method used.

This solution simplifies monitoring by providing a universal check that encompasses all known techniques for disabling the daemon, utilizing the audit record types.

Understanding the Log Activities

To design a detection rule in Splunk that effectively captures both permanent stops and prolonged stops (e.g., auditd stopped for more than 3 minutes).

We need to consider how these events are logged and how they can be queried efficiently.

Below, I’ll outline the steps and the query to achieve this.

## Understanding the Record Types

Objective

We aim to detect:

1. Permanent stops: No corresponding DAEMON_START after a DAEMON_END.
2. Long stops: DAEMON_END followed by a DAEMON_START with a time difference of more than 110 seconds.
   

Reviewing the logs

To validate our correlation search for detecting interruptions in the Linux Audit Daemon, I first conducted several simulations to generate necessary logs in Splunk:

1. Rebooting the Machine: This action confirmed the seamless restart of auditd. We should fine-tune the detection rule to exclude such scenarios, as they are considered benign activities.
2. Killing the Process: Forcibly stopping auditd helped to simulate an attacker’s evasion tactic, creating logs that would show how the system records an unexpected shutdown.
3. Graceful Stop Using System Utilities: I used commands like serviceor auditctl to gracefully stop the auditd service, then started it after more than 110 seconds. This test aimed to identify if the detection rule could spot prolonged stops, potentially indicating malicious activity.

Sample audit logs which indicate some of the activities described above

Developing the detection rule in Splunk

Disclaimer: The activities logged to test the detection rule were performed on a real Linux server, mimicking the methods a malicious actor might use.

While this approach covered the most common ways to disable auditd, it’s important to note that there are many other tactics that could turn off auditd, potentially leading to unexpected behavior.

In a corporate environment with thousands of servers, the diversity of configurations could generate false positives or false negatives, requiring fine-tuning of the detection rule.

Step 1: Filter events of interest and combine them into transactions

• The initial part filters the log data to include only those from the specified index, sourcetype, and host, and narrows down the events to either DAEMON_END or DAEMON_START.
• Transaction command combines events beginning with a DAEMON_END and include subsequent events occurring within 120 seconds, facilitated by the maxpause parameter.
• This duration adequately captures prolonged stops exceeding 110 seconds. Instances where no DAEMON_START occurs within 120 seconds will be categorized as a stopped daemon.
  
  

Step 2: Extract times and determine alert reason

• Rex command extracts the timestamp from DAEMON_END and DAEMON_START events to use them in the correlation.
• To determine the alert reason, we are going to use duration. The duration represents the time difference between DAEMON_END and DAEMON_START events and is automatically added to the events by the transaction command.
• A duration of 0 indicates that there is only a DAEMON_END event within the 120 seconds transactions period. This suggests either the daemon was stopped and not started again or it was started after more than 120 seconds.
• If the duration equals 0 AND the time difference between the daemon end time and the current time is greater than 110 seconds, an alert for auditd daemon stopped should be triggered. This additional condition prevents immediate alerts upon daemon stoppage, which could result in false positives.
• The second case condition will evaluate to TRUE when a transaction with both DAEMON_END and DAEMON_START events has a duration exceeding 110 seconds. This suggests a prolonged halt of the auditd daemon.
• All other conditions will evaluate to FALSE, causing the detection rule to return no results, as we are specifically seeking alert reasons related to auditd_stopped and long_stop.
  
  

Step 3: Adding additional context to the alert and aggregating the results

• Convert daemon_end_time and daemon_start_time UNIX times to a human-readable format.
• Assign ‘unknown’ string value to the daemon_start_time, if such time does not exist in the event.
• The event_info calculated field provides additional context to the event based on the alert reason.
• If the reason is auditd_stopped, it includes the last seen time of the auditd daemon along with a description of the event. For long_stop alerts, it adds the duration of the stop along with a description.
• Finally, an aggregation of the results is done using the stats command.
  
  

Final thoughts and next steps

The approach detailed in this article represents just one of the many methods available for detecting disruptions and potential tampering in the operation of the Linux Audit Daemon (auditd).

While this approach is vital for ensuring system security and integrity, it’s part of a broader security strategy that includes various tools and techniques.

As we continue to refine our methods and explore new ones, our goal is to enhance our defenses, ensuring swift and accurate responses to security threats and maintaining the resilience of our systems.

This ongoing vigilance is essential for protecting our audit trails and preventing unauthorized access.

Next Steps:

As we continue to enhance our security monitoring capabilities, our next focus will be on expanding the detection rule to include:

Detecting Configuration Changes: Monitoring any unauthorized or unexpected changes to the auditd configuration or its plugins is crucial.

   •  Changes could indicate an attempt to weaken the audit system’s effectiveness or an attempt to bypass security measures.

Identifying Deletion of Audit Rules: Deletion or alteration of audit rules could significantly impact the ability of the system to log important events.

   •  Detecting such changes promptly will help in maintaining the required security and compliance standards.

Improving Rule Accuracy: Ongoing refinement of the detection rules will be necessary to reduce false positives and false negatives.

   •  This will involve adjusting the rules based on feedback and anomalies observed in the log data, ensuring that the rules remain effective as attack techniques evolve.

Broadening the Scope of Detection: Extending the detection capabilities to cover more granular activities.

   •  Detecting specific audit rule manipulations and unusual access patterns to the audit configuration files.

Written by Aleksandar Matev