Threat Hunting Mis-Behaving PowerShells: Examining the Host
In the first installment of this two-part blog, we built a hypothesis that was leveraged to hunt for mis-behaving PowerShell. With MITRE’s ATT&CK matrix, the focus of the hunt was on the automated exfiltration of data. With the goal of the hunt set, one host stood out amongst the rest. The indicator that something was awry was a host leveraging a PowerShell user-agent string. The second sign was based on the beacon frequency. With Sqrrl’s hunting platform, two alerts were generated for the traffic: beacon and an exfil alert. Armed with a strong conviction, the offending host would be further examined to determine what was causing these alerts.
The network indicators pointed to two areas to hone in on: the beacon/exfil IP address. In this case, the address was the same. Additionally, only one host was observed connecting to this IP address, which easily points us (the hunters) to the source, or at least, very close. If you’re still feeling uncertain, here’s what is known so far:
|Source IP Address:||192.168.7.61|
|Destination IP Address:||22.214.171.124|
|Process (probable, based off of user-agent string):||powershell.exe|
Table 1. Known Indicators
So what is needed to continue our hunt? The hunt matrix (table 2) is a go-to reference that can guide you and your team throughout the duration of a hunt. It serves as the map, or goal, for the duration of the analysis—refer to it often to minimize hunt scope creep (HSC).
|Automated Data Exfiltration (Windows Host Environment)|
|1. What are you looking for? (Hypothesis)||Hypothesis:
PowerShell is being leveraged in my environment for automated data exfiltration
· Anomalies in HTTP user agent strings, such as PowerShell
· Consistent and reoccurring HTTP PUT methods
· PowerShell processes with related network connections
· Parent processes (and process chain) for the PowerShell processes involved with the network connections
Cyber Kill Chain Phase:
Actions on Objectives
|2. Investigation (Data)||Datasets:
For identifying mis-behaving PowerShell on a host, look at:
· EDR Logs
· Windows Event Logs for PowerShell
· PowerShell Transcript Logs
|3. Uncover Patterns and IOCs (Techniques)||
|4. Inform and Enrich Analytics (Takeaways)||The destination IP addresses and user agent string containing PowerShell can be used to help identify other hosts that may be exfiltrating data within environment.
Additionally, with an EDR or process level logs, the MD5s or behavior (scheduled tasks executing PowerShell) could be added to the Risk Trigger in Sqrrl, SIEM, or EDR system to automate future detection or response.
Table 2. Forming the Hypothesis for Host Hunting
The Host in Question
The environment, in this case, was prepared. Every host was setup with Microsoft’s Sysmon and detailed host logging before this hunt was initiated. Because of this fact, the ability to hunt on process based events is available. Without Sysmon, the ability to hunt on the process would be difficult without an EDR system in place. One other method for an organization without an EDR system would be to enable PowerShell logging and specifically, transcript logging for the greatest visibility. Transcript logging does wonders for the hunter; however, on a system that might leverage PowerShell frequently, the setting should be tested thoroughly to ensure no adverse effects. The overall process can be seen in figure 1:
Figure 1. General Process for Examining Host Data with Network Indicators
Searching for Network Connections within Sysmon logs is a technique to rapidly identify where the traffic is originating from. Since the exfil/beacon IP is known, taking the host and searching within Sysmon produces results like what is found in figure 2.
Figure 2. Searching Sysmon for the Exfil IP
This search yields a positive finding. Examining the log entry, the image (or process name) is identified as powershell.exe. This finding further confirms our suspicion of mis-behaving PowerShell.
The next step in hunting the source is identifying the process chain. This is accomplished by one: an EDR system, like Carbon Black Response, which makes easy work of this technique, or two: manual review. With Sysmon, this is a relative easy, but tedious endeavor. What we are trying to accomplish is the identification of the process that was created to make the network connection. This can be done by first filtering the log on Event ID 1, which is a process create (figure 3). Once filtered, searching on the ProcessGuid which was identified in the original search (figure 2) will help point us closer to the source.
Figure 3. Filtering Sysmon Logs on Event ID 1, or Process Create
With the ProcessGuid, searching for this reference will yield the process that actually made the network connection, shown in figure 4.
Figure 4. Search Result for ProcessGuid
Another interesting entry is the CommandLine, which points to a PowerShell script. Interestingly, the script is called with the execution bypass flag, easily going around any execution policies set for PowerShell (not to mention the interesting script name). Additionally, the ParentProcessGuid is there, which is the next level up or next process in the chain. The great thing about Sysmon is that this info is also included in the log entry. The ParentCommandLine points to svchost.exe -k netsvcs -s Schedule. This indicates the script is being called on a schedule. As a hunter, the next area to look would be the Task Schedule and the PowerShell transcript logs (figure 6 and 7).
Figure 5: Task Scheduler
Figure 6. The Task Scheduler Library for the Host Showing the Scheduled Exfil Script
Figure 7. PowerShell Transcript Log Showing a Failed Connection
Two things are now known: The script is set to automatically run on a reoccurring basis and the user that set the job is mateo. Possibly the most burning question of “did we lose data?” can be answered. In this case, the transcript logs show that the web request failed due to a 501 error on the exfil server.
At this point, the next step would be to look at the user mateo to see if there have been other lateral movement attempts/successes and to examine the script. In the case that your organization has the luxury of a SOC or malware analyst team, this could be the juncture where this information is passed along to assist with that analysis, allowing the hunter to continue to pursue the adversary. Looking at these further can help to build additional IOCs or alerts that can be used to build detection capabilities.
In this example, the host was examined for evidence of inappropriate PowerShell usage. The network indicators learned from the first post were leveraged as the starting point to track down suspicious hosts. Leveraging resources such as Sysmon and granular PowerShell logging helped to fully understanding what was occurring on the system. The environment had to be ready ahead of time for this type of data to be available, which is an important reminder to any security team. Taking what was found and developing detection capabilities or conducting further analysis can help to increase the odds of the defenders finding adversaries living off the land.