How PacketWatch Network Monitoring Foiled an Initial Access Broker

On January 1, 2024, the PacketWatch team prevented a cyberattack by detecting early signs of malicious activity in a client's network. We uncovered an active, hands-on-keyboard intrusion by an Initial Access Broker (IAB) attempting to establish a strong foothold (persistence) within the client’s network.

Despite our client having an Endpoint Detection and Response (EDR) solution, there were no alerts generated. The threat actor was able to access several hosts and deploy malware onto them.

Only our namesake tool, PacketWatch, and the security analyst team were able to detect this activity and pinpoint the source of the intrusion during a regularly scheduled threat hunt.

Through quick and effective Incident Response, we were able to lead our client’s IT team through remediation and restoration of their IT environment, effectively ousting the intruder with very little damage and no data stolen.

Keep reading for the full breakdown of the event, TTPs, remediation and recovery steps, and lessons learned.

Initial Access

Brute Force Attack on the VPN

Analysis of our client’s firewall logs revealed to us that on January 1, 2024, around 10:53 UTC, the threat actor gained access to our client’s network via a brief brute force attack that lasted until 10:55 UTC that same day.

Due to the Virtual Private Network (VPN) portal not having Multifactor Authentication (MFA) enforced and no controls such as rate-limiting or geo-fencing, the malicious actor was able to try a high volume of credential combinations from the Swiss-based IP 95[.]183[.]48[.]25.

The account the threat actor gained access to was for an employee that normally does not require access to the VPN network or domain computers.

Through our investigation, we discovered misconfigurations within our client’s Active Directory that allowed for user accounts to be created and their passwords never changed, introducing risk for a successful account takeover, such as the one that occurred.

Reconnaissance and Discovery

Scanning for Domain Admin Access

Upon gaining access to the VPN network, the threat actor immediately began to use NMap to scan the environment for active hosts, open ports, and vulnerabilities, and used NMap scripts to brute force RDP and Server Message Block (SMB) accounts.

While legitimate accounts were brute forced, the actor likely ran NMap on defaults, as we observed the username “nmap” in some of the RDP traffic our tool logged. The use of NMap to perform brute-forcing points to the threat actor having low-to-average skill and ability.

We were able to identify clear spikes in RDP, SMB, LDAP, ICMP, and DCE/RPC traffic, which made it easier to identify them and retrace their steps.

Further analysis of the LDAP scanning in PacketWatch’s tool PacketViewer showed clear signs of Active Directory enumeration, wherein the malicious actor successfully scanned for all groups and accounts within the Active Directory.

The purpose of this was to identify key groups and users, such as Domain Admins and key IT admins with those privileges.

This could be Bloodhound activity, but we have no information to be specific as to the exact tool used.

ZeroLogon Exploit Attempt

Our analysis of the DCE/RPC network traffic PacketWatch collected also shows the malicious actor attempted many times to run the ZeroLogon exploit against one of our client’s Domain Controllers. This occurred over a 32-minute period, from 10:52 UTC to 11:24 UTC.

ZeroLogon is a vulnerability that exploits a weakness in the NetLogon (MS-NRPC) protocol’s cryptography that allows for an attacker to send a large number of authentication requests to the Domain Controller (DC) and eventually “zero out” the Domain Controller’s machine account’s password.

The attacker can then reset the Domain Controller’s computer account via a NetrServerPasswordSet2 operation, effectively taking control of the Domain Controller’s computer account. The intended result is full domain compromise.

While this vulnerability was disclosed in September 2020 by Secura’s Tom Tervoort, this vulnerability continues to plague many organization’s networks due to poor vulnerability management.

As of November 2023, it was noted in a CISA report that Rhysidia Ransomware was using ZeroLogon to escalate privileges.

Further analysis of the ZeroLogon network traffic revealed that the malicious actor was not successful in their attempts at this privilege escalation.

To see a successful exploitation of ZeroLogon, we need to see the operations NetrServerReqChallenge, NetrServerAuthenticate, and NetrServerPasswordSet2 (in that order).

All PCAPs analyzed from that traffic did not have any NetrServerPasswordSet2 operation, indicating that the attacker was not able to effectively reset the Domain Controller’s password, and so could not take over the Domain Controller computer account.

We believe that this exploit attempt failed due to our client's Domain Controller being patched for CVE-2020-1472 (ZeroLogon).

Other Domain Admin Access Attempts

Despite failing at running the ZeroLogon exploit against a patched DC, the malicious actor tried several other methods to escalate their privileges to Domain Admin.

Brute Force

The attacker tried RDP and SMB brute forcing from 10:52 UTC to 10:55 UTC.

We believe it is likely the threat actor obtained credentials to an Admin or service account to facilitate lateral movement; however, we do not have the artifacts to show exactly when or where that happened.

Lsassy and Impacket

The attacker also attempted to use Lsassy to dump LSASS on five different hosts from 11:55 UTC to 12:11 UTC.

Hosts included the client’s printer server, two file servers, one of the IT administrators' laptops, and one of the DCs.

While there was no EDR agent installed on the printer server, both the file servers, the IT administrator’s laptop, and the DC had EDR agents on them. Again, no EDR alerts were fired during this incident.

We have some indications in the logs that Microsoft Defender may have intercepted the activity prior to the EDR and blocked it; however, we cannot be certain this was the mechanism that prevented the malicious activity.

This might have been caused by the LSASS dumps failing because of the stand-alone Microsoft Defender agent’s hanging the dump as it attempted (and failed) to analyze the malicious events.

We assess with high confidence that the malicious actor used Lsassy, due to LSASS dump attempt occurring through a scheduled task that spawned from a remotely created service. This is the default method in how Lsassy dumps LSASS.

Additionally, the attacker used Impacket heavily. Lsassy can be easily integrated into Impacket.

Mimikatz in the Music Folder

Next, the attacker dropped Mimikatz.exe into the printer server’s Music folder at 12:00 UTC.

Windows log analysis, as well as forensic analysis of artifacts such as Shimcache and Amcache, did not reveal any sign that Mimikatz had been executed or used.

Analysis of Run keys and other artifacts did not reveal any sign of execution or scheduled execution.

We believe it was most likely used to dump credentials on that server to allow the actor to gain further access.

DPAPI Secrets Harvesting Attempt

Lastily, we noticed in many of the SMB traffic PCAPs an anomalous amount of access to files such as “C:\Windows\System32\Microsoft\Protect\S-1-5-18\[Blob]" which occurred around 13:41 UTC.

We observed this abnormal file access occurring on eight different hosts in the environment, to include all IT administrators’ computers, file servers, printer server, a backup server, and the DC.

Analysis of this activity helped us determine with high confidence that this was a successful attempt at DPAPI secrets harvesting.

In short, DPAPI is a system all Windows OS systems use to store encrypted credentials on the local host. These secrets can be reversed and are built by design to be usable for most applications. A wide variety of software uses this system, from browsers to VPNs.

If a malicious actor were to harvest these secrets and crack them offline, they would have every credential stored on that computer. It is uncommon to see this method of credential harvesting, as it is considered more difficult.

However, by this time the attacker had gained access to a DA account. This is part of what led us to assess with moderate confidence that the malicious actor was most likely an Initial Access Broker (IAB), as the intent of their activity seemed to gain as much access as possible, not to deploy ransomware or steal corporate data.

We also assess with high confidence that they used either Mimikatz or Impacket’s dpapi.py to perform this attack.

Lateral Movement

One of the other ways we detected this activity was through the malicious actor’s lateral movement, which was anomalous when compared to our client’s network baseline.

Spikes in RDP, DCEPRC, and SMB Activity

For example, the attacker used RDP and Impacket primarily to move around the network. This activity was excessive and created clear spikes in RDP, DCEPRC, and SMB that were difficult to miss.

The malicious actor first used the compromised employee’s account to access the DC via RDP.

The malicious actor most likely realized the account was useless for their end goal, so they worked to escalate privileges to DA by gaining access to one of the IT administrator’s accounts.

One anomalously long RDP session had a high databyte count at 11:51 UTC when the malicious actor gained access to the printer server. This is also when they dropped Mimikatz into the Music folder on the printer server.

Analysis of the DCEPRC traffic spikes quickly showed us that the attacker used the open-source network protocol manipulation toolkit Impacket to access several systems.

From 10:55 UTC to 16:23 UTC, the malicious actor primarily used Impacket to access our client’s file servers, IT administrators’ computers, the DC, the printer server, and the backup server. They used both the wmiexec.py and smbexec.py.

Impacket requires legitimate account credentials to work, so the malicious actor had to compromise those accounts first before using this tool.

The RDP and Impacket Detection Gap

Unfortunately, the EDR did not detect any of this activity. Most EDR solutions generally ignore RDP activity. This is often because it is a legitimate tool, and most EDR solutions do not have context on what accounts should be using it and what accounts should not be using it, highlighting the importance of an analyst who understands an organization’s normal activity and behavior.

Impacket is also often undetected by EDR solutions. While some solutions do a good job detecting it when it spawns suspicious discovery commands or drops malicious files, many EDR solutions still struggle to detect when the Impacket shell is created on the host due to the malicious code running in memory and the traffic being a combination of RPC and encrypted SMB traffic.  This is a major blind spot for most EDR solutions.

However, another easy way to find Impacket shells outside of network-based detections is by auditing Windows Security Event code 4688 for cmd.exe process with the command line file name format “/Q /C >1 \\127[.]0[.]01\ADMIN$\__[file named after its creation time in epoch time].”

It will also often have the process tree of wmiprvse.exe > cmd.exe > [malicious commands].

Evidence of Persistence

While the threat actor was able to navigate the network via legitimate accounts and maintained access to the network with these accounts, they seemed to struggle to gain a stable and persistent foothold within the network. More skilled actors often create hidden accounts, install additional RMMs, or setup malware to act as a backdoor. We did not see that in this incident.

Remote Access Trojan (RAT): socks64

However, while conducting triage forensics on the backup server, we noticed in Shimcache a suspicious file named socks64_.exe located in the ProgramData folder. We were able to extract the file for further analysis and clean it off the server.

Our analysis quickly revealed that it was “SystemBC,” a Remote Access Trojan (RAT) that is often used as a backdoor for ransomware operators.

This was another reason why we assessed with moderate confidence that the malicious actor was an IAB, as they had dropped it on the host but did not seem to use it. The intention was most likely to sell off the access to our client’s network to a ransomware operator.

Further forensic investigation of the backup server showed no signs that this malware had been executed.

In our sandbox, the malware created the registry key:

“HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Run\socks5”

and added the value in the sandbox

“powershell.exe -windowstyle hidden -Command \& C:\Users\mike\AppData\Local\Temp\socks64_.exe\”

This was for persistence on every system startup.

Additionally, it made a “call home” to 87[.]251[.]64[.]207 over port 4366 in the sandbox.

Lastly, it spawned a cmd.exe shell in the sandbox.

We did not observe any PCAPs or network data that showed any traffic to the malicious IP or to that port, leading us to conclude that it most likely never executed successfully.

So, even with a RAT dropped on the server, the malicious actor still failed to execute it, even though this server didn’t have EDR protection.

Remediation and Restoration: A Team Effort

Thanks to the PacketWatch threat hunter team’s vigilance, our client’s quick response, and the client’s MSP’s assistance, PacketWatch’s Incident Response team was able to lead our client into successful remediation and restoration of the IT environment. The following steps were taken as part of the remediation process:

• KRBTGT (Kerberos Ticket Granting Ticket) account was reset twice, with 10 hours between each reset, to ensure there would be no chance of a Golden Ticket attack
• All administrator, user, and service accounts had their credentials reset
• The malicious VPN session was terminated, and the compromised employee’s account was disabled
• Any traces of malware (leftover Impacket shells and SystemBC) were cleaned off all hosts
• All Domain Controllers, file servers, printer server, and the backup server were rebuilt from safe, clean backups
• An investigation was opened up with the EDR vendor to determine why the EDR did not detect any of the malicious activity
• MFA was enabled on the VPN

Lessons Learned

As an experienced Digital Forensics and Incident Response (DFIR) organization, we have seen many painful lessons learned repeatedly in most incidents we are called in to triage. We wanted to share some lessons as everyone can learn from them:

• Enable MFA everywhere you can – especially on the VPN
• Segment the network via security policies and/or Access Control Lists (ACL) configurations to limit lateral movement
• Setup a Vulnerability Management Program to help ensure all applications and systems are up to date on all patches
• Ensure EDR is deployed wherever it can be; regularly test your EDR solution to ensure it is detecting threats properly
• Enable Group Managed Service Accounts (gMSA) to help secure service accounts
• Review firewall configurations and policies
• Implement frameworks like NIST or CIS to help identify and quantify risks

Conclusion

This incident highlights the importance of a multi-layered security strategy that includes not just technological defenses but also regular, proactive monitoring and incident response capabilities.

The PacketWatch team's effective response and the lessons learned from this near-miss serve as a valuable case study in the ongoing battle against IABs.

If you are experiencing a cyber incident, reach out to us immediately at our Incident Response Hotline at 1-800-864-4667 and press 9 for Priority Assistance.

Have a team of experienced, battle-tested analysts utilizing cutting-edge network monitoring technology guarding your network and on incident response standby by hiring PacketWatch today.

IOCs Generated

Below are some IOCs from the incident:

C2 IPs

87[.]251[.]64[.]207

95[.]183[.]48[.]25

SystemBC

File Name: socks64_.exe

SHA256: 3ec5f3e34c291534a131b3eb95166f925562235e8a7f7eef1eb337a2d338fbab

Referenced Resources

• https://bi-zone.medium.com/hunting-for-zerologon-f65c61586382
• https://resources.infosecinstitute.com/topics/vulnerabilities/zerologon-cve-2020-1472-technical-overview-and-walkthrough/
• https://www.cisa.gov/news-events/cybersecurity-advisories/aa23-319a
• https://msrc.microsoft.com/update-guide/en-US/vulnerability/CVE-2020-1472
• https://www.synacktiv.com/en/publications/windows-secrets-extraction-a-summary
• https://community.netwitness.com/t5/netwitness-community-blog/using-rsa-netwitness-to-detect-credential-harvesting-lsassy/ba-p/521291
• https://news.sophos.com/en-us/2020/12/16/systembc/
• https://www.crowdstrike.com/blog/how-to-detect-and-prevent-impackets-wmiexec/
• https://m365internals.com/2021/04/27/practical-compromise-recovery-guidance-for-active-directory/
• https://learn.microsoft.com/en-us/windows-server/identity/ad-ds/manage/forest-recovery-guide/ad-forest-recovery-reset-the-krbtgt-password

At PacketWatch, our mission is to safeguard your organization from cyber threats that others may miss. Our team of highly experienced and battle-hardened security professionals works directly with clients to establish full network visibility and an active defense approach to security, including full packet capture and threat hunting within their environment.

DISCLAIMER

Kindly be advised that the information contained in this article is presented with no final evaluation and should be considered raw data. The sole purpose of this information is to provide situational awareness based on the currently available knowledge. We recommend exercising caution and conducting further research as necessary before making any decisions based on this information.