Nearly a week after the news broke that software distributed by voice over IP (VOIP) software vendor 3CX had been hacked, a picture has emerged of a long-running, targeted attack possibly designed to push malicious code to the company’s customers in the cryptocurrency industry.
Reporting and analysis immediately following a March 29 disclosure suggested that 3CX was aware of irregularities with its 3CXDesktopApp as long as a week prior to acknowledging the security issue, and failed to detect evidence of tampering both before and immediately after pushing compromised software updates to its customers.
Here's what your software teams need to understand about the 3CX breach.
SEE RELATED:
- Research: Red flags flew over 3CX update
- Webinar: Deconstructing the 3CX Software Supply Chain Attack
- Deminar: Analyzing the 3CX Software Package
An April Fools joke? Unfortunately not.
In a short security incident update on April 1, 3CX CEO Nick Galea informed customers that the company had been the victim of a software supply chain attack. Galea attributed the attack to “a highly experienced and knowledgeable hacker [that was] exploiting a vulnerability in our product.”
But reports since the breach say that 3CX had reason to suspect that its desktop client had been compromised well before then. Endpoint detection software tools such as SentinelOne were flagging both the Windows and Mac version of the client as malware, according to 3CX customer form posts starting on March 22.
That prompted 3CX to test its update against the VirusTotal website, which it used to test the file against scores of anti-malware engines simultaneously, says a report in The Register. Those scans failed to generate any new warnings, so 3CX assumed that the SentinelOne finding was a “false positive,” Galea is quoted as saying.
Then on March 29th, after security firm Crowdstrike scanned the 3CXDesktopApp update and gave the company “full details,” the firm finally began urging customers to uninstall the compromised software, Galea told The Register. But in the intervening days, many organizations and managed security providers (MSPs) deemed the warnings from EDR vendors “false positives” and whitelisted the 3CXDesktopApp, leading to recriminations after the warnings were shown to be accurate.
The 3CX breach was isolated, not an open source supply chain issue
The April 1 announcement, and its reference to an attack on the “larger supply chain,” followed warnings by 3CX about an unspecified security issue with one of the "bundled libraries" compiled into the Windows Electron 3CXDesktopApp.
A ReversingLabs analysis concluded that 3CX was actually the victim of a targeted supply chain attack, not an opportunistic attack that exploited a vulnerability in a shared software library. It found discrepancies in 3CX's versions of two standard libraries used with the Electron open source framework on which the 3CXDesktopApp client is built: ffmpeg and d3dcompiler_47.
Specifically, researchers found RC4 encrypted shellcode appended to the signature appendix of d3dcompiler. A reference to the d3dcompiler library had been added to the ffmpeg library, which was used to call and execute the shell code. That’s a common strategy attackers use to shuttle malicious code onto a system alongside a signed executable, although it has legitimate applications as well. (BleepingComputer published a history of why this is still possible.)
ReversingLabs researchers say it's likely that the compromise of the repository from which the 3CXDesktopApp Electron application binaries had been fetched during the build process caused the supply chain incident. As part of the attack, the bad actors probably replaced the legitimate versions of ffmpeg and d3dcompiler libraries with malicious versions that they compiled after modifying the publicly available ffmpeg source code.
The compromised libraries enabled attackers to download shellcode to systems running the 3CXDesktopApp that facilitated communication with malicious command-and- control infrastructure operated by the attackers. The malicious payload downloaded so-called “infostealer,” dubbed “ICONIC,” which was programmed to collect the system information and browsing history of the infected computer and uploaded it to the malicious command and control server. (Read our full analysis of the 3CXDesktopApp update.)
So far, there's no evidence that publicly shared open source components used by 3CXDesktopApp played a role in the attack, or that other non-3CX applications or development organizations were targeted. However, confusion about the circumstances leading to the compromise have forced some of organizations in 3CX’s supply chain to step forward. The Twitter account for the ffmpeg open source project, for example, posted a notice on March 30 to clarify that its source code was secure:
“There have been several incorrect reports that FFmpeg has been involved in the distribution of malware. FFmpeg only provides source code and the source code has not been compromised. Any 'ffmpeg.dll' that has been compromised is the responsibility of the vendor.”
Malware links 3CX to Lazarus Group crypto campaign
While 3CX appears to be the only application targeted in this compromise, there is evidence that the attack was months in the making, and that the malicious actors behind the compromise had their eyes on a broader set of targets.
Analysis of the command-and-control infrastructure infrastructure by incident response firm Volexity, as well as public GitHub artifacts, suggests that the attackers had access to 3CX’s environment as early as November 2022. While the first observed attack on 3CX took place in late March, Volexity notes that the intervening months may have witnessed other, as-yet-undetected malicious acts.
Also, a Kaspersky Labs analysis on April 3 determined that a malicious DLL named guard64.dll had been loaded into the infected 3CXDesktopApp.exe process. Kaspersky has observed that same library alongside recent deployments of “Gopuram," a backdoor that had been previously used to attack a Southeast Asian cryptocurrency company. Researchers found the malware alongside AppleJeus, a backdoor attributed to the North Korean APT group Lazarus.
Kaspersky has noted “a few victims” compromised with Gopuram in recent years, but “the number of infections began to increase in March 2023,” with cryptocurrency firms being the main target, it said. As it turned out, that increase was the result of the 3CX supply chain attack, though Kaspersky didn’t make the connection at the time.
The lesson for development teams: Don’t be complacent
The message for software vendors here is to not be complacent. In this case there was ample evidence of tampering with the desktop client updates that 3CX sent out. Even in the absence of warnings by EDR vendors, those should have been enough to put a pause on distributing the update.
There was ample evidence of tampering with the desktop client updates that 3CX sent out.
With sophisticated actors increasingly interested in abusing the hard-earned reputation of software publishers to distribute malware, vendors need to be on guard for signs that malicious actors are at work within vendors' development and build processes. That awareness may not stop compromises, but will make it less likely that a software vendor's customers end up suffering the consequences of the company's security failings.
