Credentials leaks are a major threat to supply chain security. In the hands of an adversary, application development secrets will open the door to a host of attack strategies. The CircleCI hack made this crystal clear for software supply chain risk management.
Many potential leaks can be discovered through scanning software repositories. GitHub recognized that earlier this year when it began offering secrets scanning free on the public repositories that it hosts. But scanning can create its own problems, especially when it comes to remediation of scanning results.
A key to remediation is determining whether tokens — which are used for things such as pushing and pulling code, creating and managing repositories, and opening and closing issues — are active or not, a process that can be ornery and time-consuming. GitHub is tackling that problem with its validity checks feature.
Validity checks remove a lot of the manual effort and friction from the remediation process. A token’s status can be seen within the UI, saving time and allowing remediation tasks to be prioritized so they can be performed more efficiently. That's especially useful when scanning produces hundreds or even thousands of alerts.
With its latest secrets security enhancement, GitHub is extending validity checks to select tokens on other platforms, namely Amazon's AWS, Microsoft, Google, and Slack. "These account for some of the most common types of secrets detected across repositories on GitHub," the platform's senior product marketing manager, Zain Malik, and senior product manager, Courtney Claessens, wrote in a blog post.
GitHub's secrets-scanning efforts are a step in the right direction and will help development teams reduce some secrets leaks. But managing the risk from secrets leaks is bigger than that — and requires a holistic software supply chain security approach.
[ See Special Report: An Essential Guide to Securing Secrets in Software ]
The importance of enhanced secrets scanning
There are many ways for software teams to leak development secrets, and many places to do so. So it's valuable for organizations to be able to scan as many places as they can, said Justin Cappos, a professor in the computer science and engineering department at NYU's Tandon School of Engineering.
"It's really good that [GitHub's] effort is looking in a broader way to secure the ecosystem of credentials API tokens and similar things."
—Justin Cappos
John Bambenek, a principal threat hunter at Netenrich, said such scanning has become increasingly important as organizations become more dependent on cloud services. “Traditional perimeter defenses and other security tools just aren’t available. “When you are talking APIs, it’s not even possible to deploy MFA [multifactor authentication].”
“Lost secrets give adversaries worldwide immediate access to your cloud data and services, and it requires minimal skills to scan repositories to find them."
—John Bambenek
Philip George, executive technical strategist at Merlin Cyber, said GitHub’s extension of its security tools into popular cloud environments is welcome, what "with the growing amount of private and public sector organizations migrating workloads into the cloud."
“CI/CD pipelines will adjust accordingly and take advantage of the cloud's programmable infrastructure, which presents an even greater attack surface and overall risk of threat actors exploiting vulnerabilities posed by inadequate secrets management. However, choosing to extend validity checks and static cryptographic scanning tools across cloud service provider environments can be an effective way to manage this risk.”
—Philip George
One key problem with secrets scanning: Alert fatigue
As important as scanning has become, it often produces an avalanche of alerts. When security alerts become too noisy, legitimate alerts get lost — or worse, or they get ignored, Bambenek said.
“Rapid triaging to remove false positives is essential to prevent analysts from developing muscle memory in bulk closing tickets without sufficient thought."
—John Bambenek
Such scans are known for creating false positives, but Tandon School of Engineering's Cappos said GitHub is “pretty good” at avoiding them. However, while things like credit card numbers are more easily identified, he said, it’s not always possible to tell whether something is a secret or not.
“For example, if I give you a nine-digit number, it could be someone's Social Security number, or it could just be a nine-digit number."
—Justin Cappos
Secrets security demands a holistic approach
As significant as the CircleCI hack of 2023 was, Philip George, executive technical strategist at Merlin Cyber, said the Codecov supply chain breach of 2021 better illustrates the ramifications of inadequate secrets management.
“Threat actors were able to obtain access to the targeted code repository, scan for production secrets and authenticators, then utilized the discovered secrets across production systems, resulting in direct access to protected data and widespread compromise of build and production code."
—Philip George
Preventing secrets compromises is essential to software supply chain security, George added. "Obtaining a level of zero trust coupled with continuous validation across the CI/CD pipeline and container environments will raise the level of assurance for the consumers at the end of the supply chain.”
While discovering secrets in software repositories is a good start in addressing the risk from leaked secrets, it’s only a start, because secrets can be exposed in many other places, Bambenek said.
“I have found them in scripting wrapped around the DevOps workflow, stored in flat files, and even in scripts or other documents stored in SharePoint, OneDrive, or Google Drive. There are many digital equivalents of the ‘password on the Post-it note’ in the digital world, and many of those equivalents are also cloud services that can either be compromised or have overly permissive access controls.”
—John Bambenek
George cited containers as another potential area of attack. “Ensuring that all facets of the container ecosystem are being scanned for secrets management compliance is equally as important as covering code repositories,” he said.
Modern software development environments are complex, and that means risk management needs bigger thinking.ReversingLabs Field CISO Matt Rose said it isn't just about the code or the compiled package, it's the technologies — the tooling— that actually poses the bigger threat to organizations from things like secrets leaks.
"That's why the CircleCI hack was an eye opener to a lot of organizations out there."
—Matt Rose
Secrets risk management: All together now
Cappos said he's glad to see GitHub taking clear steps on security secrets in its repository, noting the fact that they are "the easiest targets in that space."
"There are other places they could look, but you get diminishing returns. What people are trying to do is target the easiest things and knock those out because you get the biggest bang for the buck."
—Justin Cappos
While repository validation is an essential step, Rose said the secrets problem was also growing along with the complexity of modern software development — an organizational risk that could not be ignored.
"Modern applications, software, and cloud infrastructures do not exist without secrets. The problem is that organizations are managing more and more secrets every day. Without prioritization of which secrets are most dangerous you may not be focusing on the right secrets and miss something."
—Matt Rose
[ See Special Report: An Essential Guide to Securing Secrets in Software ]
