.jpg)
Properly securing containers has never been easy, but the rise of software supply chain attacks — and new threats coming from AI — makes additional security controls essential. Threats and risks must be identified and addressed before containers are deployed, of course, but because the size and complexity of these virtual, self-contained software applications can grow enormously post-deployment, security efforts must never cease.
Here are seven critical best practices to guide your development team on securing containers against today's threats to the software development lifecycle (SDLC).
[ Get White Paper: Close the Supply Chain Security Gap with Complex Binary Analysis ]
1. Start with a clean, minimized container image
The first step in securing and protecting your containers is to begin with clean, minimized container images. That will let developers use stable code that immediately reduces the attack surface, making the containers safer from cybercriminals, said Dave Ferguson, director of technical product management at ReversingLabs.
“Slimming down a container image is probably the No. 1 thing you can do to reduce your attack surfaces, because you don't want anything in your container that you don't actually need to run for your application."
—Dave Ferguson
Once the container is created with stable code, the developer can add specific modules while retaining the minimized approach, focusing on clean code and avoiding accidentally leaving vulnerabilities behind.
By building with purpose, the resulting image is more secure and organized, said Ferguson.
“The way development teams create container images is they start with a base image, where a lot of unnecessary stuff is stripped out of it, and that goes to reducing your attack surface. A minimized container image that is clean and does not have a lot of bloat is a key starting point before you add any of your own application code to the container.”
—Dave Ferguson
2. Scan early to find and fix initial flaws
The initial container image should be scanned early to detect any vulnerabilities that even the cleanest code could have. “We are looking for things when we're building in the early stage of a container construction that could cause a problem down the road,” Ferguson said. But these early scans are merely the foundation for a continuous process of code scanning.
Jasmine Noel, senior product marketing manager at ReversingLabs, said additional security scans are imperative as the container building process continues. That's because vulnerabilities to several types of cyberattacks that are on the rise — malware, tampering, and secrets exposure — can only be found later as the containers approach deployment.
Noel said that these scanning steps become even more important as containers grow to 50GB or more, a size that requires additional scanning power and resources.
3. Control access for all container users
When finalizing containers, teams must ensure that the container environment is secure against unauthorized access from outside users, whether cybercriminals or others, such as former employees. That means auditing the lists of users who have access to the containers.
Ferguson said that security teams must always know who is accessing the containers while also monitoring the systems that provide that access using authorization controls, identification management, principles of least privilege, and other steps.
The monitoring needs to extend to the systems that manage and build the containers, including the continuous integration/continuous integration (CI/CD) systems, to best protect the security and integrity of the container-building tasks, Ferguson said.
4. Lock down container runtime environments
Testing containers during runtime is critical to protect them from external threats, Ferguson said. An excellent resource to help with this, he said, is the Center for Internet Security's benchmarks, which are community-developed secure configuration recommendations that organizations can use to harden their software defenses and infrastructures.
Noel said all components should be tested before you deploy so that you get a complete picture of what you're going to be putting into your runtime before you run it.
“At that point, you're mainly looking for something malicious that has been embedded in there – malware, or someone has tampered with something and it's behaving strangely. It's that sort of stuff that you want to catch just before you deploy.”
—Jasmine Noel
5. Use the best available software testing tools
Deploying the most secure containers starts with having the best available testing tools. Some tools look for vulnerabilities, others check for software license inclusion and validation and for container misconfigurations, and still others provide a wide range of additional checks and tuning for containers and their eventual deployments.
While traditional tools such as static application security testing (SAST), dynamic application security testing (DAST) and software composition analysis (SCA) can help with the baseline stuff, modern threats targeting the software supply chain require new controls such as complex binary analysis, which can scan compiled software package binaries before they are released, Noel said.
The use of artificial intelligence in development poses another security risk. Earlier this year, researchers from cloud security company Wiz found multiple issues with free AI components on Hugging Face, including one that gave attackers the ability to write to internal container registries. In February, JFrog reported finding as many as 100 machine-learning models on Hugging Face with backdoors that could have enabled attackers to access the development environments of organizations using those models.
Anthony Tam, manager of security engineering at Tigera, said the risk from the open-source AI-development ecosystem is of particular concern. He said, however, that frameworks such as Google's SAIF (Secure AI Framework) can help by providing developers solid understanding of how open-source ML models can be best secured.
"Open-source AI has risks just as any open-source software has risks, by being community-driven and -developed."
—Anthony Tam
Several new tools and approaches are also becoming available to help teams mitigate open-source AI risks.
6. Don’t forget the SBOM
Including a software bill of materials (SBOM) with your container code is important so that users receive a list of all the known components that are in the applications that your team builds. But going beyond traditional SBOMs is also a new requirement for dealing with threats.
Today, there are bills of materials for ML-BOMs, AI-BOMs, and cryptographic requirements (CBOMs), ReversingLabs' Ferguson said. But most important to making them actionable is to keep them up to date, he said.
“Ideally, you should be generating an SBOM as you are generating the container image so that you have it every time you build it. You can have a history of how the container changes, including its components. Not everybody does this, but this is something you must use as a best practice."
—Dave Ferguson
Many SBOM requirements are contained in government initiatives that require SBOMs for federal software procurement, but the standard is extending beyond the government. Growing awareness of SBOMs' usefulness has led to their growing adoption. Gartner predicted three years ago that by this year, 60% of organizations would require SBOMs from their vendors when procuring mission-critical software. The other 40% should do so as well.
“It is about software transparency. There have been so many security issues in the past because people are running software and they don't know what's in it.”
—Dave Ferguson
The best example of why transparency is essential is the Log4j vulnerability. Found in the Apache Log4j logging library in 2021, it allowed hackers to easily take control of devices that were running unpatched versions of the code. When potential users later saw that Log4j was included in SBOMs for applications, they knew not to use it if it had not been patched.
With AI's rise, developers are including ML models in their software, making specialized ML-BOMs essential, Ferguson said. CBOMs are becoming more critical with coming quantum computing advances, Ferguson said.
7. Expand your security practice beyond vulnerabilities
Modern threats require expanding container testing practices to examine suspicious files for malware and to seek out the presence of any sensitive data that is inadvertently included.
“Think beyond vulnerabilities. Look at malware injections, tampering, leaked secrets in the code, all these things. These are security issues you need to be aware of and be reviewing. They're not vulnerabilities, but these are huge security issues you need to also be protecting yourself from.”
—Dave Ferguson
ReversingLabs' Noel said that traditional AppSec testing — involving checking source code for vulnerabilities at various stages during development — is not right for final-stage build testing before release. That's where binary analysis-based scanning comes in. “There is a certain group of attacks that you literally can only find at that point,” she said
“With those types of attacks, it's not just a vulnerability that you're fixing now. This is about finding if someone who has broken into your environment in one way or another, and is now interfering with the thing that you're going to deploy and run.”
—Jasmine Noel
A secure SDLC — containers and all — is key
ReversingLabs’ director of product management, Charlie Jones, explained recently in an Open Source Security Foundation (OpenSSF) blog post that the attack surface within software supply chains has grown exponentially. Because of this, it has become even more difficult for organizations to manage risks that stem from open-source, proprietary, and commercial off-the-shelf software. To address these risks head on, Jones urges organizations to look beyond traditional AppSec technology to binary analysis to properly secure their supply chains.
“Compromises such as those of VoIP provider 3CX highlight the gaps in software supply chain security — and the need for a new approach to supply chain risk management.”
—Charlie Jones
A secure software development process — including containers — is key to preventing or minimizing the sorts of issues that could trigger legal exposure under new rules such as the European Union's Product Liability Directive, said ReversingLabs' chief trust officer, Saša Zdjelar. He said organizations should make sure the process includes threat modeling for identifying threats during the design phase, industry-standard coding practices, code reviews, and testing using static and dynamic scans.
Zdjelar said the rapid adoption of containers and the rise of AI-generated code mean that organizations need to bolster their capabilities with advanced technologies such as complex binary analysis and reproducible builds.
