Don't you hate it when you're using a container image, and it turns out to be infected with cryptomining software?
It can happen, and you might not even know that it's happening, since the goal of the mining software is to steal your CPU cycles, not your data. So, while you happily crunch numbers, or work hard to sell stuff on your website, someone else is getting rich off your cloud bill, or the data center electricity you're generating.
Contaminated containers are a thing. However, they are really hard to detect because what they're stealing is CPU -- as well as electricity. The cryptomining applications love to siphon off processor cycles, but don't affect storage or network bandwidth. (See Cybercriminals Using Kubernetes, Docker to Bitcoin Mine.)
I first heard about this in June, when Kromtech Security reported that it found 17 infected containers on Docker Hub -- the biggest repository for open source pre-configured container images. According to that report:
Even after several complaints on GitHub and Twitter, research made by sysdig.com and fortinet.com, cybercriminals continued to enlarge their malware armory on Docker Hub. With more than 5 million pulls, the docker123321 registry is considered a springboard for cryptomining containers. Today's growing number of publicly accessible misconfigured orchestration platforms like Kubernetes allows hackers to create a fully automated tool that forces these platforms to mine Monero. By pushing malicious images to a Docker Hub registry and pulling it from the victim's system, hackers were able to mine 544.74 Monero, which is equal to $90,000.
In its own report on that study, researchers at Fortinet noted: "We documented Docker images hosted on the Docker Hub registry that were seen to embed malicious malware. This was particularly the case for the Docker account docker123321, which was created in May 2017 (see Figure 1), and which currently provides 19 images under popular project names like Cron, Tomcat, and MySQL."
The Fortinet report explains that the hackers are finding misconfigured containers on Docker Hub or elsewhere, or containers that contain unpatched vulnerabilities.
"Attackers have most probably developed a script to find misconfigured Docker and Kubernetes installations. Docker works as a client/server architecture, meaning the service can be fully managed remotely via the REST API," according to the report.
While I've not found huge numbers of these, I have found other evidence of contaminated containers. As long as the malware is not trying to steal data, or consume too much bandwidth, it can probably be undetected for a long time -- if not forever.
Zero in on the most attractive 5G NR deployment strategies, and take a look ahead to later technology developments and service innovations. Join us for the Deployment Strategies for 5G NR breakfast workshop in LA at MWCA on September 12. Register now to learn from and network with industry experts –
communications service providers get in free!
What can you do about it? Several things:
- Run anti-virus and anti-malware software against your static Docker images and running containers. There are AV signatures that look for ties to known coin-mining software, and also intrusion prevention system (IPS) signatures that look for known issues that might allow hackers to subvert the containers.
- Monitor your deployed containers without any activity by your own applications. If you're not using any CPU resources of your own, coin-mining software or other malware may reveal itself by using CPU. No guarantees there, but it's worth checking.
- Use your firewall settings to prevent network traffic going to addresses or ports that you don't want. For example, the default ports for Monero are 18080 and 28080 for the peer-to-peer network, and 18081 and 28081 for remote procedure calls for accessing the Monero wallet. Similarly, Ethereum uses ports 30301 and 30303. Each coin uses its own ports. There's no reason for traffic on those ports to be allowed from any of your containers, unless you're legitimately doing cryptocurrency transactions. In general, though, tell the firewall to keep your containers locked down tight.
- Always know where your container images come from, and do your due diligence on the container, and its source, if you're using a container from a library. There's nothing wrong with the vast majority of images on Docker Hub, for example. They try to keep the library clean, but ultimately it's the user's responsibility to make sure everything is 100% safe.
Finally, if there's a chance you might have downloaded, tested, or deployed any of the contaminated containers -- such as those from docker123321 -- make sure you've cleaned up the mess.
Docker Hub deleted those known bad container images -- but that doesn't affect images that have already been downloaded and deployed. When was the last time you ran AV against your containers? Today might be a good day to run a quick test.
— Alan Zeichick is principal analyst at Camden Associates, a technology consultancy in Phoenix, Arizona, specializing in enterprise networking, cybersecurity, and software development. Follow him @zeichick.