Supply Chain Attacks: A Ripe Area For Research

Let’s discuss about Supply Chain Attacks and why it’s a great research area in the recent times.

Introduction

I am sure you must have used some sort of software, service, hardware from different vendors. Even the system you are using to view this blog on: be it a mobile application or a desktop has all of these components. You have a bunch of apps from a lot of vendors, your system is built out of components that might have been prepared by different vendors again and you even are using some online services like DNS, CDNs, ISPs, Cloud Service Providers, VPN Service Providers, distribution mediums like Dropbox, Github and what not. So these make up different components you rely on. And we can say that these components are in your “Supply Chain”.

If you are yourself a company making some product then anyone who relies on that product relies on you. So you are in their “Supply Chain”, that is, they depend on you supplying them some software, service, hardware, etc.

The Issue

What are Supply Chain Attacks?

Now imagine that if any application, service or hardware vendor that you are trusting implicitly shipped some malicious code in some update. Now you did not implicitly added any malicious code to your system but the parties you trusted (intentionally or unintentionally) did that. And that’s what a Supply Chain Attack is.

Attack Vectors

It’s a kind of attack that targets software developers, build systems (DevOps Pipelines) and suppliers. Some of the ways the attackers find their ways into your networks and systems are:

modifying source code of legitimate and trusted applications,
infecting the build process, and
tampering with the update mechanisms

Dependency Confusion

Now you might have lately heard of a relatively new class of issues called as Dependency or Namespace confusion.

It’s also a supply chain attack but in these types of attacks, the attackers are not actively attacking anything but instead leaving out malicious code in the package repositories like PyPI (Python Package Index), npm registry, rubygems and so on.

These packages would be of the same name as the packages used by companies internally (and are not available in the public repositories) with higher version numbers and thus whenever the updates are detected, the higher version of the packages get pulled and thus the companies install a malicious package.

The Impact

Now when a package gets installed, it can run code in preinstall scripts like for npm packages and thus once the package is downloaded, an attacker can do all sorts of stuff as they now have code execution capabilities and you are running their malicious code in your trusted environments. That’s a game over right. They can perform recon, privilege escalation, data exfiltration, lateral movements, infect other systems, delete files, and much much more.

These classes of attacks we discussed above are all Supply Chain Attacks. When you are trusting a component, be it an OSS package or a hardware or software vendor, you essentially are trusting them to be secure. And if they turn rogue or get compromised, how can you know about that beforehand? Not an easy task! And that’s why these attacks are quite dangerous and they spread very fast depending on the component you compromise.

The Bigger Picture

For instance, if someone compromised a popular package like “requests” package from Python by pushing malicious code to the package. What do you think the implications would be? I am sure you must have used that package once in a while or it must be used by some other tool that you would have used since the package is quite popular. And thus infecting that package gives an attacker a wide reach. As soon as the package is updated to the malicious version, either automatically by the software build pipelines, by the package managers or even manually, you and many other parties get affected. And that’s what we call as the amplification effect of Supply Chain Attacks!

If a piece of hardware, software or service gets compromised, which is quite widespread, then effectively anyone using that component is compromised too. And you didn’t even had to do anything at all, all thanks to the auto-update mechanisms you have.

So hopefully now you understand what these attacks are and why they are dangerous.

It’s not just the software, services or hardware.

And just to note, it’s not only the hardware, software or the services but even the people can be targeted. Imagine your employees getting a phishing email or their credentials leaked being a vector for gaining the foothold into your systems. Even that could lead to Supply Chain Attacks. An employee might have access to private company repositories and if a malicious actor gets hold of it, not only would they be able to alter the code but even steal any secrets like API keys and Access Tokens; as well as anything ever published to any of the private repositories they could access. The attackers can even generate fresh access tokens to access your repositories without having to bother about passwords being updated. In turn, attackers end up scoring a much more reliable access to your repositories!

Remediation

Why is it hard to prevent?

First of all, I would like to say that it’s not as easy to defend against these issues as it might appear at a first glance.

And here’s why I say so:

How many products: software, services, hardware you use?
Are all of those components secure and use safe coding practices for the software as well as the firmware code?
Are the services those components rely on secure? And are the services lower down the chain secure?

Are you 100% sure about all the above questions?

If yes, then I would say you did a really great job in managing your security against such threats. I would still like to make a point that prevention against Supply Chain Attacks is not an individual’s play but rather it’s a team’s play. It’s because of the last question I asked. You are not only relying on your code or services or hardware. At the end of the day, you might be using some other components which might in turn be using some other components and so on.

A classic example could be: I am using Firefox for instance; Mozilla in turn would have Linux servers and would be using other OSS packages internally. Mozilla might also be using some in-house packages which might be relying on different services like Github or Dropbox. They might even be using some CDNs to deliver their packages. So you see there are a lot of players involved. Those players would be relying on even more players and the (supply) chain continues. Thus, if any party that is being relied on is compromised or turns rogue, we have a big problem. It’s quite difficult to detect things at the deeper levels.

This is what makes this issue harder to eradicate.

What you have to realize is that at the end of the day, your job is to make it harder for an adversary to attack you and raise the cost of the attack to such a level that it’s practically infeasible or unprofitable for an attacker. That’s the best that we can do. With the level of dependency on a lot of hardware & software vendors and packages, it becomes difficult to audit each and every one of them. The level of expertise required in multiple domains is not feasible for smaller organizations or even for a big organization. It requires huge budgets and time commitment!

Some preventive measures

There’s still hope.

Always assume that you have been breached and plan your security accordingly.
Follow the Principle of Least Privilege model holistically.
Have strong authentication and enable 2FA.
Locate and redact secrets from the code being pushed in your build pipelines. Also make sure to have thoroughly code reviewed before pushing it.
Sign your code and safeguard the developer certificate and keys used for code signing.
Add branch protection rules to avoid letting the malicious code enter your codebase and pollute your software. Avoid auto-merges.
Perform code scanning for open source packages that you are installing. Also make sure to see how many users have downloaded the package and how popular it is. Popular software and packages are used by more parties and thus any issues with them would definitely make it in the news (thus spreading awareness) and they also get patched as soon as possible. But this might not be the case with the projects maintained by individuals in their leisure time.
As a company, have contracts with the vendors you are partnering with, outlining the best practices that you would want when receiving services from them.
Defense in depth strategies always helps. Always segregate your components and have security layers around them so that if anything gets compromised, the blast radius is not that huge.
Logging and alert monitoring is also quite critical. If you see any suspicious activity that deviates from the normal behavior of your systems or applications, then it’s worth looking in and performing the analysis.

The Great Solution

There’s one great solution that I left for the end:

Maintain a SBOM (Software bill of materials). It would contain the list of components a service or a software itself relies on.

Think of it like a list of ingredients that a cooked dish has. If you know all of the ingredients you can be sure that there’s nothing dangerous like poison in the dish.

With the list of ingredients, you can dig down and see what components the “compound” ingredients had. And so on…

You get the idea.

And that in nutshell is a great solution to manage the list of dependencies you have and if you see any deviations from the normal/expected set of dependencies, then this list would guide you in further analysis.

Conclusion

So that is it for Supply Chain Attacks and I hope that this was a fun read. And hopefully you now get the idea on why I said it’s a ripe area for research. We are using a lot of applications on a daily basis: cloud services, ISPs, VPN services, Dropbox, Github, etc etc. Any components used by these services can result in compromise of a lot of parties! And that’s why this class of issues become quite juicy to look out for!

Extra Resources

If you wish to learn more on the topic then here are some of the amazing resources:

If you think all this was amazing and wish to try these issues in a live environment, then we do have a lot of labs covering malicious package repositories: both Git, PyPI and APT. And we even have some more labs to show you how to exfiltrate the code repositories (to possibly find weaknesses and secrets within them) for SVN and Git. So give them a try and learn to become a better defender.

In my next posts I will take case studies of some interesting Supply Chain Attacks.

See you next time. Happy learning!