NordVPN Promotion

Home / Blogs

Why Container Security Is Still Fighting Yesterday’s Battle

Picture a security team reviewing a Kubernetes cluster after a compromise. The attacker used nothing exotic. They created a symlink so that a common utility ran under the name of a trusted system process, split a suspicious command across ordinary-looking steps, and moved on. The rule-based runtime detection engine watching the cluster never fired, because nothing it was written to look for technically happened.

That scenario is illustrative rather than a specific incident, but every technique in it is publicly documented. Researchers at BlackBerry showed that simply creating a symlink named after one of the binaries the default Falco rules treat as an exception was enough to read a sensitive file such as /etc/shadow without an alert.1Falco maintainer Leonardo Di Donato made a similar point at KubeCon North America 2020: a rule that detects execution of a specific binary is only reliable if it also watches copies, renames and symlinks of that file.2 Independent researchers have since shown that copying a network tool under a new name is enough to break detections that match on process names.3

This should feel familiar, because the industry solved a version of this problem on the endpoint. Signature and rule-based detection on Windows and Linux servers was progressively supplemented by behavioral detection precisely because attackers learned to evade static pattern-matching by looking slightly different each time. Container and Kubernetes security is where that lesson is now being relearned.

How today’s leading open-source runtime engines work

Consider two of the most prominent open-source container runtime security engines: Falco and Tracee. Falco, originally created by Sysdig and now a graduated Cloud Native Computing Foundation (CNCF) project, parses Linux system calls at runtime and evaluates that stream against a rules engine; its own documentation describes rules as the conditions under which an alert is generated.4 Tracee, an Aqua Security open-source project, uses eBPF to collect events and detects suspicious activity against a pre-defined set of behavioral patterns, which it calls signatures.5

This is not a criticism of engineering quality. Both are mature, well-maintained and genuinely useful, and their rule and signature libraries reflect years of expertise. Tracee’s signatures, in particular, describe behavioral sequences rather than single events. But in both cases, detection depends on a human having anticipated the pattern in advance and written it down. The Falco project is candid about the trade-off: a comment in its own rules repository notes that matching command-line arguments can generally be bypassed quite easily.6

Some vendors have recently layered large language models on top of these engines; Sysdig, for example, markets Sysdig Sage as an agentic AI analyst for cloud security.7 It is worth being precise about what this kind of layer adds. In my assessment, an LLM working from rule-generated alerts can explain and prioritize those alerts well, which is a meaningful gain for analyst workflow. What it cannot do is surface activity that no rule flagged in the first place. Unless the underlying detection changes, the intelligence has been added to the explanation layer, not the detection layer.

Why the gap matters more in containers

Containers make the weaknesses of static detection more acute, for three structural reasons. First, they are short-lived. Sysdig’s 2023 Cloud-Native Security and Usage Report found that 72% of containers live less than five minutes, and noted that gathering troubleshooting information after a container is gone is nearly impossible.7 A malicious process that runs for seconds leaves little room for a detection that depends on after-the-fact rule updates.

Second, the volume and uniformity of containers means a single missed technique can be replayed identically across many pods built from the same image. Third, containers are Linux processes with added isolation, so the evasion techniques that defeat static detection on a Linux server, such as renaming binaries, abusing symlinks and obfuscating arguments, work just as well inside a container, as the research cited above demonstrates.1, 2, 3

The counterargument, and where it falls short

Defenders of rule-based detection make a fair case: rules are transparent, auditable, cheap to run at scale, and effective against the large share of real-world attacks that rely on commodity techniques. That is exactly why rule-based detection should not be discarded. But effectiveness against common attacks is a different claim from effectiveness against novel ones. A technique that no one has written a rule for is, by definition, the case a static rule list cannot cover until someone updates it, and that update typically follows discovery rather than preceding it.

The stakes are real. In Red Hat’s 2024 State of Kubernetes Security report, a survey of 600 DevOps, engineering and security professionals, 46% of respondents said their organization had lost revenue or customers as a result of a container or Kubernetes security incident in the previous year. 8

A more honest way forward

The fix is not to declare rule-based detection obsolete. It is to stop treating it as a complete answer for an environment as dynamic as Kubernetes, and to ask of container security the question the industry now routinely asks of endpoint security: does this system model behavior, or does it only match patterns? Teams evaluating runtime tools should press vendors on a distinction that is easy to blur in a sales conversation: whether “AI-powered” means a model trained to recognize anomalous behavior, or an LLM summarizing what a rule already caught. Those are different capabilities sharing the same marketing language.

What does behavioral detection look like in practice? One approach, and the one Eunomatix takes with its Logster platform, is to model activity as a graph rather than as a list of individual events. Processes, files, network connections and the relationships between them become nodes and edges, and a learned model scores how anomalous a pattern of activity is over sliding time windows. This changes what an attacker has to hide. Renaming a binary or routing it through a symlink changes a name, but it does not change what the process does, what it touches or what spawned it, so detection does not hinge on a string match. Because the model evaluates activity across a window of time, a malicious command split into two ordinary-looking steps can still be correlated into a single suspicious sequence. Explanation is then layered on top of that detection: an LLM describes the attack chain and maps it to MITRE ATT&CK, rather than narrating an alert a rule produced. Logster applies this today to Windows and Linux hosts using an eBPF-based Linux sensor, which is the same kernel telemetry foundation container runtime tools increasingly rely on. That is why I believe the approach carries over naturally to containers, although behavioral models bring their own trade-offs, including the need for good baselines and careful tuning to keep false positives manageable.

It is equally important to recognize that container security has two layers that are often evaluated separately: what happens inside a running pod, and what happens in the Kubernetes control plane, the administrative layer that decides where workloads run and who may change them. Attackers already target the latter. In 2023, Aqua Security’s Nautilus team documented what it described as the first evidence of attackers abusing Kubernetes role-based access control (RBAC) in the wild. In the campaign, named RBAC Buster, attackers used a misconfigured API server to create a cluster role and binding that gave them persistent, near-admin access, and the campaign targeted at least 60 clusters.9 Much of that activity takes the form of legitimate-looking API calls rather than suspicious process behavior inside a container, so a strategy that watches only in-container runtime activity, rules-based or otherwise, will see little of it.

Kubernetes adoption is not slowing, and neither is attacker interest in it. The industry has already run this experiment on the endpoint and knows the outcome: static detection buys time, not safety. The organizations that get ahead of this will not be the ones with the most rules. They will be the ones that recognized, before an incident report forced the point, that a lesson about detection architecture still applies when the workload moves into a container.

NORDVPN DISCOUNT - CircleID x NordVPN
Get NordVPN  [74% +3 extra months, from $2.99/month]
By Bismah Malik, AI & Security Product Manager at EUNOMATIX

Bismah focuses on AI-driven, behavioral threat detection for enterprise environments, working at the intersection of product strategy and applied security research. (Disclosure: The author works for Eunomatix, which develops Logster, a behavioral, AI-driven threat detection platform referenced in this article. The views expressed here are the author’s own.)

Visit Page

Filed Under

Comments

Comment Title:

  Notify me of follow-up comments

We encourage you to post comments and engage in discussions that advance this post through relevant opinion, anecdotes, links and data. If you see a comment that you believe is irrelevant or inappropriate, you can report it using the link at the end of each comment. Views expressed in the comments do not represent those of CircleID. For more information on our comment policy, see Codes of Conduct.

CircleID Newsletter The Weekly Wrap

More and more professionals are choosing to publish critical posts on CircleID from all corners of the Internet industry. If you find it hard to keep up daily, consider subscribing to our weekly digest. We will provide you a convenient summary report once a week sent directly to your inbox. It's a quick and easy read.

Related

Topics

Cybersecurity

Sponsored byVerisign

Domain Names

Sponsored byVerisign

DNS

Sponsored byDNIB.com

IPv4 Markets

Sponsored byIPv4.Global

New TLDs

Sponsored byRadix

DNS Security

Sponsored byWhoisXML API

Brand Protection

Sponsored byCSC

NordVPN Promotion