The cloud brought with it its fair share of security challenges, expanding the attack surface well beyond the static perimeter protections of the web application firewall, network IP-based rule, and infrastructure layer configurations.
The rapid emergence of AI has only exacerbated those challenges and “forced security into the runtime — the elusive attack surface that everyone knows they need to secure but that security technology has not been able to keep up with,” said Vrajesh Bhavsar, co-founder and CEO of Operant AI, the four-year-old startup whose runtime application platform aims to defend not only cloud applications but also AI models and APIs.
The San Francisco-based company’s platform includes such capabilities runtime risk scanning and analysis, enforcement of runtime security measures across APIs, data stores, legacy endpoints, and Kubernetes clusters, vulnerability scanning of APIs to smoke out vulnerabilities and generate real-time insights, and policy as code for scaling security policies across multiple clouds.
At the recent KubeCon + CloudNative NA show in Salt Lake City, Utah, Operant AI took another step, introducing the 3D Runtime Defense Suite, which pulls capabilities around real-time discovery, detection and defense into a single package to ensure protection at every layer of cloud applications, including AI models and APIs.
A Suite of Defenses
The enhanced discovery capabilities include instant live blueprints of AI workloads, models and AI APIs, identification of ghost APIs and shadow AI data flows, and tracking data-use patterns from third-party APIs to data stores.
Runtime detection targets Open Web Application Security Project’s (OWASP) top 10 large language model (LLMs) threats, including prompt injections, sensitive data exfiltration, model theft and data poisoning. Operant’s current platform already covers more than 80% of OWASP’s threats across APIs, Kubernetes and LLMs. Another feature is detecting the leakage of sensitive data like personally identifiable information (PII), secrets and API keys.
The defense section includes automated in-line block and redaction of sensitive data flows, quarantine for suspicious third-party containers and AI models, and enforcing rate limiting and token use for sensitive APIs, including endpoints.
Auto-Redacting of Sensitive Data
The automatic redaction of sensitive data flows is a key feature that addresses the issue of data privacy, both from the point of defense as well as data governance and compliance with the growing number of government regulations. AI development now is a mixed bag, with some companies stopping AI development over data privacy concerns while others build AI products and features that may be sharing private data with third parties, either by accident or design, Bhavsar told The New Stack.
“In today’s GenAI [generative AI] application world, companies are building all sorts of business-critical capabilities with their models,” he said. “But in order for AI to work, it needs really, really good data. So, how do we protect the data and get the capabilities we need without shutting down the entire AI application flow?”
Operant’s in-line auto-redaction detects common forms of private data — think Social Security numbers, phone numbers and API keys — as they move through the live application, flags them and prioritizes their risk profiles based on how critical they are.
“The system allows engineers the option to either shut down the whole data flow by … or to let it run with in-line auto-redaction protecting the private elements of the data,” Bhavsar said. “In-line auto-redaction redacts the private data before it leaves the internal application environment, which is a huge win both for data privacy and data governance because it means that the core sensitive elements are not able to be sent anywhere, while the application itself can continue to function.”
Moving Beyond Logging and eBPF
Operant isn’t the only vendor looking to protect the runtime environment. Other companies rely on tools like eBPF — which enables programs to run in a privileged context like the operating system kernel — and logging, which creates large numbers of alerts that software engineers have to deal with. While they warn developers of attacks, Operant’s technology takes steps to shut them down.
“These approaches cause a reactive overload for strapped teams while lacking the multidimensional context required to actually make sense of attack paths that in reality connect from external third parties, through APIs, through services, all the way to data stores,” Bhavsar said. “This was true before teams were building sophisticated GenAI products and features but is made even more exaggerated by the data flows and application architecture required to make GenAI work, whether a team is using an AI API or a third-party model deployed on Kubernetes, as most of them are.”
In addition, attacks at runtime — including prompt injections, zero-day vulnerabilities data exfiltration, data poisoning and distributed denial of services (DDoS) — need to be blocked at runtime, something these other runtime mechanisms can’t do.
AI application security tools also aren’t enough. AI applications don’t live in a vacuum, and organizations can secure just AI elements of a cloud application.
“You have to secure the entire cloud stack in which it is embedded, including all of the ephemeral and complex interactions happening within Kubernetes, which has become the de facto platform for AI application development,” he said. “To do that, you need to be securing the data an application or model is using before it leaves the perimeter.”
Security by Default Is Key
Operant ensures all API and Kubernetes interactions inside an application’s perimeter are secure before data is sent out, enabling AI software and models to provide business value without security concerns blocking their ability to function. The security-by-default mode lets teams develop AI faster and more responsibly with noninvasive runtime defense controls in place before attackers enter the environment.
Operant is looking to cast a wide net with its 3D Runtime Defense Suite, not only in terms of capabilities but also in its coverage of generative AI platforms like OpenAI’s GPT models, Google’s Gemini, Cohere and Anthropic’s Claude. Given the rapid evolution of the AI industry and the wide range of choices development teams are making, an effective AI defense product needs to be vendor- and platform-agnostic to cover changes and ensure resilience, Bhavsar said.
“We wanted to build the right plug and play architecture from the beginning so that adding another provider in the future or customizing the ones we support within customer stacks would remain simple like the rest of the implementation of our product,” he said. “This was a key strategic decision we made … so that it will allow security engineers and developer teams to incorporate security practices that will stand the test of time and scale without extra cost or heavy manual work as they continue to evolve their development processes and tooling.”
Building on Experience
Both Bhavsar and Priyanka Tembey, a co-founder of Operant and its CTO, each bring more than a decade of experience building machine learning and AI for cybersecurity use cases, which gave them a sense of which approaches were short-term measures and what can be transformational at scale.
“There is a ton of noise in the industry, but we are building to last with technology that is, by default, designed to scale and be flexible in a way that matches real engineering teams where they are, and also sets them up with effortless cyber-resilience for the future even as AI continuously evolves,” Bhavsar said.
They’re getting support for their ideas. The company, which launched publicly in April 2023, raised $10 million in capital venture funding in September from SineWave Ventures, Felicis, Alumni Ventures, Massive, Calm Ventures and Gaingels, giving it a total of $13.5 million raised in just over a year.
The post Kubernetes Runtime Defense Evolves Beyond eBPF appeared first on The New Stack.
Operant AI’s 3D Runtime Defense Suite gives developers an option beyond eBPF to stop threats in real time against inherently unpredictable LLMs.