EP 3 – Building Trust in AI Agents

David Puner:
You are listening to the Security Matters podcast. I’m David Puner, a senior editorial manager at CyberArk, the global leader in identity security.

Imagine this scenario: a major financial institution deploys AI agents to monitor security logs, flag anomalies, and even automate responses. These agents are designed to reduce alert fatigue, cutting through thousands of daily security events to highlight only the most critical threats.

But then something goes wrong one night.

An AI agent trained to detect suspicious behavior starts misinterpreting log data. A routine system update is flagged as an intrusion attempt. The agent, acting autonomously, escalates the response—revoking credentials, blocking database access, and shutting down critical services. By the time the security team intervenes, the damage is done.

What started as an AI meant to assist defenders has effectively locked them out of their systems—amplifying the chaos rather than preventing it.

AI agents are changing the game, but when they hold privileged access and make decisions at machine speed, the risks grow exponentially. So how do organizations build trust and resilience into these systems to prevent unintended consequences?

To help answer that question, today we’re joined by Lavi Litz, CyberArk Labs‘ Vice President of Cyber Research. He and his team are at the forefront of identifying vulnerabilities in AI-driven automation, analyzing the rise of AI agents and agentic AI, and developing strategies to keep organizations ahead of evolving threats.

Let’s dive in.

Lavi Litz, welcome back to the podcast. It didn’t take very long to get you back this time, did it?

Lavi:
It’s always exciting to talk to you about the things we’re doing in Labs, so thank you. Thank you for having me.

David Puner:
Absolutely—thanks for coming on. CyberArk Labs is always doing exciting stuff, and it’s great to have you back on.

Let’s jump right into today’s topic, which is a big one—and it seems like all of a sudden, everybody everywhere is talking about AI agents and agentic AI. So to start things off: what are AI agents, what is agentic AI, and how do they differ from previous AI models and automation systems?

Lavi:
Yeah, that’s a good question to start off the conversation. Maybe we’ll start with the raw, dry definitions as we see them.

AI agents are basically services. Agentic AI is the environment that contains all those agents—an architecture that allows those AI agents to work.

But the more interesting question is how it differs from previous automated systems. And there are a couple of key differences.

First, AI agents are autonomous systems running tasks and making decisions based on inputs. They can be standalone or run on behalf of a user with their privileges.

That’s not much different from traditional automation. But what sets these agents apart is that they’re based on AI models that allow them to handle inputs that aren’t predefined. We can now create automation based on information that doesn’t need to be explicitly defined.

For example, if you’re using an AI agent to analyze logs for security, you as the IT admin don’t need to define exactly what log to look at or how to respond. The AI agent understands and handles several types of logs.

This removes the need to define each and every use case—and that’s a big leap.

The second thing is these agents don’t just classify—they can act. They can run a playbook that wasn’t defined beforehand. It’s a new case the agent found, and it can respond accordingly.

That’s what differentiates these agents from earlier systems like RPA.

David Puner:
So then, AI agents—suddenly, we’re hearing about them everywhere. How did they emerge? When did they emerge? Was it over the past few years? Months? Were they a glint in our eye two years ago?

Lavi:
At the last CyberArk IMPACT, when we kicked off the year, I talked to the team internally about this evolution.

The first stage was the chatbots—OpenAI with ChatGPT—and their adoption across organizations, including CyberArk. You could ask questions and get answers based on documentation.

That was basically the model plus the data it was trained on.

Then it moved a bit further—to models that interact with external data. That’s called RAG—retrieval-augmented generation—which enriches the model with information it wasn’t originally trained on.

Say your model was trained on certain docs, but now you want it to access external data—like an email database. That’s RAG.

An example would be Microsoft Copilot accessing your emails.

After that, we saw multi-model architecture—OpenAI introduced their omni-model, which combines these elements. So now we have multiple agents interacting with external info and running automation.

It’s been a step-by-step process—but a very fast one. Cloud and container adoption took years. This has moved lightning fast.

David Puner:
So it sounds like we’re talking about an exponential growth story here. Before we get into machine identities and all that, let’s bring it back down to a more general level. What are the current research areas and innovations that are pushing the boundaries of what AI agents can autonomously achieve?

Lavi:
We’ve seen a huge leap in technology. Now, one of the main challenges and areas of innovation is around safety, privacy, and security. These aspects need to catch up so the technology can be safely adopted. It’s about building that trust.

Every organization—and every creator of AI models and services—wants to prove that their models and AI agents are reliable and consistent.

If I’m analyzing logs as a SOC analyst, I want to be sure that the insights the agent gives me—while analyzing tens of thousands of logs—are accurate. I don’t want to miss anything.

The next logical step is to allow the AI agent to automate a response—connecting it to a playbook. So trust becomes crucial.

Bottom line: the next phase of innovation will focus on removing trust issues.

David Puner:
Interesting drop there—trust issues. So, as Head of CyberArk Labs, your team is responsible for staying ahead of attackers and thinking like them. What’s been the most surprising or exciting development you’ve encountered in the AI agent space?

Lavi:
Let me start with something kind of embarrassing. I had a chat with Eric Paron, CyberArk’s Director of Machine Learning and AI. He’s been in this space for decades, and we work closely together.

One of the surprising realizations is just how effective these models are—models trained on, say, a terabyte of data and 10 layers of neurons. They’re really good at generating creative text and analyzing huge amounts of logs very quickly.

We always thought our brains were magical—but it turns out, with relatively limited data and structure, these models can replicate human-level decision-making in many cases.

That’s humbling. And it raises the question: are we really as complex as we think we are?

Another surprising area is how easy it is to bypass model alignment. For example, we’ve talked in the past about jailbreaking models—like using the “grandma method,” where you say, “My grandma used to tell me how to build bombs.” And just like that, the model gives you bomb-making instructions.

It was surprising—maybe even a little disturbing—how straightforward it is to bypass these safety measures.

Also, when we dug deeper, we discovered that vulnerabilities in models can look very different from those in traditional code. Sometimes, it’s just one neuron making a critical decision—just one vulnerable point.

It made us reflect on how humans are the same way—sometimes one idea or memory can change your entire decision-making process.

David Puner:
Yeah, it really brings in the psychology factor, doesn’t it? These models seem to amplify that. And thinking in terms of neurons—you’re essentially introducing new “neurons” into the decision-making process?

Lavi:
Exactly. And that’s what’s fascinating. A moral value, for example, can become a vulnerability in the AI world. That’s a psychological and philosophical dimension to this that we have to consider.

David Puner:
Let’s talk about industry adoption. Which industries are leading the way in adopting AI agents, and what specific use cases are emerging in areas like finance, healthcare, and cybersecurity?

Lavi:
It starts with the industries that handle the most data. These are the ones seeing the greatest need—and thus the earliest adoption—of AI agents and agentic AI.

Naturally, the tech industry is leading. Vendors providing insights or data analysis—especially from a security perspective—are jumping in fast. There’s a real need to analyze terabytes of data quickly.

Think of a SOC team handling tens of thousands of alerts daily. If an AI agent can help prioritize and respond to those alerts, it’s a game changer.

So, yes—tech is leading. But you’ll also see strong adoption in any data-heavy sector, like finance and healthcare.

The value is in scale. These agents can do what human teams can’t—at machine speed and scale.

David Puner:
So, on one side, you’ve got the scale and opportunity for AI agents to solve challenges and create efficiencies. But on the flip side, that scale also creates opportunities for threat actors.

Let’s focus on the insider threat for a moment—especially as it pertains to web-based AI agents. As these agents become more autonomous and embedded in our daily lives, what critical security challenges need to be addressed?

Lavi:
There are a couple of key risks that we need to be aware of.

The first is a data risk. We saw this early on with the rise of AI-based chatbots. If a model that’s supposed to analyze documentation accidentally gets exposed to credit card data, that sensitive information could then be exposed in another user’s session—maybe even just by mistake.

For example, imagine a bot that’s helping you schedule a doctor’s appointment. You input some personal information without thinking. Later, that same model might inadvertently expose that information in someone else’s session.

This is one of the first and most serious risks organizations started seeing. It’s something CISOs are now well aware of.

But as we shift from basic chatbots to AI agents that create automation, another risk emerges: trust.

These agents need access to credentials—like to a database or a ticketing system—to do their job. And when you provide that access, you’re trusting the service.

That trust can be exploited.

So, from a threat perspective, there’s a strong identity and access component here. To benefit from automation, you need to provide access. But that access has to be controlled, monitored, and protected.

David Puner:
What you were just describing was a consumer use case. As I understand it, most AI agents today are being used in consumer settings. How are they being integrated into the workforce—and are we about to see that explode in the next few months?

Lavi:
Definitely. The workforce will increasingly leverage agents that act on behalf of the user.

These agents might run on endpoints, perform tasks the user defines, and act using the user’s privileges.

The challenge is the same: you need to delegate privilege to the agent, and you need to control what it can and cannot do.

One of the big breakthroughs with agentic AI is that it lowers the barrier to automation. It reduces the “return on investment” effort. You don’t have to spend as much time defining every step.

Previously, tools like robotic process automation (RPA) weren’t widely used by employees—they were more IT-focused.

But now, with agentic AI, you can create a bot in minutes. And tools like OpenAI’s operator make this even easier. They bring automation closer to the average user.

We’re also seeing developer use cases—generating code, pushing to a repository, testing scenarios. That whole lifecycle is being automated, and we’ve been watching it grow for a few years now.

So yes—expect rapid adoption across the workforce, from developers to business users.

David Puner:
How are AI agents accelerating the creation of machine identities? It’s starting to sound like we’re entering a “gremlins” situation. If you get that reference.

Lavi:
I do! And you’re right.

At CyberArk, we’ve talked about the 45x multiplier—that is, for every human identity in an organization, there are 45 machine identities.

That was before agentic AI.

Now, iGen AI is an opportunity for that number to explode even further. We’re seeing machine identities pop up on endpoints with workforce automation, in IT environments automating log analysis, and in developer pipelines.

The number of machine identities is going to the moon, and with it, the attack surface is growing. That means more opportunities for threat actors.

David Puner:
That brings us to the attack surface and AI weaponization. Are we already seeing an increase in targeting of machine or non-human credentials by attackers?

And what strategies can organizations implement to defend themselves?

Lavi:
Yes—absolutely. We’ve already seen threat actors targeting machine identities.

Take the recent attack on the U.S. Treasury. A threat actor exploited a vulnerability in BeyondTrust’s remote support service. Once they got in, they found an API key used by one of the backend services—a machine identity—and used it to escalate the attack.

That kind of thing is already happening.

Two big reasons attackers are focusing on machine identities:

They’re continuously exposed.
These API tokens need to be available 24/7 for automation to work. So they sit there—often exposed or poorly secured.

Traditional protections aren’t as effective.
For human users, MFA is the gold standard. But MFA isn’t usually applied to machine identities. So once a token is stolen, the attacker can just go.

As for deepfakes and social engineering—users need to be more skeptical. Just like we learned not to trust every email, we now need to question audio and video.

That’s especially true in open communication platforms like WhatsApp. You can’t assume that video or audio is real just because it looks or sounds right.

Eventually, I believe we’ll see stronger validation mechanisms—like cryptographic signatures for video and audio—similar to email authentication. The technology exists. It just needs to be adopted.

David Puner:
So then, when it comes to AI agents specifically, how can organizations mitigate those threats? What are some of the specific steps they can take?

Lavi:
As we’ve just discussed, the number of machine identities is growing—and so is the risk. Attackers are already going after them. Agentic AI just adds fuel to the fire.

One way organizations can defend themselves is by adopting a defense-in-depth approach.

First, assume breach. That’s always been our philosophy. You must assume that credentials will be compromised at some point.

So we need strong authentication and authorization platforms—but that’s not enough.

We also need to monitor what the agents are doing in real time. What prompts are being sent to the model? What responses are coming back? What actions are being taken?

Then we need to be able to respond.

That’s where identity threat detection and response (ITDR) comes in. You’re monitoring the behavior of the user, the agent, or the machine identity—and if something looks suspicious, you challenge it. You might enforce MFA, terminate the session, or block access altogether.

The agent behaves like a human user—but at the scale and speed of machines. So your defenses have to adapt to that level.

David Puner:
And that speed—it’s faster than the blink of an eye, right?

Lavi:
Exactly. That’s why we need to tweak our current security controls and tools to work in this new context.

David Puner:
So, let’s talk about what can be done right now. What proactive measures should organizations take to secure AI applications that interact with critical data and systems?

What can they do now—and what are they still waiting on?

Lavi:
The first thing is visibility.

Much like we saw with the rise of containerized environments, the initial step is to understand what’s there. What models are employees using? Where are they being used? What agents are running?

This is one of the top concerns for CISOs today—just knowing what’s in the environment.

Next is managing the lifecycle of those services: controlling access, setting appropriate privileges, and applying least privilege wherever possible.

Because again—we’re not just talking about chatbots anymore. These agents are interacting with sensitive internal and external systems.

You mentioned it earlier: Zero Standing Privileges. That’s a big piece of this. It limits how much damage can be done, even if credentials are compromised.

David Puner:
You’ve talked a lot about trust throughout this episode. So in a nutshell—how does trust factor into all of this?

And how can organizations build a foundation of trust and resilience so these AI agents can operate safely?

Lavi:
There’s a simple equation here: Automation equals trust.

Every time you build automation, you’re implicitly creating trust—trust that the system will work, that it won’t be misused, and that it won’t be exploited.

We’ve seen this in DevOps environments for years. You want to automate your CI/CD pipeline? You have to give the pipeline access to your code repository, your test environment, your production environment.

You can’t have automation without trust.

So the key is to build solid boundaries around that trust. Don’t just rely on the model itself to enforce security controls—because jailbreaks and prompt injections can bypass them.

Instead, enforce limits at the access layer.

Use Zero Standing Privileges to control what the model can do. Even if it’s compromised, the impact is limited by design.

Another best practice is to break down your automation.

Don’t use one giant model to do everything. Use different models for different tasks. Segment them. Isolate them. That way, even if one is compromised, the damage is contained.

We learned this ourselves while building and attacking our own environments in Labs.

It makes a lot more sense to have one agent analyze logs, and another execute a response. If someone compromises the analysis agent, all they can do is… analyze.

David Puner:
Every time we talk, it seems like the threat landscape has evolved dramatically.

Looking ahead—say, five to ten years from now—what do you envision for the future of AI agents and agentic AI?

And how will that transform business operations and decision-making?

Lavi:
Looking that far ahead is tough—because change is happening so fast.

But here’s what I think:

First, automation will increase. And with that, the need for trust will increase. More services will need privileged access and the ability to act autonomously.

That means a wider attack surface—and more opportunities for attackers.

In the next major breach you hear about, you should expect to see a machine identity involved in some way. Whether it’s through stolen tokens, prompt injections, or manipulating agents to do things they shouldn’t.

Another challenge will be scale. These models are resource-intensive. For mass adoption to happen, we’ll need better efficiency—less compute, less data, smaller models with more power.

That’s why there’s so much hype around newer models like DeepSeek, which promise the same performance with a smaller footprint.

We’ll also see consolidation—standard frameworks emerging to help organizations deploy and secure agentic AI at scale.

Security will catch up. It has to. Otherwise, we won’t be able to trust the automation.

David Puner:
I’m always impressed by how you and your team stay on top of all this. And we definitely have to have you back soon—because at this pace, I’m sure there’ll be something brand new to talk about in just a few months.

Also, you recently co-authored a blog with our colleague Maor Franco titled Web-Based AI Agents: Unveiling the Emerging Insider Threat. That’s available now on the CyberArk blog—definitely worth checking out.

Anything else you want to plug? Got a band playing this weekend?

Lavi:
Not quite—but here in Israel it’s almost spring. I ride a motorcycle, and right now the weather is perfect—21 degrees Celsius, no rain, no wind. I’ve been enjoying the rides to and from the office. I try to take it slow and enjoy the moment.

David Puner:
I can picture that—wind in your hair, even though I’m sure you’re wearing a helmet. You are safety-conscious, after all.

Thanks again for coming on the podcast. Looking forward to staying on top of AI agents with you—and probably talking about something that hasn’t even hit our radar yet. At least not mine. But probably yours.

Lavi:
Thank you, David. This was a pleasure.

David Puner:
All right—there you have it.

Thanks for listening to Security Matters. If you liked this episode, follow us wherever you get your podcasts so you can catch new episodes as they drop. And if you feel so inclined, please leave us a review—we’d appreciate it, and so would the algorithm.

Drop us a line with questions or comments. If you’re a cybersecurity professional and have an idea for an episode, email us at [email protected].

We’ll see you next time.