Mani Masood: If we want to experiment with a high-risk technology, then that risk has to be shared. So someone else at the senior leadership would have to say, "Hey, we do understand that this is a high-risk project, high-risk product that we want to come up with. And we know that we can't do everything that they're asking us to do. Hence, the risk has to be shared between all stakeholders within this project."
Johnny Hand: Welcome to AI Security Brief, where we're unpacking emerging AI threats, vulnerability research, and the strategic decisions that security leaders are making right now. I'm Johnny Hand.
Dustin Childs: And I'm Dustin Childs. Let's say your organization sets a target of 200 AI agents, not just built, but deployed and safely sustained. What does it take to get there? In this episode, we talk with Mani Masood, VP of Information Security and CISO at AMN Healthcare, whose organization set that very aspirational goal. Mani's governance approach really held up for the first dozen agents, but then began to stretch. So what needs to change as deployment scales?
Johnny Hand: Exactly, Dustin. You may have measured success at the start of your deployment, but passing that initial review does not guarantee that agent will always behave as expected. So how do you govern systems whose behavior can change after approval?
Dustin Childs: As Mani tells us, you start with what the platform can enforce and assess the risk of the specific use case, and make sure business leaders share responsibility for the decisions. In this episode, we cover real lessons from putting an AI scaling mandate into action and what Mani would do differently next time. So let's dive right in.
Johnny Hand: Mani, I do want to ask, you know, we're hearing more and more scenarios where a security leader is actually handed an AI scaling target. And it's usually far above, like, internal expectations. Usually, you don't have a playbook to run by, and the teams usually have a fair amount of skepticism on whether that move is actually strategic or feasible. I understand that you found yourself in that very scenario this year. Can you tell us a little bit about that?
Mani Masood: Yeah. So, I mean, we're just like everyone else in this space. We're excited about what AI can bring to our organization and how it can help us boost our efficiency and our ex -- you know, clinicians' experience and customers' experience. But at the beginning of the year, we set an aspirational target for ourselves that we're going to build 200 AI agents, and not just build them, but also deploy them and sustain them in a safe and secure manner. How do we come up with 200? Not quite sure. It's like I said, it's an aspirational number, but we're -- you know, now that we are, you know, at the end of Q3 and end of September, I think we're well on our way as compared to where we were when we first set this target. But that -- and it was an interesting journey. And when we were starting out, we did not have a lot of the things that are now into play. For example, the capabilities of agentic AI has just changed exponentially in just the past eight to 10 months, I would say, right? From a simple agent with a built-in prompt to now an agent that has hooks and MCP and API and has the actual ability to learn and hold that in the memory, that learning experience as well. So, long journey.
Johnny Hand: Yeah. I have to ask because I think for a while we were talking about AI adoption, and this is much different than AI adoption. This is really scaling. What does it feel like to receive that mandate that is so big without having that existing playbook? And then, you know, what was -- what was it that your team was thinking about? Would they may have had a -- would it be more realistic if they would've -- if you would've started and had the choice of how many agents to start with, would that be a much smaller number, or do you feel optimistic about that large number?
Mani Masood: I mean, coming from information security, which is a conservative place, I would always say the same thing that I say whether it's AI or anything else. Let's just do one and see what it does, and then we'll worry about the next one and the few hundreds. But if you look at it in the larger history of information technology, even though there is no comparison to where we are today, it is truly transformational. But we have been in this place before, right? So the whole.com evolution where everyone was just going crazy, you know, take your name, stick a.com to it, and it's very important to get to the web. And then after that, we came with those application service provider era, and then followed that by, you know, virtualization and cloud. And so we've been through this, I would say, transformative phase where you set aspirational goals and they kind of, you know, go through that learning curve and taper out as things mature. So I personally took a lot of parallels from the cloud era where, you know, if you recall, once that exploded to the scene, the whole idea was, "Hey, let's see how fast we can get into the cloud and without actually architecting or, you know, building applications from grounds up to be suitable for cloud." The idea was to take everything to the cloud. And once you took everything to the cloud and you ran into similar challenges that I think we're seeing today, which is the cost factor. Cloud was not inexpensive in the initial stages of the adoption. So then you had to, many times, bring things back to on-prem because just of the resource requirements. And I think we're seeing similar things play out today where, "Hey, let's take everything and put AI layer on top of it and let AI process this." And it's as you see that things are progressing, the actual real cost of AI use is becoming more and more visible, right? Something that was invincible just 12 months ago, or hidden, if not invincible.
Johnny Hand: Mani, you know, I come from an IT background as well, especially in intrusion detection and information security, so skepticism is part of my DNA. And there's also real skepticism about these AI agents, whether they're doing genuine strategic thinking versus just sounding confident. How do you separate the two in practice?
Mani Masood: Oh, that's a million-dollar question, right? If you look at the technology at its core, the way it is designed, it is designed to be statistically in the middle, at average, right? So if the data that has been -- that it's building upon is skewed towards falsehood more than accuracy, it's going to pivot to that average mark, and the information that you're going to get is not going to be very reliable. So that's problem number one, and that's where what models you're using, who are the builders and the providers and the keepers of those models, matter. To me, it was a major surprise that 60% of the token use is directed towards the Chinese models. Now, I'm not saying that the Chinese models are bad or there's something wrong with them, but again, it goes back to that if you have to certify the accuracy of your information, do you have that entire supply chain certified, right? Same principles, different technology. You know, you can't let go of the ancient IT adage, "Garbage in, garbage out," right? It applies here as well.
Johnny Hand: So let's talk about the risks that might have been unbudgeted. So, like, six months ago, the worry was all about adoption speed. Now, it's about token cost, credential theft, and identity sprawl across agents. What changed and what should leaders be watching for?
Mani Masood: Well, part of it's the same problem. It's just the scale that it is operating on. So if you look at, for example, stolen API problems, that has existed, you know, for the better part of the internet since data interchange on internet became available. So it's not a new threat, but the scale of that threat is just unbelievable. And I did read a report, I believe it was a couple of weeks ago, where there's now an aftermarket very similar to the Silk Road and the other stuff that had existed on the dark web where stolen API keys are being marketed, and the consumers on the other end are not only individuals. They're nation-state actors, and it's always very difficult to deal with nation-state actors because, regardless of how good your organization is, you cannot go up against, you know, well-trained -- a well-funded, government-backed outfit trying to come against you. So while the nature of the problem is the same, the magnitude of the problem is much bigger. And these now keys are being sold on this aftermarket. A lot of people have always had this problem on "Where are my API keys? Where are they being used?" And some of these API keys are being used for legitimate uses, and they're being used for nefarious purposes as well. And there exists a router mechanism, an open-source router mechanism that's being marketed very openly in these dark web marketplaces, that instead of stealing one company's API key and exploiting it, you take a thousand keys and then you spread your load across those thousand keys. So it makes it even more difficult for the, you know, for the blue team, for the defenders to find out if something is going wrong with the API keys. So that's what changed, right? The magnitude, the market suddenly where nations that were banned from obtaining certain technologies entered that marketplace, the volume at which the theft is happening. Insider threat has become a major dimension into this landscape. If you go back couple of years and you look at Verizon's data breach report, if you look at, you know, AT&T's or IBM's cybersecurity outlook, you would see that insider threat was in -- not even in the top 10, and then it broke into top 10 about five years ago at the height of the COVID. And at that time, it was just people trying to do multiple jobs and things that could go wrong if you were working for two competitors. And from there, that morphed into where we are today, and they're saying it's going to be the number one threat vector in the next three to five years, and it's already breaking into top five. So part of it is malicious. Part of it is accidental. I mean, you know, more damage has always been done by human beings with the best of intentions than with the worst of intentions. So that also plays into this. You know, you have 100 keys, plain text, you store them, then you send your laptop for repair, and then hard disk gets changed, and then the hard disk gets given to someone else. And before you know it, some person who should not have access to those keys has access. So it is a compounding problem. I don't think there is a solution out there that's a silver bullet that will help us solve this. I do know that there are organizations that are working in this space trying to look at the threat vectors, TrendAI being one of them. But it's a evolving problem. And who knows what's the next phase that it will take.
Johnny Hand: Yeah, going back to those markets. It's hard to outbid someone who can print money. So yeah, I totally understand what you're saying there with the governments and the state agents.
Mani Masood: Yep. Yep.
Johnny Hand: And I have to ask, you know, I'm sure that you have some hard-won wisdom from just doing this project and setting up the governance and scaling agents to the scale that you guys have. So, for our listeners, if you were handed a mandate tomorrow that you need to scale this 200 by 5x to a thousand, what are three things that you would do differently starting tomorrow than you did maybe a year ago when you started this project?
Mani Masood: Oh, that's a tough question. I would definitely say that we focused on the words -- the what's allowed, what's not allowed in terms of the language of the governance. In hindsight, I would say we should start from focusing on the platforms because that's where the capability of implementing what you're writing would exist or would not exist. So if I could go back, the start of the conversation would be from, "Hey, let's take a look at the platform." Instead of saying, "Hey, these are the good practices that NIST CSF is saying, so let's take those and convert them for AI governance. Let's keep that on the side. Let's start with the platform where this technology is going to exist." What are the platforms where we're going to operate? And try to solve for that and come up for a framework in that space. So if you are, for example, an organization that exists in the AWS sphere versus an organization that exists in the GCP sphere versus an organization that exists in Microsoft Azure cloud, if you write certain policies, especially going back couple of years, if you write those policies, if you write those frameworks, you may not be able to apply them universally in these three environments. But if you started from the environment and its capabilities, I think you would have a better understanding of, hey, what I can and cannot do, and hence what I can apply in terms of a framework or governance policy. The other thing I would say to look at is focus on risk, not the threat vectors. So when we started out, we said, "Okay. AI is coming, is going to interact with the data. So let's look at the threat vectors associated with the data." In hindsight, which we are doing now as of today, bring risk front and center. Why would it matter if you are working with a certain set of data, if there is a GPT on top of it, versus if, you know, you were uploading it into a data lake, for example, right? And if the profile is the same, that hey, it's only internal, and we have identity and access management to a certain degree, and that is satisfactory, then your risk profile is different. So you don't have to apply all of your controls because it's a different risk profile. So these are at least two things that I would do differently. That, hey, focus on the -- start from the platform, then work your way towards the governance policy. And in your governance policy, structure it around a risk profile, not a control profile.
Johnny Hand: That kind of leads directly into my question, which is, what's the one governance or security lesson from this whole process that you've identified that you want security leaders to take away from this conversation?
Mani Masood: So it's a evolving situation, right? And the one thing I would say -- and it's not just for the security leaders, but to the management structure that the security leaders report to. By nature, information security is conservative, risk-averse, and they are concerned about the consequences when things go bad. Now, things don't go bad every day, but when they do, that's a really bad day for everyone, right? And we have seen examples where, you know, established hotel, global hotel franchises were impacted not because of AI-based breaches, but traditional cybersecurity breaches, and the entire business collapsed to a point where they had to be, you know, acquired by a much smaller hospitality player. We have seen what happens to airlines when they get cyberattacked. We've seen infrastructure. So we see this, and these are deep scars that we carry within the information security world, and it's a very tight community. And those experiences are shared, and they're shared over and over again. So by nature, we are very worried of the consequences of risks that we would take. So the one thing that I would say is that if the risk profile is ultimately going to be entirely the responsibility of the information security teams or the CISO or the leaders within the information security arena, then you end up in a scenario where you have conflicting targets, right? You have a developer team who wants to build and deploy, and then you have an information security team whose KPIs are, "Hey, we can't get hacked this year," right? But if we want to experiment with a high-risk technology, then that risk has to be shared. And again, the legal framework does not exist for this, right? So someone else at the senior leadership would have to say, "Hey, we do understand that this is a high-risk operation, high-risk project, high-risk product that we want to come up with, and we're not going to engage information security for the sake of just, you know, having them here. But at the same time, we know that we can't do everything that they're asking us to do. Hence, the risk has to be shared between all stakeholders within this project." I think that will bring more comfort, and it will reduce some of the challenges that people see day to day in dealing with information security. Because in our minds, we're constantly seeing -- playing that worst-case scenario that we have dealt with in the past, now multiplied with AI as the adversary on the other end. And that can give ulcers to anyone.
Johnny Hand: So, Mani, I have to ask, if you had to bet on what the biggest AI security surprise for 2027 could be, what would that bet be?
Mani Masood: Oh, I can tell you what my fear would be. My fear increasingly is that it is going to be biological more so than electrical or technical. And we are seeing things move in that space. And if you see what OpenAI and Anthropic published very recently in terms of their own concerns in this space, but within 24 hours they announced that they're going to be building a fully AI-managed bio -- you know, bio lab, right? So these two things don't connect. Your actions and what you were saying don't add up. If I was to bet money on the short term, I would say nation-state actors unleashing a multidimensional attack on a target, which would involve a physical or kinetic element as well, which is something that we have not seen in information security.
Johnny Hand: Mani, thanks so much for having a great conversation with us today. We really appreciate you sharing your insights on scaling agentic AI safely and governance, and we can't wait until you join us again on the show in the future.
Mani Masood: Thank you very much for having me. It was a pleasure.
Johnny Hand: What a great conversation. Mani really brings governance into practice, and it makes sense that you begin by understanding the platform and its capabilities before you translate policy into requirements. The policy really only helps if the environment supports implementing it. So before you expand your deployment, check whether the controls your governance requires can actually be enforced by the platforms you're using.
Dustin Childs: Yeah, he also highlighted that you need to return to the agents that worked initially and then stopped delivering the same value. You have to keep checking whether agents still perform their intended task as data, dependencies, and workflows change.
Johnny Hand: And you have to match your governance to your use case. Mani emphasized structuring governance around the business risk profile you have. This means that you have to view a specific agent use case and ask whether controls reflect the potential business impact.
Dustin Childs: This is about navigating the tension between deployment targets and security accountability. Mani's message for organizations is to establish who accepts the risk of a deployment and how that decision is shared across the stakeholders. And truthfully, it should include stakeholders across the organization, executives, development, security, legal, and more.
Johnny Hand: So for our listeners, here's a question to leave you with. If your agent count increased fivefold tomorrow, which part of your governance process would struggle first?
Dustin Childs: Thank you to Mani for sharing his hard-won wisdom on scaling agentic AI safely. To connect with Mani and to learn more about governance frameworks that work in practice, see our show notes.
Johnny Hand: And that does it for another episode of AI Security Brief. We want to thank you for joining us. And as always, our goal is to host conversations that get you thinking differently about security. And if it does, consider subscribing so you don't miss what's next.
Dustin Childs: AI Security Brief is mixed and produced by Elliott Peltzman, with original music by Amneajynx. Our show producers are Melany Gallant and Alyssa Gonneau. Our executive producer is Jennifer Eiben, with content strategy by Ma'ayan Plaut. Additional production help by Liz Stokes. Video editing by Sarelle Joppy and Brigitte Criqui Wild. We'd love to hear what you think. If you have feedback, a guest suggestion, or a topic you'd like us to explore, email aisecuritybrief@n2k.com.
Johnny Hand: Thank you so much for listening, and we'll see you next time on AI Security Brief. [ Music ]