Jon Oltsik: The two extremes are this is hype, this is nonsense, or this will solve all my problems. Move to the middle, it will solve your problem. There is a lot of hype around that. How do you get the most benefit while managing those two extremes? [ Music ]
Johnny Hand: Welcome to "AI Security Brief," where we're unpacking emerging AI threats, vulnerability research, and the strategic decisions security leaders are making right now. I'm Johnny Hand.
Dustin Childs: And I'm Dustin Childs. We've shown how Agentic AI agents can certainly take on more security work, but a few successful actions does not prove that they are ready to handle everything. Today's episode, we talk with Jon Oltsik, cybersecurity researcher and community activist who has studied security operations since well before the Agentic SOC conversation began. Jon sets up a clear distinction between automating processes and removing human judgment altogether from the SOC.
Johnny Hand: Yes, it was a really great conversation with Jon, and we talk about the concerns security leaders are talking about often, what an Agentic SOC means for our workforce. And Jon has some incredible insights into whether we can really expect more automation, which SOC jobs could actually disappear, and how humans remain essential for the future of agentic security operations. We also dive into establishing pilot SOC programs and why successful early results might actually lead to complacency.
Dustin Childs: Yes, I found the conversation to be really interesting and full of practical advice. So, let's go ahead and dive right in. Well, Jon, welcome to the "AI Security Brief." We're very excited to have you with us.
Jon Oltsik: Thank you. It's great to be here.
Johnny Hand: So, I think we'll just jump right in. Is an autonomous SOC or an Agentic SOC becoming unavoidable, or is it still an option?
Jon Oltsik: Oh, I think it's unavoidable, and I think that's a good thing. We've got a history of patching together the SOC as best we can using lots of different tools, using a lot of manual processes. We have to have different sets of skills, and ultimately, we depend on some highly skilled people. So, if AI agents can help us advance those kinds of things, make it -- make the SOC a more stratified platform, make the tools more coordinated, it's a really good thing. But to your point, if the bad guys are using AI, then we need AI to defend ourselves, and so it is absolutely inevitable.
Dustin Childs: Yes, and I understand that you've been researching AI or Agentic SOCs for a long time. So, what do you think is the biggest misconception about what an Agentic SOC can actually do today?
Jon Oltsik: Sure, well, in -- in fact, I've been researching SOCs for a long time, well before Agentic. I came up with an acronym SOAPA, Security Operations and Analytics Platform Architecture, back in 2016 when I was working at ESG. But to answer your question, I think the biggest, well, there's a lot of hype, there's a lot of misconceptions, but I'd say the biggest one is the difference between process automation and autonomy. So, there's this narrative in the market that suggests that the Agentic SOC will make the SOC autonomous. So very little, if any, human interaction, and that's just nonsense. In fact, I think it's self-defeating because we need humans in the loop to orchestrate the SOC, to really get the tools in place, get the APIs in place, the data structure in place to make the Agentic SOC effective. And we need humans in the loop just for judgment purposes. So, that is that narrative of autonomy is really doing the industry and security professionals a disservice. I hate it, but there's so much money in the market. There's so many vendors. They have to try to differentiate themselves. I understand. I'm not happy about it, but I understand.
Dustin Childs: Yes, that's really interesting. I was going to ask how you distinguish an Agentic SOC from an autonomous SOC, but from what you just said, it sounds like you don't believe in such a thing as an autonomous SOC.
Jon Oltsik: Well, not in 2026, I don't. I think we'll never get there, but we'll automate more and more, and we'll connect more tools, more data. We'll start to connect SOC to SOC over time. And so, the intelligence level increases, the accuracy of our actions and the speed of our actions increase, but we're still talking about business systems, and so to preserve them, we need human judgment. And so, there'll always be humans in the loop. There'll always be humans training the models and orchestrating or orchestrating the agents. But we will get more and more autonomy over time.
Dustin Childs: And in your vision, what happens to SIM and SOAR?
Jon Oltsik: They've become part of the autonomous or the agentic SOC infrastructure. SOAR is like hard-coded SOAR, brittle SOAR, script-based SOAR. That's gone. Because as you know, we would come up -- we would need people to understand the process. We would need people to program in the process the -- or to -- for automation. And then one subtle change would ruin the whole thing. And subtle changes are the nature of cybersecurity. It's the nature of the SOC. So, that goes away. The SIM, you know, it -- we still need a system of record. And the SIM was especially good at that for compliance reasons. But the thought of a central SIM being the brain and the storage unit and the compliance engine, no, that goes away. That gets distributed to lots of different systems, and really the agents and MCP and APIs bring it all together.
Johnny Hand: Yes, it's a good point because you think about like the concept of a SIM was really to be human readable for us to alert on, and we really are talking about translating back into a data lake that you know, agents and you know models can look at and interact with.
Jon Oltsik: Yes, absolutely. And the thought of centralizing all your data. So, first of all, that hasn't worked in years because there's too much data. And then you've got things like EDR and NDR and threat intelligence and thinking about putting that all in one place just never made sense financially or from a technology or a latency perspective. And so, that becomes more distributed. And the -- I mean things like event correlation. Well, that's what agents will do that based on intent, based on looping, so the reasoning loops. So, SIM. Now, there's an evolution there, but I think SIM as a concept goes away into a more -- into an architecture which again, I like to call SOAPA or SOAPA on AI, but some type of architectural SOC.
Dustin Childs: That's interesting. So, as agents take on more and more triage and investigation within a SOC, which SOC roles do you expect to change and which could disappear altogether?
Jon Oltsik: Yes, I think the industry is pretty accurate on the whole level one, level two analyst roles becoming more and more agentically based, especially level one type analyst or tier one type analysts. Doing eyes on glass triage, trying to figure out which alerts to escalate, that's going to go away and that's in a -- if it hasn't gone away yet, it's certainly soon to do so. Investigations will also become more agentic based. Now, we need to train those systems. We need to make sure that they're accurate. We need to tune them all the time, but that will also, not go away, but it'll become -- will become more training the agents versus doing the investigation, swivel chairing, all that kind of stuff. But there'll be new opportunities too, and I've written about that. I posted an article on CSO Online about new opportunities and cybersecurity professionals should pay attention because yes, there will be jobs that are eliminated, but there'll be new jobs created too. And so, if I were a hands-on keyboard cybersecurity specialist, I'd be looking for those opportunities because, especially in the short term, they'll be extremely lucrative.
Dustin Childs: Yes, what's one task in the SOC that you would not hand to an agent without human approval today?
Jon Oltsik: Good one. Good question. Well, two that I can think of off the top of my head. One is again, understanding the business context. So before, and maybe this is in an incident response type of environment, but before an agent took an action like taking a system offline, putting in a new firewall rule, stopping a process or halting a process on an endpoint, we need to know the business context. Is that the CEO's endpoint? Is the CEO especially protected with things like zero trust? Is this a critical system like a manufacturing system that can't go down? So, we need that human judgment. And the way I always -- I talk to a lot of people about these things. Some people less technical than others. And the analogy I always use is, we'll get to a point where I can ask an LLM to summarize -- look at every recipe for a certain dish, pick out the best ones, pick out the best ingredients, and then order those ingredients from the supermarket. And then sometime in the future, I'll have robots that can do the chopping of vegetables and the blending of sauces and things. But there's still going to be a chef that tastes the final product and maybe adds some salt or adds some spices based on where they are and what the local tastes are. And so, those things will persist in cybersecurity, too. So, you'll need the humans to understand the nuances of the business, the nuances of risk tolerance, and make the associated adjustments. And the other role that I wouldn't have is I don't want agents tuning the system. So, I don't want self-tuning. We need humans in the loop for that as well.
Johnny Hand: It's interesting because we have a lot of conversation obviously because of the marketing hype and really just the rapid maturing is probably a good word of organizations and their leverage of AI, especially in the security operations space, and I think there's -- it sounds like what you're saying is there's a very real need for high human context within that SOC. So, If you're trying to do that work where you're trying to offload that work of you know, to agents for that tier one work and maybe some cases maybe some tier two work, how do you still develop that next generation of those tier three analysts or threat hunters that have high human context, not just on the organization, but really know what needs to happen and also like you know what stands out, what's weird, what you know, what you got to pay attention to, like how do we facilitate that? Have you thought about that?
Jon Oltsik: Oh, a lot. I've done a lot of work with ISSA, the Information Systems Security Agency over the years, and it's been about skills development, career development, things like that. So, there's a couple of things. One is that at first, when these models, when Agentic SOCs come in, we're going to run them in shadow mode. So, take your people, have them watch what's happening. Have the senior people quiz them on was that a good action or was that a bad action? Why did it take this action? What does that mean in the overall context of an investigation or a threat? So, things like that. I think we'll also move to really smart flight simulators, as you have in the aviation industry. So, there'll be junior people watching what happens, being given scenarios, giving being given the keys to the kingdom to train the models, albeit in a synthetic perspective, and just being able to learn by watching more than learn by doing. And let's face it, the current or at least the historical way that we did this was to burn people out. So, there's a high degree of burnout for tier one analysts. What I've read is they've got about 45 seconds to make a decision as to whether to escalate an alert or ignore it. They tend to ignore at best like, you know, the industry data suggests that they ignore anywhere from 65 to 75 to 80 percent of alerts. So, that's inefficient too. So, I think we'll do a lot of simulation. We'll do a lot of work on training. We'll do a lot more work on data engineering. So, there will be paths. There'll just be different paths.
Johnny Hand: Yes, I think it's an interesting one because we even for our approach and I've been doing cybersecurity for many years and I hear a lot about the security operations center of the future and I always think to keep that high, you know, high human centric context I want to train -- I want to try to train them through those positions we want to replace so that they understand the position in depth. So, that you know, train them through that level one position and maybe at that level two position they become the quality control for the agents that are doing that level one triage. And that sounds like what -- the model that you're mentioning there, right? Do you see any changes in the industry where you expect that there's new roles are emerging and maybe if there's listeners out here saying "Hey, how do I transform myself?" What are those skills those analysts should look at building now to be able to transform into this Agentic era?
Jon Oltsik: Well, we talked a little bit about that, Johnny, and a couple of thoughts. And again, I think we'll link my article from CSO Online. But security data engineer is a really important -- not necessarily a new role, but I'd say a more important role than it has been in the past. As we know, garbage in, garbage out in terms of data. And one of the dirty little secrets of this Agentic SOC era is that when you do -- let's say you hire or you contract with an Agentic SOC vendor and they're going to sit above your SIM and your EDR, the first thing they do is spend weeks or months looking at your data pipeline, making sure that the logs are clean, making sure that they have a common schema or that they can read the data. So, a security data engineering position will become more and more important. Model trainers and orchestrators will be really important. And then again, to know what to do, I think you really have to understand the threats and the kill chain that an adversary would use, so that you can program, or at least give the agents intent, on what to reason about. Security red teamers. So, this is a general thing, but I think we need red teamers to attack, not just security like Agentic SOCs, but all AI-based applications that we develop. So, those are just a few. I know I mentioned a few again, and I give details in the article.
Dustin Childs: You know, as a former tier one SOC analyst, you are absolutely correct about burnout. It is a very real thing. But my question is, you kind of touched on it just a little bit in your last answer, but I want to kind of dig in a little bit more. So, before organizations can really trust agents to defend the enterprise, how do they defend the Agentic SOC itself, especially when agents can access telemetry, identities, and response tools?
Jon Oltsik: Yes, I think the first thing that you have to do is establish and this should be again enterprise-wide, but you have to establish the right governance and policies. So, and then of course the guardrails and the monitoring follow. So, agents need identities. Those identities should be clear. And we should establish zero trust for agents. We need boundaries on what the agents can do, and then boundaries on things like API access. We need to make sure that our MCP servers are configured correctly. We need to make sure that whatever we -- whatever technologies we end up using, we have to make sure that there is an audit trail associated with what we're doing that we know why agents did what they did and took the actions that they did. And then, of course, we're going to need behavioral monitoring. So, UBA -- UEBA is old and new again. But now, we're going to look at agent behavior. So, the thing is that these models or Agentic AI increases the attack surface and provides adversaries with some novel types of attacks. And so, we have to be cognizant of those attack patterns and monitor our behavior accordingly. And then we also need some kind of a you know, a shutdown apparatus so that if we do see things going a little bit haywire that we can shut things down and really examine -- do some forensic analysis of what went wrong.
Dustin Childs: Got you. So, I want to ask if we have a security leader that's listening today and they're wrangling with this conversation of, "Well, how do I even move towards an Agentic SOC?" Right? Like what is -- what are my first steps? What are the first things in a greenfield you know, conversation, what would be your recommendation to those security leaders?
Jon Oltsik: Yes, that's a good question and boy, was that question prevalent in -- at RSA and Black Hat. But I do see evolution there, even from RSA to Black Hat. Well, the first thing we've already talked about is get your data in line. So, what data is important? How can you access that data? Is that data in a format that's understandable and readable? Because if it's not, you're going to have lots of hallucinations. The second thing you would do is start in shadow mode. We talked about that, too. So, let's do this in a pilot project. Let's watch what happens. Let's get comfortable with it. Let's understand what we need to do to train it and improve it. So, then once we do implement it, we want to focus on training. We want to focus on orchestration, so we want to understand again what it's doing and then start to build the model accordingly. Nothing out of the box is going to be perfect, and especially as we think about different vertical industry requirements. So, a manufacturing organization is going to have different threats and different risk tolerance and different processes than a healthcare organization, for example. So, those things will have to be customized. And then finally, we want to bring our staff along. We want to make sure that as we go through this process, we are transitioning our staff, we're educating and training our staff, we're filling in any skills gaps. So, that's really what I would suggest, and think about it as supplemental versus any kind of a rip and replace. Maybe over time, we will change our technology stack, but for the -- in the short term, what we really want to do is take advantage of this really rapidly moving, evolving new technology, but have it be additive to what we do. We just -- there's just too many things to change to do things all at once.
Johnny Hand: So, for our listeners, based on what we talked about today, is there one thing that you want our audience to think about or maybe do after they hear this conversation?
Jon Oltsik: Yes, I want them to plan before they react. There's so much money going into this. There's so much hype as we've discussed that there's a notion that this is a panacea. Now, I will say security people are paid to be paranoid. And so, that's one behavior. But figure out how this will help and how it will help in an evolutionary or phased approach, and then move forward intelligently. So, on you know, the two extremes are this is hype, this is nonsense, or this will solve all my problems. Move to the middle. It will solve your problem. There is a lot of hype around that. How do you get the most benefit while managing those two extremes?
Johnny Hand: Well, Jon, we appreciate the conversation. I especially appreciate the kind of honest and ground truth of what's happening today within the Agentic SOC. Thank you for joining us on the "AI Security Brief" and we look forward to talking to you in the future.
Jon Oltsik: Thank you, guys. It was a pleasure and I'm happy to come back anytime.
Johnny Hand: Wow, Dustin, what a fantastic conversation. I really appreciated that Jon said the honest part out loud that we still need good data. If you automate an unreliable workflow, it's just going to repeat its problems faster with AI. And even with the Agentic SOC, vendors are often spending weeks or months cleaning up data pipelines before their agents are actually very useful. So, if you're starting this journey today, please choose one workflow, check the data it relies on, and make sure that that process is sound before you start piloting automation.
Dustin Childs: You know, I really appreciated John's warning about agentic AI complacency. Just because you get several correct actions, you could be tempted to just let the thing go and stop questioning the system. I think his takeaways are fantastic, which is first define which actions need human approval, and give agents clear identities and access boundaries, and keep audit trails. My kingdom for audit trails, behavior monitoring, and most importantly, a way to turn them off.
Johnny Hand: Yes, we also jumped into something that we're not talking about a lot in the AI era, and that's the idea of doing a real pilot. So, I love that recommendation of just bringing your staff into a real pilot, making sure that you have your experienced analysts that are helping the junior colleagues evaluate why an agent actually took that action, and then whether it was appropriate and what it means for the investigation.
Dustin Childs: Yes, that's always part of that plan before you react. I mean, I always say, plan the work and then work the plan. I would like to leave our listeners with a question. Before your next agent gets more responsibility, can your team explain what it can do, why it took an action, and when a human needs to intervene? You better. Our thanks to Jon for sharing his insight, and you can see the Show Notes for resources related to this episode and how to find out more about his work with Agentic SOCs.
Johnny Hand: And that does it for another episode of "AI Security Brief." We want to thank you for joining us. Our goal, as always, is to host conversations that get you thinking differently about security, and if it does consider subscribing so you don't miss what's next.
Dustin Childs: "AI Security Brief" is mixed and produced by Elliott Peltzman with original music by Amneajynx. Our show producers are Melany Gallant and Alyssa Gonneau. Our executive producer is Jennifer Eiben with content strategy by Ma'ayan Plaut, additional production help by Liz Stokes, video editing by Sarelle Joppy and Brigitte Criqui-Wild. We'd love to hear what you think. If you have feedback, a guest suggestion, or a topic you'd like us to explore, you can email us at aisecuritybrief@n2k.com.
Johnny Hand: Thank you so much for listening, and we'll see you next time on "AI Security Brief." [ Music ]
