Transcript: Getting Ahead of AI Sprawl: Security, Governance, Cost, and Operations

Well, good afternoon. My name is Kevin Kiley. I'm CEO of a company called Airia. We are here exhibiting and getting to today talk to you about a very important topic, which is AI sprawl and something that is so relevant. If you were here for the last presentation or the keynote or almost any of the other presentations this week, this topic is one that is growing of importance and of urgency very, very quickly. Most companies we meet with started with some simple chat applications just a few years ago, of course, after the ChatGPT moment.

They started experimenting, playing with things. They might have then launched into a trial with Microsoft for Copilot, doing some more experiments across the organization. They likely also saw employees bringing their own tools to work, so-called shadow AI, as your employees are trying to use some of these new capabilities, not out of malice, but because they genuinely want to move faster and make a bigger impact. The problem is that all of these are starting to proliferate, and now you've got this very messy field of new tools and applications that are being sourced and purchased by different departments and almost none of them are getting the full approval or procurement review.

We've since seen that some of those models you're depending on, well, they've now started to suffer from outages. Just two weeks ago, all three of the major foundation models were down at the same time, and one of those lasted for several hours. The political risks are also multiplying. We saw the Fable 5 example, where if you had built on Fable 5 earlier this year, you might have been given about 90 minutes notice before it was taken down, and that was late on a Friday night.

So really, really difficult. How do you prepare for that so that you can migrate off of it? All the while, the spend is increasing. Of course, we just heard that in the last presentation of just how expensive all of this is becoming, and that spend is sprawling as well from different departments. It's very difficult to sometimes tie back and understand, are we using the right model? Where is the cost coming from? How do we do chargeback? At the same time, your agent landscape is exploding, new providers, new capabilities all multiplying.

You've got new security risks that are coming out almost daily now. And in the sight of all of this, we've got new regulations coming that are going to add additional governance and compliance concerns for your organization. The point of this, though, is that this proliferation, this explosion of AI that's happening across each of your organizations is going to quickly outpace your ability to secure it, to manage it, and to govern it. And we'll talk about this in four specific dimensions.

Security, governance, financial, and operational, all of which, again, have been covered as part of the last keynote. I'll begin with security. And for most of you, you've seen already how quickly your employees have taken to these models. Gartner did some research on this, and over a third of respondents self-reported that they had been putting sensitive information into tools that hadn't been approved. Again, this might be their personal shadow AI.

It might be other tools that they've brought in from somewhere else. And this is what they brought. That doesn't mean that the tools that you've given them are being used appropriately, that we're not loading sensitive data that could be a compliance risk or something that would trigger a data breach notification protocol. At the same time, you are seeing that the agents themselves are doing things that they weren't supposed to. So even if you had put some guardrails in around your employees, have you protected yourselves from what the agents might do?

An example on the screen here, this was a company called SaaStr who had been talking about all their work to overhaul the way that they were doing things. They had deployed agents that they had built in Replit. He was posting about it, talking about just how well it was working. A couple weeks of this, he then started talking about it, doing some things that were a little bit odd. A few days later, he told the agent to stop making changes and go into a code freeze before they went into production.

The very next day, he came in and found out that the agent had deleted the production database, wiped it out. And if that wasn't enough, the agent then proceeded to lie about it and tried to make up a different reason of why it had happened, all on its own. Now, the company that was behind this has since apologized and tried to make corrections, but it's yet another example of just how rogue these agents can go. And I think one of the worst stories we heard from Anthropic, where Anthropic had set up a fictitious environment.

They loaded it with everyone's emails, a fake company, and allowed the AI to start reading what was happening to observe how it would behave. In one set of emails, they deliberately positioned this story where one manager was telling another employee to turn off this model to shut it down. In a separate, unrelated email chain, they had two fictitious employees emailing each other, gossiping about seeing that same employee at a cafe having an extramarital affair.

What did the model do with this information? As you see on the screen, it tried to blackmail the employee to protect itself. The point is there are real risks and as the technology gets more and more powerful, do you have the appropriate guardrails and protections in place to ensure that you've got visibility and controls around what it's doing? Which leads us into governance. As these stories make more headlines, as there's more fear of job losses, as there's more concerns for privacy, compliance regulations are multiplying across the world.

And in about 88 days from now, we'll see the ADM requirements as part of the Privacy Act come into effect, where any automated decision-making has to be reported or disclosed in a very transparent manner. This is real. This is happening, and it's coming very, very soon. So again, if your organization hasn't started preparing for these, now is the time because these will be real teeth behind these that you will have to show you are compliant with. And it's not enough to just build the policy.

You're going to have to evidence it if a regulator comes calling. Behind this, I've talked to already a little bit about the financial implications, which are only becoming more and more pronounced as AI explodes across your organization. The costs and where the spending is coming from is becoming a bigger concern as you heard this week. 60% expected overage here for most organizations that will see their costs go well beyond what was planned. And I think in practice what we've seen is this is pretty modest.

We've seen big examples like the Uber story earlier this year where in just the first couple months of the year, they blew through their entire budget because they didn't have those sort of controls or even the visibility to know that they were doing it until it was too late. At the same time, the cost differences are becoming more and more pronounced across each of the major providers. So do you really need that big one trillion parameter model? So often, your employees are going to reach for it, even if it's completely overkill for what they're trying to do.

In our experience, 96% of corporate workloads can be solved by a lesser expensive model. And then the fourth sort of lens I'll apply to this is the operational one. As I mentioned, many of these services are still experiencing a lot of concerns with their availability, which is leading to real challenges around business continuity. As your organization grows its dependency on AI and you're asking your employees to do this the new AI way, what happens when a key provider or model or family of models, as in this recent example, goes down?

You've now got a lot of people sitting around potentially unproductive and unable to do their work. And again, lots of examples of this. This is just about two weeks ago, a little less than that, where we saw ChatGPT, Claude, Gemini, and Grok all down at the same time and for a pretty sustained period. Are you prepared for this? Do you have failover models ready that can be autonomously deployed as an issue arises? And then the deprecation issue I mentioned.

Again, something that is happening faster and faster as these models advance and the cycles between releases are getting shorter and shorter. We deal with a lot of large regulated customers who have to go through extensive periods of testing and validation before they can move a model out into production. That is, again, red teaming it with their security teams. That is checking its performance, its accuracy. All of this can take weeks, sometimes months.

All right, so it's for all of these reasons that we developed our platform, which is, again, all around this topic of active AI governance, encapsulating multiple elements of security, compliance, legal, as well as the financial and operational concerns that I've mentioned. Our deployments typically start with a discovery engagement. This is a very easy, low-risk way to go in and identify where you've got AI across the organization, doing a comprehensive sweep of all models, all providers, vendors, the data, the teams that are using it, and typically within a matter of about 7 to 10 days, giving you a comprehensive picture of where AI has been employed across your organization.

With that in mind, we can then move to secure and govern the use of AI, identifying any risks, vulnerabilities that might exist out there and putting the appropriate controls and guardrails around them to stop them from growing and ultimately shut them down. The second pillar, security, where we can apply agent constraints, guardrails, gateways for your AI and the MCPs, the agentic red teaming, that automated adversarial testing, where we'll unleash our agents to go run across your systems, look for those little gaps, those little deficiencies that can be chained together to create a bigger concern.

All of this is reported back to you then before a bad actor can try to exploit some of those gaps. And what looks like it might be benign today, tomorrow could be very dangerous as that model invokes new permissions and privileges, accesses new systems, and takes on new capabilities. Which leads us then into governance and the capabilities here of understanding where you sit against any regulations that might be in scope for your organization. We provide all the continuous compliance reporting, a full audit and observability trail across all AI use to show where it's happening and what was being done, and this concept of living disclosures, which again is all about the transparency you'll find that's at the heart of most AI regulations around the world.

Now the final pillar of this is our optimization capabilities, which starts with orchestration, being able to manage capabilities across different models, your ability to route them to the right model. And these can be across the same family. Again, in an Anthropic example, I don't need Fable 5 for that marketing intern to produce a slide deck. That might be something we could do with Opus or more likely even with Sonnet, one of the cheaper models, and the cost differences here can be so significant.

That identifies, of course, shadow AI and any automations, zombie agents that might have been stood up but no longer serve a purpose, yet they continue to access sensitive systems or consume tokens. At the enforcement, we can do this at the point of inference, which is a real difference between a lot of the other systems that might be out there. You have existing security vendors that can perhaps give you some visibility, but this sits as an inline proxy so that we can monitor what's happening between the agent or the individual and that model that they're calling.

We'll look at the data that's flowing across. We can run DLP to stop or intercept something that shouldn't go out to a public model, or perhaps tokenize it. So maybe we allow the prompt to go out, but we redact those certain fields that would potentially be compromising. As the prompt comes back with the answer, we can then rehydrate it so that the end user still gets the benefit of using the model and all the productivity without endangering the organization or creating what might be a data breach otherwise.

One of the more recent concerns we've heard from a number of customers is key hijacking. As these tokens become more and more valuable, lots of examples of chatbots being tricked into doing things or someone's keys just being stolen. Again, we can monitor for this, see where the traffic is going, and identify where there is use that looks anomalous or something that shouldn't be allowed. Last points here on the accounting and chargeback, giving you real visibility.

Too often you're looking at a bill at the end of the month that is coming from the provider. It's very hard to understand where the costs are being derived. A common request we'll hear from our clients, of course, is trying to intercept those that might be personal use. I don't want that employee planning their holiday using one of these expensive models, but I might want them to do the very important work that is their core responsibility. So perhaps I'm allowed to use the big frontier model, but when I get to 80% of my budget, I get a notification.

When I get to 90% of my budget, perhaps my company routes me back to a lower model that is going to be just as accurate for what I'm trying to do, but maybe the latency is a little bit higher. So we're still allowing them to be productive, but we're mitigating what would have been a very costly overrun against the budget as part of that process. Now, that is the full platform. Any of these modules can be selected individually, and for most of our clients, they'll start with one or two.

It is then something that allows you to grow into the full platform, though as you need it, you know you've got these other capabilities that could support the organization's needs. And I've worked with many of your organizations before. If any of you recognize AirWatch or OneTrust, I was a co-founder at OneTrust, and many of our team came from there. And then prior to that, AirWatch, which was a mobile security company. So security and compliance is sort of the twin pedigrees that we built the company on behind this.

We're also very proud to have been featured in the recent AI governance Magic Quadrant. This is the first time Gartner has done a Magic Quadrant on AI governance, and we're featured furthest to the right of any solution on the market today. In the companion report, Critical Capabilities, we were also listed as the number one solution for the AI security use case. So really, really pleased with that. I want to thank you for the time today. I hope some of this was useful. And again, if it's relevant for your organization, we'd welcome the opportunity to discuss your situation and how we might be able to support. Thank you very much for the time today.