Enterprise AI Governance – From Hype to Accountability
In this exclusive article for Mix Works, global AI governance expert Kay Firth-Butterfield examines why so many organisations have deployed AI without understanding how it works, how it fails, or who’s accountable when it causes harm, and what good AI governance looks like.
…First, Some Clarification
Before organisations can govern AI effectively, they must understand what it is – and what it is not. We’ve had AI in some form for a long time, certainly since the 60s, but the hype hitting the headlines now is around Generative AI (GenAI). Whereas ‘traditional’ AI analyses existing data, GenAI uses learning patterns from massive datasets to create new content. GenAI responds to prompts; ‘AI Agents’ are capable of autonomously performing tasks on behalf of a user or another system.
This output can all feel very human-like, and it’s designed to be just that. But, despite what the media might tell us, an AI Agent is not sentient. It is not a legal person; does not possess intent, judgement, or accountability. It is, fundamentally, a prediction engine: a machine that’s really good at spitting out the next word or the next picture and making associations and predictions. Although it might be doing something autonomously, it’s been trained to do that thing autonomously by a human being.
This distinction matters, because AI governance failures often stem from misplaced assumptions. And misplaced assumptions about GenAI are rife.
Although GenAI feels futuristic, it’s built on earlier, traditional AI systems that already exhibited bias, opacity and unpredictability. It accelerates these issues and also – in its generative capacity – introduces additional risks: hallucinated outputs; the ability to make ‘deep fakes’ (like the recent Tom Cruise / Brad Pitt rooftop ‘fight’ and, more sinisterly, the ‘Nudify’ app); automated decision-making at scale.
The first step in AI governance, then, is clarity. As a leader, you have got to be switched on to the hype surrounding GenAI and know how to separate truth from reality. At minimum, you need to understand:
- Which AI systems are deployed in your organisation
- What they can and cannot do
- Where they are used and who’s using them
- What risks they introduce
Without this foundation of clarity, any governance frameworks will be performative rather than effective.
Debunking the “Governance Blocks Innovation” Myth
A persistent myth in enterprise environments is that “regulation kills innovation”. I see a lot of companies saying “we want to move fast and this responsible AI will cause us to slow down or not deploy fast enough”.
Just recently, a leader said to me, “We’re about to deploy AI agents, so we don’t want to think about an advisory board yet.” I replied: “if you’re going to think about an advisory board after deploying your agents, you’re going to have a lot of problems to unwind”.
In reality, governance is an enabler of sustainable innovation. Cars are a useful example. Regulation did not destroy the automotive industry – it improved it. Regulations around brakes, safety standards, and testing created safer products and increased public trust.
AI is no different. Governance provides:
- Trust for customers, regulators, and investors
- Protection against legal and reputational risk
- Clarity for employees deploying AI systems
- Stability for scaling AI safely
Organisations that deploy AI without governance often discover problems later – and then have to roll back deployments at significant cost. We’re starting to see that now, with Deloitte’s 2026 State of AI Report finding only 20% of organisations have seen revenue grow as a result of AI. I’ve spoken to multiple leaders who’ve rushed to deploy AI initiatives and then, a year later, come to me asking: “how do I make money out of this?”
I can’t over-emphasise: AI governance is not friction. It is infrastructure.
Now we’ve got that myth out of the way, we can start discussing what regulation looks like. And – to be clear – if an AI is capable of harming a person, then we do need to have some form of regulation for it.
Enterprises Now Carry the Legal and Operational Risk
One of the most important realities for enterprise leaders is this: when AI causes harm, the enterprise is accountable.
Even where external vendors provide the technology, liability frequently sits with the organisation deploying it: it’s not enough to point the finger at the software. This risk arises because, ultimately, it’s the enterprise that makes the decision to deploy AI. They choose the vendor, the use case, the level of oversight and the guardrails.
Negligence claims and lawsuits related to AI bias and system failures are already emerging. And, according to IDC FutureScape 2026 predictions, inadequate controls and governance of AI agents will result in up to 20% of G1000 organisations facing lawsuits, regulatory fines, or executive firings by 2030.
Procurement – vendor reliability, data protection practices, risk exposure, legal protections – must be baked into your governance and guardrails. When harm occurs, you must be able to demonstrate you acted responsibly. Governance provides this protection.
AI Governance Is a Board-Level Responsibility
In any organisation, right now, you’ll find a lot of the people on the board don’t really understand what an AI agent is. But every board member should be able to answer questions such as:
- Where are we using AI?
- What guardrails do we have?
- If we were sued, can we say that we have at least tried to use it properly?
When it comes to AI governance, everyone is responsible – it can’t be delegated solely to technical teams or HR. Simply, without board-level governance, organisations can’t demonstrate responsible control. Having AI-literate board members isn’t optional anymore – it’s a business and legal imperative.
I’m starting to see an increasing number of organisations with internal AI governance boards or advisory groups that bring together legal, compliance, risk, and operational leaders. Having an entity that stretches across all the silos prevents fragmented deployment, where different departments adopt AI independently without oversight. It’s a wise move I’d recommend for all enterprises.
Leading the AI Driven Organisation
Equip senior leaders and boards with the knowledge and strategies to navigate AI transformation successfully.
Workslop, Hallucinations & Shadow AI
‘Shadow AI’ – employees using unapproved AI tools independently or in an unapproved manner – is one of the most urgent governance challenges. This behaviour is already widespread, and the consequences can be dire.
Here’s a real-world example for you, shared with me by a client. An engineer was tasked with designing a safety critical piece but, when it was inserted, it blew up. When the engineer was asked to show their design thinking and calculations, they revealed that ChatGPT had told them to do it. And now the organisation is being sued.
When I heard this story, the CEO asked me if I thought they had a chance in the law case. I said no. Not because a mistake was made, but because they had no policies in place to prevent it happening. And one policy would at least have protected them.
Hallucinated outputs are another example. If your employees rely on hallucinated AI outputs and incorporate them into proprietary data, those errors can propagate through enterprise systems, contaminating business intelligence and decision-making.
In some cases, this might be fairly harmless. In others, for example a hallucination in financial reporting, it may be catastrophic.
We think about risk in other aspects of our business. Why aren’t we thinking about risk in regard to AI?
The answer, I think, is that the hype cycle is so strong, we’re constantly thinking we’re on the back foot. And that, if we don’t deploy AI right now, we’ll get left behind. But, as we’ve seen, the cost of unregulated AI – from productivity-destroying ‘Workslop’ to hefty litigation – far outweighs the benefits of “deploy now, figure the rest out later” thinking.
Do These Two Things (Now)
If you’re dragging your heels on AI Governance, I urge you to do two things – and start right now:
1. Establish AI Guardrails
Guardrails ensure AI systems operate within defined boundaries, preventing drift, hallucinations, and unsafe outputs. Organisations can use them to define:
- Acceptable risk thresholds
- High-risk vs low risk use cases
- Monitoring and oversight mechanisms
- Human review requirements
- Procurement rules
2. Train Everyone. And then train them again (and again).
Training must occur across all levels, from board to frontline employees, and it can’t be a one-time exercise. It has to be continuous, because the technology evolves constantly.
Training should include:
- Understanding hallucinations
- Understanding limitations of AI systems
- Appropriate use cases
- Risk awareness
- Governance policies
All AI training needs to start by saying, “this is what AI is. And you might have read this, and you might have read that, but this is the AI that we are deploying in our company. And this is what it can do.” It’s important to establish the parameters of AI for your particular organisation from the outset.
Bringing in an external consultancy to support cross-company AI training can provide the expertise and objectivity that even the best internal teams struggle to deliver alone. Most organisations lack comprehensive in-house knowledge across governance, legal risk, culture, and technical implementation, and internal teams may operate in silos without a clear, enterprise-wide view.
Mix Works AI Training
Upskill your organisation in AI with practical, business-focused training that supports real transformation.
“Wise AI”
When I first started out, we used to talk about ‘ethical AI’. But ethics vary. I might be speaking to someone from a different country and they could reasonably ask – “are these my ethics or your ethics?” Today, we talk about ‘responsible’ and ‘trustworthy’ AI but, actually, when I’m talking to businesses, I simply say:
“Let’s deploy AI wisely. Let’s make sure that it works as a great tool for you and you don’t end up in court because of the way that you used it.”
AI governance is not about slowing progress. It’s about enabling sustainable progress. Enterprises that treat governance as an afterthought face increasing legal, operational, and reputational risk. But if you embed governance within leadership, culture, procurement, and training, you’ll realise AI’s potential safely, turning it into a strategic asset rather than an unmanaged liability. But you need to begin today.
About the Author
Kay Firth-Butterfield is a leading global expert on the use and governance of AI. A Barrister, former Judge and Professor, technologist and CEO of Good Tech Advisory, Kay was the world’s first C-Suite appointee in AI Ethics in 2014. In 2015, Kay contributed to the Asilomar AI Principles and became Vice Chair of the IEEE Global Initiative for Ethically Aligned Design. Kay was the inaugural Head of AI and Machine Learning at the World Economic Forum from 2017-2023. In 2024 Kay received the TIME 100 AI Impact Award.
Her new book, Coexisting with AI, is essential reading for business leaders making strategic decisions about AI implementation, professionals adapting to AI-augmented workplaces, parents guiding children through a technology-saturated world, and anyone hoping to better understand how AI will shape our collective future.
Kay can work with you on topics including organisational governance, responsible adoption of AI, the impact of AI on workforce diversity, regulation, management and leadership in the age of AI. She is also available for keynote speeches, seminars, and workshops. Contact us for more information.
