When AI Moves from Insight to Action

AI has officially moved from experimentation to operations. In KPMG's Keeping Us Up at Night 2026 survey, AI-related issues overtook inflation to become the number one challenge facing Australian business leaders. That concern isn't philosophical. It's structural. 

The data points to a widening governance gap: 

• New technologies, including AI, are the number one challenge for the year ahead, cited by 63% of business leaders. 
• 88% of organisations are already investing in agentic AI 
• Yet ASIC has found variable maturity in how businesses manage AI governance risks. Oversight isn't keeping pace with adoption. 

In short, adoption is moving faster than accountability. That gap is now visible to regulators. 

In 2024 and 2025, AI lived in the sandbox. In 2026, it lives in production. AI systems are now:  

• Handling customer claims, 
•  Drafting and executing contracts,  
• Setting prices and approving transactions 
•  Interacting directly with customers and counterparties. 

And the legal landscape has caught up. Courts and regulators are increasingly applying existing laws to AI-driven outcomes. From a liability perspective, the position is becoming clear: if an AI system acts on behalf of the organisation, the responsibility is likely to remain with that organisation.  

“The system made a mistake "is unlikely to remove an organisation responsibility from a legal standpoint, it is treated as your decision, executed at machine speed. 

Most board-level AI discussions focus on ethics, governance and explainability. Fewer ask the more uncomfortable question: if an AI-driven decision causes loss, who actually pays? 

Many cyber and professional indemnity policies were written before autonomous and decision-making systems became operational. As a result, some policies: 

• exclude algorithm-driven failures 
• be silent on non-malicious AI errors 
• not have been designed to respond to regulator-led action tied to AI behaviour 

The point isn't that cover doesn't exist. It's that coverage is often assumed rather than confirmed against how a business uses AI. That gap is worth closing before it is tested.

This is not a call to “buy more insurance”. It is a call to align insurance architecture with operational reality. As AI becomes embedded in core processes, insurance needs to be examined through a different lens:  

• what decisions does AI actually make?  
• Where does human oversight begin and end?  
• How would a regulator characterise a failure?  
• Which policy would be expected to respond, and why? 

These are not technical questions. They are governance, resilience and balance-sheet questions. 

The organisations best positioned for 2026 are not those slowing AI adoption, but those closing the gap between how AI operates and how risk is transferred. 

That work doesn't start with buying insurance. It starts with testing assumptions: pressure-testing how AI decisions would be viewed by a regulator, how losses would be characterised legally, and how existing policies would actually respond in practice. 

At Knightcorp, we help leadership teams run that reality check while it's still a choice. We turn assumption into evidence, and exposure into a plan. That clarity, ahead of the market, is the advantage. 

1. Who is responsible when an AI system makes a decision on a company's behalf? 

This is an area of developing law, but the direction regulators have signalled is consistent: accountability for an AI-driven outcome is expected to sit with the organisation, not the technology. ASIC has emphasised that existing laws are technology-neutral and already apply to AI, including directors' duties and consumer protection obligations that apply to businesses generally, and licence obligations for ASIC-regulated licensees. It has also indicated that AI supplied by third parties should be governed to the same standard as in-house systems. How that applies to any specific decision depends on the circumstances and is a question for legal advice. 

2. Can a business rely on “the AI made the mistake” as a defence? 

Regulators are signalling that this is unlikely to be a reliable shield. ASIC's stated position is that organisations remain accountable for the governance and oversight around the technology they deploy. So, the question that tends to matter is less whether the AI erred and more whether the organisation can demonstrate appropriate control over how it operates. Whether a particular decision creates liability is a legal question that turns on the facts. 

3. Does cyber insurance or professional indemnity cover AI-driven errors? 

It depends on the wording of your specific policy, and this is general information rather than a view on any individual policy. Many cyber and professional indemnity policies were written before autonomous, decision-making AI became operational, so some may exclude algorithm-driven failures, stay silent on non-malicious AI errors, or were never designed to respond to regulator-led action tied to AI behaviour. The common exposure is not that cover is missing, but that it is assumed rather than confirmed against how the business actually uses AI. 

4. What is the AI accountability gap? 

It's the widening distance between how fast organisations are adopting AI and how slowly their oversight, controls and risk transfer are keeping up. AI is being embedded into core workflows while governance, board oversight and insurance arrangements lag behind. It's that gap, rather than the technology itself, that creates exposure. 

5. Why has AI accountability become a more pressing issue? 

Because AI has moved from experimentation into day-to-day operations, increasingly used to handle claims, draft and execute contracts, set prices and deal directly with customers. AI-related issues have also become the number one concern for Australian business leaders, overtaking inflation, according to KPMG's Keeping Us Up at Night survey. As reliance has grown, regulators including ASIC have flagged AI and agentic AI as priorities, and existing laws are increasingly being looked to where AI-driven decisions cause harm. 

6. What is agentic AI, and why does it change a company's risk? 

Agentic AI describes systems that don't just recommend but plan and act on their own, making and carrying out decisions without a human approving each step. That autonomy is what sharpens the accountability question, because the organisation still owns the outcomes of actions it never individually signed off. ASIC has noted that agentic AI can compound risk given its capability to independently plan and act. 

7. How can a business find out whether its insurance would respond to an AI-related loss? 

The reliable way is to test it against your organisation's actual AI use and actual policies rather than assume, since general statements about cover can't tell you how your specific policies would respond. That means mapping what decisions the AI genuinely makes, where human oversight begins and ends, how a regulator might characterise a failure, and which policy would be expected to respond and why. Knightcorp helps leadership teams work through exactly this kind of review before an incident forces it. 

8. Isn't the answer just to buy more insurance? 

Not on its own. The work is less about buying more cover and more about aligning insurance with how AI actually operates inside the business. Cover that was never designed for autonomous decision-making may not respond as expected, however much of it you hold. What any given policy does is determined by its wording, not by the amount of cover in place. 

9. What should boards and executives be asking about AI and insurance? 

The sharper questions are governance and balance-sheet questions, not technical ones: What decisions does our AI actually make? Where does human oversight start and stop? How would a regulator characterise a failure? Which policy would respond, and why? Working through these tends to surface false certainty before a loss does.