When AI Agents Start Working, Responsibility Must Be Designed In

Reuters recently reported something quite interesting: as AI agents begin autonomously performing more and more real tasks, cyber insurers are re-examining their policies.

In the past, cyber insurance mainly covered hacking, data breaches and system failures. But when an AI agent — without step-by-step human direction — autonomously accesses systems, calls tools and makes decisions, any loss it causes starts to blur the traditional definitions of liability:

Does an AI agent count as an "attacker"? Among the model provider, the software developer and the company using the agent, who exactly should be responsible? If an agent's actions go beyond its original instructions, does the insurance still pay out?

News link: Reuters: As AI agents go rogue, cyber insurers are adapting their policies

This report made me think: when we discuss AI agents, everyone tends to focus first on what they can do — and rarely on what happens after they do something wrong.

A chatbot answering an ordinary question incorrectly may have limited impact. But if an agent can access a company's email, modify customer records, contact suppliers, create payment instructions, or even handle financial and tax matters, then a single mistake can turn from an "inaccurate answer" into a real business loss.

For SnapLedger, this question matters especially.

In the future, Snappy can help users classify transactions, match documents, generate invoices, prepare accounting records, follow up with customers, find suppliers — and even connect to banking, tax and e-invoicing systems.

But the more an agent can do, the less we can pursue "degree of automation" alone.

We must also design boundaries for responsibility.

For example: a transaction can be auto-classified by AI, but when confidence is insufficient, it should be handed to the user or an accountant for confirmation. An invoice can be generated automatically, but before it is formally sent, the customer's identity, tax rate and payment details may need checking. A tax filing can be prepared automatically, but its formal submission should go through a clear review and authorisation process.

Different actions carry different risks — so they should not be given identical automation privileges.

I believe a truly reliable AI agent system must at least be able to answer several questions:

• Why did the agent take this action?
• What data and rules did it use?
• What permissions did it hold?
• Which steps can be completed automatically?
• Which steps must be approved by a human?
• If an error occurs, can it be traced, explained and corrected?

This is why SnapLedger needs not only AI, but also deterministic accounting logic, permission controls, a complete audit trail, and clearly defined human review checkpoints.

In traditional software, responsibility is usually easy to trace, because every click and every submission is made by a specific person.

But in the agent era, the human role is shifting from "executing every step" to "setting goals, granting permissions and reviewing results". This does not mean humans are no longer responsible — it means responsibility must be written into the system design in a new way.

I don't believe the solution is to prevent agents from doing anything.

If every action requires step-by-step human confirmation, then the agent ends up as nothing more than traditional software with a new interface — and it cannot genuinely improve efficiency.

A more sensible approach is to let the agent earn trust gradually, within explicit boundaries:

Make suggestions first, then execute low-risk tasks; Keep human review in place first, then gradually expand permissions based on actual performance; The closer a task gets to money, taxes and external commitments, the stricter the system's requirements for evidence, approval and traceability should be.

The fact that insurers are starting to rewrite their policies shows that AI agents are no longer just a laboratory concept. They are entering real business systems — and beginning to create real questions of responsibility.

For AI startups, this is an important reminder:

We cannot only design an agent's capabilities. We must also design its responsibilities.

A truly trustworthy AI agent is not one that never makes mistakes. It is one that knows its boundaries before acting, leaves complete evidence while acting, and knows when it exceeds its authority — and hands the decision back to a human.