The AI agent does not just answer. It can act wrongly
An AI agent could pull information from your inbox and CRM, then prepare a draft. But what happens if it treats an old note as current, or makes a change without asking? That is where you find out whether an agent is taking work off your hands or setting off a chain of errors. The key question is not just what an agent can do. It is where it is allowed to act, and who will notice if a step goes wrong.
An agent is a chain, not a magic button
A chatbot usually answers a question in a conversation. An AI agent can go further: it can break a task into steps, call tools and process the results. A tool is a permitted point of access, such as a calendar or document. The business difference is that an answer can become an action.
Imagine asking an agent to prepare a proposal for a prospective customer. It might search a CRM, find relevant documents and produce a draft. Each handover is a step in its own right. If it picks the wrong customer record at the start, the finished draft may still look convincing, even though it is based on the wrong case.
The error does not have to be dramatic. Perhaps a tool call works as intended, but the information it returns is out of date. Or a sensible suggestion is mistaken for an action that has already been carried out. ChatGPT Work, for example, can suggest tasks based on personal information. But according to Latent Space, the work starts only when the user carries out the suggestion. Where do you put the approval point between suggestion and action?
Tools turn answers into actions
An agent becomes more useful when it can access the places where work already happens. Latent Space describes ChatGPT Work as a system that can connect services such as Slack, email, Drive, calendars, CRMs and project trackers. It can then produce finished work. For a small business, that might mean gathering information from several systems for a proposal draft.
Every new point of access also expands the area where something can go wrong. A wrong file, an out-of-date calendar entry or a misunderstood instruction can carry through the next steps. That does not mean every agent will fail this way. It is a practical consequence of several actions depending on one another.
When an action happens is part of the design, too. According to Latent Space, ChatGPT Work uses a scheduler, a feature that can run tasks later or on a recurring basis. A one-off draft and a recurring action have different consequences if circumstances change. Before automating a task, decide what the agent can read, what it can change and which steps need your approval.
Memory can drift out of date
An agent with memory does not have to start from scratch for every task. GitHub describes a system in which agents can save useful experience from previous tasks as memories. At the start of a new session, the latest memories for the relevant repository are retrieved and added to the prompt. The prompt is the instruction and context given to the model.
The tricky question is not just whether the agent can find something. GitHub identifies the core problem as whether stored knowledge remains valid as code, branches and observations change. The same principle applies to business processes. A memory of an earlier approval rule may no longer apply if responsibility has since changed hands.
GitHub addresses this by making memories point to specific locations in the code and checking them in real time when they are retrieved. Memories can also be created only by contributors with write access, and used within the same repository by users with read access. These are specific safeguards, not a guarantee that the information will always be right. Which source should your agent check before trusting a stored note?
Some errors show up after testing
A successful test tells you something about the cases that were checked. It does not automatically tell you what will happen with a different input, outdated context or an unexpected tool result. In a VentureBeat survey from September 2026, 52 per cent of organisations that test AI before release reported at least one case where a tested system later affected customers. Thirty per cent reported more than one such case.
Another figure deserves attention. Among organisations that allowed agents to release certain changes without human review, 80 per cent reported a customer-impacting AI error after tests had passed. That is an association in the survey, not proof that the lack of review caused the errors.
VentureBeat explicitly noted that the survey did not establish whether an error came from an agent change that had not been reviewed. At the same time, just 28 per cent of organisations using agents named automated quality checks of live outputs as their main monitoring method. For your business, the lesson is that testing before launch does not replace monitoring what the agent actually does. Who will spot the error before your customer does?
What this means for your business
Start with a task that comes up often and produces a result someone can check easily. In a small business, an agent could gather customer details and draft a reply without sending it. A medium-sized business can try the same approach with a clearly defined process. In a large company, you also need to agree which teams and permissions are involved.
Write down the process in plain English. What goes in? Which source takes precedence if two details conflict? What can the agent prepare, and which actions need approval? This makes it easier to see where a mistaken assumption could be carried forward.
Then test more than the standard case. Give the process incomplete details, conflicting information and an out-of-date source. Decide how staff should report a wrong result, and who is responsible for correcting access or memory. This takes time before the automation goes live. Without clear ownership, a small error can quietly become routine.
- Choose a task with a result that is easy to recognise.
- Start with read access and drafts before letting the agent make changes itself.
- Name someone to check results and reported errors.
- Document when a memory is out of date and how to remove or update it.
Set a boundary for the agent
The boundary should be clear before anything goes wrong. Define it before the first use: what data does the agent really need? Which actions can it prepare, and which can it carry out itself? When should it hand over to a person because it is unsure?
Memory also needs a defined scope. GitHub limits the memories it describes to a specific repository and ties their use to permissions. That does not provide a ready-made solution for other working environments. The underlying principle is still useful: knowledge should not automatically apply to every task and every person.
The key decision is yours: which actions can the agent handle on its own, and which must a person check?
Frequently asked questions
What is an AI agent?
An AI agent can break a task into steps and use permitted tools, for example to retrieve information from a shared drive or prepare a draft. That differs from a chatbot, which usually provides an answer in a conversation. Once an agent is allowed to take action, access rights, approvals and ongoing checks matter.
How does an AI agent’s memory work?
An agent can store information from earlier tasks and bring it back into its working context later. GitHub, for example, describes repository-specific memories that are retrieved in new sessions. Stored information can become outdated, so the agent should check its source and use knowledge only where it applies.
Why do AI agents fail after passing tests?
Tests cover the situations that were actually checked. Different inputs, outdated information or unexpected tool results can still cause errors later. VentureBeat reported that organisations had experienced customer-impacting errors after tests had passed. The survey did not establish that a lack of human review caused those errors.
How should a business introduce AI agents safely?
Start with a clearly defined task whose result a person can check easily. Give the agent only the access it needs, and let it prepare drafts before allowing it to make changes itself. Test incomplete and conflicting inputs, and decide who will monitor results and errors.
What is the difference between an AI agent and a chatbot?
A chatbot typically answers questions in a conversation. An AI agent can also call tools, bring together information from several sources and carry a task through multiple steps. It can prepare or perform work. The more freedom it has to act, the more important limited permissions, traceable sources and human approval become.
Interested?
Want to know what this means for your company?
Sources
- Building an agentic memory system for GitHub Copilot · The GitHub Blog, 2026-01-15
- Unpacking ChatGPT Work: the Agent for a Billion Users · latent.space, 2026-08-04
- VB Pulse Agentic Reliability and Evals Report - September 2026 · VentureBeat, 2026-10-13
- The share of enterprises trusting AI agents to make production changes on their own has fallen from 75% to 56%, VB Intelligence finds · VentureBeat, 2026-10-08
This article was researched and written with AI assistance and automatically checked against the sources listed before publication. More on our AI transparency
Related solutions
- Klarframe OS →
Bring knowledge together, create transparency and act instead of react. Connect knowledge from documents, email, Notion, Slack and CRM with evidence and an interactive knowledge map.
Klarframe AI Readiness Check
How ready is your company for AI?
About ten minutes, instant result with your stage, a 90-day plan, suitable solutions and notes on the EU AI Act.
