Disclosure: BlueBear sells a platform that provides several of the controls below, so we have a commercial interest. Most of these questions are answerable with what you already run, and the article says which ones genuinely need a product and which do not.
A demo and a production system answer different questions
A demo answers can it do the task. By the time you are considering real work, that question is usually settled -- and it is the least interesting one, because it is about the path that worked.
Production is about the other paths. Not because agents are unreliable, but because the properties that matter when something goes wrong are ones a successful demo never exercises.
These seven are the questions worth answering first. Each has a page that goes deeper.
1. Who is the agent acting as?
If it borrows a shared application credential, every action it takes is attributed to that application rather than to the agent or the person it served. Attribution then depends on inference from timing. Give it an identity of its own, and record the person it is acting for on each action.
This is the question that governs several others, because you cannot scope, approve or audit an actor you cannot name. See the credential decision behind provable agent actions.
2. What can it reach that it does not need?
An agent's blast radius is whatever its credentials permit, not whatever its job requires, and the two diverge quietly. Enumerate what it can reach today. The list is usually longer than expected, because credentials get reused for convenience during the pilot and never narrowed afterwards.
3. Which of its actions cannot be undone?
Reversibility is a better sorting criterion than importance because it is objective: could an operator put this back within ten minutes? Anything that fails that test needs a human before it, not after. If nobody has made this list, no gate is protecting those actions. See where to put the human.
4. Could you prove what it did?
Not could you find it in a log -- could you demonstrate, to someone who is not inclined to take your word, which identity took which action against which system, and on what authority. That requires an authorisation record written per action. It cannot be reconstructed later from telemetry.
5. What does an accepted outcome cost?
Including failed attempts and the human time spent checking. The per-call price is not the unit cost, and an agent with a poor first-pass rate can cost several times what its headline suggests. This is also the number that tells you whether the thing is worth running. See cost allocation and unit economics.
6. What is in your telemetry that should not be?
Tracing captures prompts, tool arguments and responses, which is to say customer data, usually into a store with wider access and longer retention than the source. Worth checking before the trace store becomes your largest uninventoried copy of it. See what your traces are actually holding.
7. What happens when a dependency is unavailable?
If the component enforcing policy is down, do agents stop or continue unpoliced? Continuing is the option that keeps things running and removes every control in the window where something is already wrong. This should be a decision on record, not an emergent behaviour.
How to use this
Answer all seven for one agent before scaling any of them, and write the answers down. Most of these are cheap to fix while there is one agent and one integration, and expensive once there are ten of each, because the fixes are structural rather than incremental.
The one that is genuinely hard to retrofit is the fourth. Identity, scope, approvals and cost tracking can all be tightened later with effort. Evidence cannot be created retrospectively for work that already happened -- if the record was not written at the time, that period is simply undocumented, and no amount of later diligence recovers it.
How to run this without it becoming a committee
Seven questions can turn into a quarter of meetings. The point is to answer them, not to govern them, so give the whole exercise a fixed shape.
Take one agent — the one closest to real work, not the most impressive demo. Answer all seven for that agent only, in a single sitting, writing one paragraph each. Where the answer is "we do not know", that is the answer; record it and move on rather than pausing to investigate. The output is a page, not a programme.
What that page gives you is a list of unknowns ordered by cost of being wrong, which is the only prioritisation that matters here. Most teams find two or three genuine gaps and four questions they could already answer but had never written down. Writing down the ones you can answer is not busywork — it is what makes the gaps visible by contrast.
The mistake that makes all seven harder
Scaling before answering them once. It is tempting, because each agent individually looks small and the questions feel like overhead you can retrofit.
The trap is that the answers are structural. Identity, scope, approval routing and evidence are properties of how agents connect to your systems, not properties of any single agent. Ten agents built without them are not ten small problems; they are one architectural problem with ten instances, and fixing it means touching all ten.
Answering the seven for one agent costs a day. Answering them for the tenth costs a quarter, and by then something is usually in production that you cannot pause while you fix it.
What "ready" actually means here
Not that nothing will go wrong. Ready means that when something does, you can say what happened, on whose authority, and whether it can be undone — and that the answers come from records rather than from reconstruction.
That is a lower bar than it sounds and most teams are further from it than they expect, because a working demo provides none of the evidence and every incentive says ship. The seven questions are worth answering precisely because nothing about a successful pilot forces you to.
BlueBear Production Readiness Decision Record
| Decision | Required evidence | Accountable owner |
|---|---|---|
| Release scope | Versioned workflow, input boundary, tools, models, users, and accepted outcome. | Business workflow owner. |
| Security boundary | Identity, workspace, action policy, credential resolution, approval, and denial tests. | Security owner. |
| Reliability | Normal, timeout, retry, duplicate, downstream failure, repair, and containment results. | Service owner. |
| Economics | Cost per accepted outcome with model, tool, runtime, review, and exception inputs. | Budget owner. |
| Operate or stop | Monitoring, queue, incident, change, rollback, retention, and retirement procedures. | Named operational owner. |
The signed decision should include gaps, compensating controls, expiration/review date, and the person authorized to stop the workflow. “Pilot succeeded” is not a production approval.
Questions people actually search for
- what does production ready mean for an ai agent
That you have answered what happens when it goes wrong, not that it works. Specifically: which identity it acts as and what that identity may reach, which of its actions cannot be undone, whether you could prove afterwards what it did and on whose authority, what an accepted outcome costs including retries and review, and what happens when its dependencies are unavailable. A demo exercises none of these because a demo only shows the path that worked.
- why do ai agents fail when they move from pilot to production
Because a pilot runs on selected inputs with a person watching and no permanent consequences, and production supplies awkward inputs, no observer, and effects that persist. The failures are rarely about model quality. They are about authority the agent turned out to have, actions nobody realised were irreversible, and records that cannot answer questions after the fact.
- what should you check before letting an ai agent act on real data
Four things at minimum. That it has its own identity rather than borrowing a shared application credential. That someone has enumerated which of its actions have no undo. That an authorisation record is written per action, not per session. And that you know the cost of an accepted outcome including failed attempts. Everything else can be improved later; these are the ones that are expensive to retrofit.
- how do you know if an ai agent pilot is ready to scale
When the answers to the readiness questions are recorded facts rather than intentions, and when the failure path has been exercised at least once deliberately. If nobody has ever seen the agent fail, or seen what the audit record looks like for a failed run, the pilot has demonstrated capability and nothing about operability.