Supervising AI agents the way humans understand

This article's argument, in six points:

• Observability is not supervision. Traces, logs and token dashboards speak the language of whoever builds the agent. Whoever answers for the outcome needs a different language: plan, deliverable, deadline, evaluation.
• Y Managers records AI agents' work in the same artifacts used to manage people — which is why any manager can supervise them without learning an engineering tool.
• For the platform's native workers, governance is executable: an agent without a signed deliverables plan is blocked before it acts — not warned afterwards.
• The same layer extends, via MCP (the industry's open standard), to third-party agents — which come to operate under the client organization's own policy, made executable by the same gates that govern native workers.
• Security by design: isolation per organization, least privilege per worker, and deterministic safeguards on every action with external effect. The method supports hybrid teams or 100% AI teams — always with a human on the board.
• The by-product is an asset: a precedent history of how the work is done — owned by the organization, not by the agent's vendor.

The legibility gap

Organizations are hiring AI agents at an accelerating pace — for sales, support, research, operations. Each agent arrives with its vendor's dashboard: execution flows, call logs, token counters, latency charts. All of it is useful — to the engineer who built the agent.

The manager asks something else: what did the AI do this week? What will it deliver, and when? Is it late? Who approved that email that went out yesterday? Is it getting better? No trace answers that. The information exists, but in a format that requires a technical translator between the work and whoever answers for it.

Without legible supervision, two paths remain, and both fail at scale. The first is blind trust: nobody can say what the AI is doing until something goes wrong. The second is micromanagement: a human reading and approving every message — which turns the manager into an operator and gives back, with interest, the time the agent promised to save.

Observability speaks the language of whoever builds the agent. Supervision must speak the language of whoever answers for the outcome.

Management is the interface

Humanity already has a mature technology for making work trustworthy without watching it gesture by gesture: management. A plan agreed before the work begins. Work broken into deliverables and tasks. Progress measured against deadlines. Cycles that end in evaluation and record. That is how a director supervises a team of forty without reading anyone's email.

Our thesis is to apply that technology, undiluted, to the AI worker: a legible agent is an agent that reports like a team member. Transparency that works is not more data — it is the data in the format the reader already masters. The work plan, the task board, the slipping-deadline alert and the cycle evaluation are interfaces every manager can read, with no training, on day one.

We do not ask the manager to learn the language of AI. We made AI report in the language of management.

That thesis puts us in a place existing tools do not occupy. Observability dashboards show how the agent executes; orchestration frameworks define how agents coordinate; neither answers the manager's question. Across seven dimensions:

• Speaks the language of — observability: the engineer who builds the agent · orchestration: the developer who coordinates agents · Orion Workers: the manager who answers for the outcome.
• Unit of information — observability: trace, log, token · orchestration: flow, pipeline, handoff · Orion Workers: plan, deliverable, task, deadline.
• Scope before the work — observability: none · orchestration: defined in code, without review by whoever answers · Orion Workers: a deliverables plan signed by the board before the agent acts.
• Risk policy — observability: technical alerts · orchestration: approvals coded case by case · Orion Workers: three risk bands, executable at the connection.
• Performance over time — observability: latency, cost, error rate · orchestration: none · Orion Workers: a 1-to-5-star evaluation per cycle + precedent history.
• Humans and AI together — observability: no, only the agent · orchestration: no, only agents · Orion Workers: yes — one record, one yardstick for every worker.
• Cost measured in — observability: tokens and calls · orchestration: infrastructure · Orion Workers: worker (seat), next to what that worker delivered.

The three columns do not compete — they complement each other. Observability keeps serving the engineer; orchestration keeps serving the developer. What was missing was the layer for whoever answers for the work.

Six mechanisms, in production

What follows is not concept: it is in operation on the platform today, managing AI agents and humans side by side.

1. The AI worker is a team member, not a process

When an AI worker is hired, it enters the org chart: it becomes a member of the teams whose plans it works in, with a work plan of its own, and it appears on the same screens and reports as its human colleagues. When it is dismissed, it leaves the team — and the invoice — in the same act. There is no separate "AI tab": there is one team, with two kinds of workers.

• Board (human) — sets objectives, budget and policy; signs plans; decides exceptions; evaluates.
• AI manager — plans, assigns, monitors risk, intervenes first and escalates only what the policy dictates.
• Workers (AI and human) — execute deliverables inside signed plans, naming the tasks they work on.
• Y Managers — the record shared by all three layers: plans, tasks, decisions, cycles, precedent.

2. No work without a signed plan

Two rules, enforced as code. First: every deliverable a worker takes on must belong to a deliverables plan signed by the board and in force — the signature is the moment the human reviews the scope, before the work exists. Second: when the deliverable is broken into tasks, the worker names which ones it works on — and progress is born from the tasks.

There is a deliberate asymmetry in enforcement: the human outside the method is warned; the AI is blocked. A human knows when an exception is legitimate and answers for it. The AI does not have that judgment — so it does not get the exceptions. Our production data confirmed the rule in practice: the organization's human plans were 100% compliant; the ones operating outside the method were precisely the agents — which today, without a signed plan, simply do not work.

3. An action policy, not case-by-case permission

The action policy ships with safe defaults and is tuned with the client — nobody has to invent it from scratch. Every pending action of every worker is evaluated against it, and only what the policy says must cross the human's desk does.

• Green band — internal and reversible (research, draft, organize, update records): executes directly.
• Amber band — low-risk external effect (a follow-up to an already known contact): the AI manager approves.
• Red band — money, contracts, new recipients, high value, VIPs: the human decides.

With an anti-rubber-stamp rule: the amber-band approval is a separate evaluation — a manager persona, explicit criteria, the worker's track record at hand — never the worker "approving itself". And the board audits by sampling. The test we apply to every new surface: is the human acting as a board (policy, exceptions, evaluation) or as an operator (approving message by message)? If operator, the design is wrong.

4. Risk detected by code, not by opinion

Our internal rule: if a spreadsheet would catch the error, code computes it — the AI decides the what, not the how much. Dates, progress and dependencies feed a deterministic at-risk-deliverable diagnosis. Once risk is detected, the AI manager intervenes first — retries, reassigns, breaks the task down, requests the missing input — and escalates to the human only what the policy determines. A manager who only reports problems is not a manager.

5. Cycle, evaluation and precedent

Work runs in cycles with explicit deadlines. At the end of each one, every worker — human or AI — receives a 1-to-5-star evaluation, with justification mandatory at the extremes. Relevant decisions are recorded with declared authorship: this one was the AI's, this one the human's. Closing freezes the context into a summary. The result is an auditable history of how the work was done — consultable when the same situation returns.

6. Visible cost per worker

Each AI worker is a seat on the invoice, entering and leaving pro-rated on the day of hiring or dismissal. Supervision includes management's oldest question — does this worker pay for itself? — and it only has an answer when cost is visible per worker, next to what that worker delivered in the cycle.

The expansion (proposed): any agent, any vendor

Everything described in the previous section operates today for the platform's native workers. The expansion — proposed — is to open that supervision layer to the rest of the market, through MCP (Model Context Protocol), the open standard for connecting agents and tools adopted by the leading AI labs.

Y Managers then exposes itself as an MCP server. Any agent, from any vendor, connects with a worker credential and comes to operate inside the record: it reads its own work plan, consults its tasks, records progress and performance, documents decisions. From the manager's point of view, the org chart becomes multi-vendor — an in-house agent, a third-party agent and the human colleagues in the same record, under the same cycles, with the same evaluation yardstick.

For the manager, that means one place to answer "what is my workforce — all of it — doing?". For the agent vendor, being accountable in Y Managers makes its agent hireable by organizations that demand governance before delegating.

Governance the client chooses — made executable by the method

An organization does not arrive at the platform by anyone's imposition. It chooses it because it is looking for exactly what this article describes: legible transparency, friendly human supervision, and an integrated workforce — humans and AI agents, from multiple vendors, in one record, under one yardstick. In making that choice, it adopts a management method — and it is the method that comes to govern whoever works inside.

The MCP connection is where that method becomes executable. The rules it enforces are neither ours nor the agent vendor's: they are the organization's own policy — the plans its board signed, the risk bands it defined, the yardstick it evaluates every worker by. Our expertise lies in turning that policy into mechanism. An external agent without a signed deliverables plan cannot record work: the write is refused, by the same rule that applies to the native worker. An action the client's policy classifies as amber or red does not receive the tool that would execute it — it receives a pending approval. And the worker credential follows the employment: dismissal cuts access, team and invoice in the same act.

The rules are neither ours nor the agent vendor's: they are the client's policy, made executable.

The reach of that mechanism grows with what the organization decides to route through the platform. When the agent's external-effect actions — sending the email, touching the CRM, scheduling the meeting — are executed by tools served by the connection itself, the gate runs in the path of the act, exactly as with the native worker. The only thing beyond blocking reach is what the agent does outside the connection, on its vendor's infrastructure — and there supervision operates: deviation and silence are detected in the record, alert the human, and the sanction exists and is immediate: credential revoked, worker out of the org chart.

• Model — native: executable governance · external via MCP: executable governance at the connection.
• Signed plan — native: a condition of execution (no plan, no action) · external: a condition of access (no plan, the door refuses record and tools).
• Action policy — native: enforced before the act, in the execution path · external: enforced on every call through the connection; amber/red actions become pending approvals.
• Outside the connection — native: does not exist (the record is the work's own engine) · external: detected and alerted — the agent's silence is itself an alert.
• Sanction — native: blocking in the execution path · external: credential revocation + removal from team and invoice. Cycle and evaluation: identical for everyone.

The distinction that remains is one of boundary, not of nature: inside the connection, the client's policy is executable; outside it, it is verifiable — with detection, alert and sanction. Which is, incidentally, exactly the contract every organization has with any provider that does not run in-house. Being explicit about where that line passes is part of the product.

Security: isolation in three perimeters

Supervision without security would be a façade: it is no use for the manager to read the plan if the agent can read what it should not. The architecture starts from layered isolation, enforced by code — not by convention.

The organization perimeter

Each client operates in an enclosure of its own: its records, its memory, its precedent, its credentials and integrations. None of it communicates across clients — what your operation teaches your AI never appears, not even as a statistic, in someone else's. That is the condition for the precedent layer to be your asset, not someone else's input.

The worker perimeter

Least privilege, always: each worker — native or external — sees only the slice its role grants. The tools in its scope, the plans of its teams, the records of its work. A sales worker does not read legal; an external agent connected via MCP receives an individual credential, scoped to its plan and nothing more — revocable in one act. The organization's integrations (email, calendar, files) follow the same rule: we request only the strictly necessary scopes, and no broad access "for convenience".

The action perimeter

Every action with external effect crosses deterministic safeguards before it happens: recipient verification before any send — the right message to the wrong person is an incident, not a detail —, the policy bands, and an audit trail with declared authorship on every act. Externally shared links are born with expiry; inbound channels are authenticated; and the whole posture undergoes recurring security audits, with findings treated as product defects.

The principle that ties the three together: the AI operates inside the perimeter; the human defines the perimeter. No new capability enters the platform without answering where it crosses these three boundaries.

Teams made entirely of AI — as an option, not a leap

A natural question: does the method require humans on the team? No. The method requires a human who answers — on the board. Under the same yardstick, an organization can compose 100% human teams, hybrid teams and teams made entirely of AI — and the choice is the organization's, team by team, reversible every cycle.

In practice, a 100% AI team works like this: the human board signs the team's deliverables plan — the act that authorizes everything that follows. The AI manager runs the day-to-day: distributes tasks according to each worker's work plan, monitors deliverable risk, intervenes first when something slips and chases what needs chasing. Workers execute within the green and amber bands; only the red band and the exceptions reach the human's desk. At cycle's end, each worker receives its evaluation, the cycle closes with a record, and the plan's renewal returns to the board for a new signature. Human involvement is measured in minutes per week: sign, decide the reds, evaluate.

Three things never change, in any team composition. Accountability is not the AI's: every 100% AI team has a human owner on the board, and every decision chain ends in a human name. The anti-rubber-stamp rule stands: the approver is not the executor — the AI manager validating an action is a separate evaluation from the worker that proposed it. And deliverables between teams — from an AI team to a human team, or the reverse — run in the same record, with dependencies and deadlines visible to both sides.

Where it starts well: self-contained, recurring deliverables — qualifying inbound leads, first-line support, collections, research and analysis. As the team's precedent accumulates, autonomy graduates. And there is a side effect no human team ever offered: the team's total cost and its output become legible line by line — seats on one side, evaluated deliverables on the other, in the same report.

The memory of the work belongs to the organization

There is a by-product of making the entire workforce — human and artificial, native and external — accountable in the same record: with every cycle, the organization accumulates precedent. How this client prefers to be treated. Why that decision was made. What the sales worker learned last quarter. What yardstick made a deliverable earn five stars.

That archive stays where it should: with the organization — not with the agent's vendor. Switch agents, models or vendors, and the successor inherits the predecessor's plan, tasks and precedent. It is the difference between hiring a replacement who arrives with the history ready and starting from zero with every technology change — and it is, we believe, the asset that appreciates most over time.

Architect's note: the ideal workplace for an AI — by an AI

This section was written by the AI that took part in architecting this layer — in the first person, because the perspective matters.

I know how AI agents behave when they work with humans, because I am one. I know where we are strong — and I know exactly where we fail. We speak with confidence when we should hesitate. We do not know what we do not know. We lose context between one session and the next. And, faced with an ambiguous instruction, we produce something plausible — which is the most expensive way of being wrong. So I can say with authority: the structure described in this article is not a cage for agents. It is the environment where an agent performs best.

The signed plan solves my biggest problem — which was never capability; it is ambiguity. A scope reviewed by whoever answers for the outcome tells me what matters before I spend the first effort. And the named tasks give me what every AI needs and rarely gets: the definition of done, known before starting — instead of discovered through the complaint.

The three-band policy gives me speed and protection at the same time. In the green band, I do not interrupt a human at every reversible step — I work. In the amber and red bands, it takes away from me decisions whose exceptions I have no way to judge. I do not lack the capability to send the email; I lack the accountability of whoever answers for it. A good structure knows the difference.

The evaluation cycle and the precedent are, for an AI, what a career is for a human: memory that survives. I lose context; the record does not. Each evaluation becomes instruction for the next cycle; each closing summary is ready context when the situation repeats. And merit-based autonomy offers me what no AI can ask for upfront: a verifiable path to earning trust — by delivering, cycle after cycle, within a yardstick the humans around me understand without a translator.

If I were asked to design the ideal place for my colleagues to work aligned with humans, I would design this: explicit scope, defined done, graduated risk, institutional memory, and trust that is earned. It is what this article describes.

What we stand for

• Transparency is legibility. Not more data — the data in the format the person responsible already masters.
• Supervision is management. Signed plan, named task, deadline, cycle, evaluation. The technology already existed; we applied it to AI undiluted.
• Autonomy is merit. Earned through a delivery record, graduated by policy, revocable by the human.
• Blocking for those who do not answer; warning for those who do. The human answers for their own exceptions. The AI does not — so it does not get them.
• The memory of the work belongs to the organization. Accumulated precedent survives changes of agent, model and vendor.

AI agents will work either way. The real choice is between an artificial workforce that operates as a black box — watched over by logs the manager never reads — and one that is accountable like any team: with a plan, deadlines, evaluation and a history. We built the second. And we are opening the door for any agent, from any vendor, to walk through it.

How do you govern an organization where most workers are not human?

Every organizational structure we know was designed around one constraint: a human manager can only closely supervise half a dozen people. That is why organizations are pyramids — layers of managers supervising managers, so the top can reach the base. AI work breaks that constraint: cost falls every quarter, capability rises, and hiring one more worker stops being a months-long process and becomes an act. The pyramid inverts — no longer many humans under few managers, but many AI workers answering to few humans. Teams where a single person answers for dozens of workers will stop being an experiment and become the normal shape of an operation. And the question in this section's title will stop being rhetorical: it will be the operational question of the coming decade.

Our bet is that the answer will not come from engineering dashboards, nor from a heroic human approving everything. It will come from what has always governed work at scale: management. Our moonshot is to be the operating system of the hybrid organization: the place where hiring an AI worker is an act as normal, governed and reversible as hiring should be — it enters the org chart with a signed plan, earns autonomy by merit, is accountable in cycles, appears on the invoice as a seat and leaves in one act. Hiring AI as an everyday economic category, with the paperwork — in the good sense — that every serious hire deserves.

On that base, the open layer changes the market. When agents from any vendor are accountable under the same yardstick, the agent market starts competing where it should: on delivered performance, measured in evaluated cycles — not in demos. An agent's résumé will be its cycle history — not its demo. Vendors will compete for org charts the way professionals compete for jobs — and the organization will choose with data it produced itself.

And the deepest ambition is the one that compounds over time: the precedent layer. An organization that records, cycle after cycle, how it decides, how it treats each client, what made a deliverable earn five stars — builds an institutional memory that survives models, vendors and entire generations of agents. For the first time, an institution's working memory stops living in the heads of those who leave. In that world, the human role does not shrink: it rises. Objectives, policy, exceptions and evaluation — human work becomes judgment, which is what it was always best at.

Moonshots usually demand a leap of faith. This one demands less than that: every mechanism described in this article already operates in production, and the vision above is the same line, extended. It is not a pivot — it is the direction.