Running Discovery for AI Agent Deployments

Every failed agent deployment I've seen — mine included — failed in discovery, weeks before anyone wrote code. The build was fine. The agent did what the spec said. The spec described a workflow that didn't exist.

Discovery for AI agents is harder than discovery for normal software, because the customer isn't asking for a feature — they're asking for a colleague. "We want an AI agent for customer support" carries a hundred unstated assumptions about judgment, tone, authority, and failure behavior. Surfacing those assumptions before you build is most of the forward deployed engineer's job.

Here's the playbook I've converged on after running this across voice agents, LLM pipelines, and automation deployments.

Step 1: Ignore the stated requirement (politely)

The first thing the customer tells you is a solution, not a problem. "We want an agent that answers emails." "We want a voice bot for inbound calls." Treat it as a symptom and work backwards.

The question that unlocks everything: "Walk me through the last time a human did this, end to end, including the annoying parts." Not the process doc. The last actual instance.

The gap between the two is where projects die. One deployment I worked on started as "AI should draft responses to customer emails." Walking through real cases with the ops team revealed that drafting was the easy 20% — the painful 80% was triage: figuring out which system the answer lived in, whether this customer had an open escalation, and who internally had to sign off. The stated requirement was a writing task. The real workflow was a routing-and-approval task with a small writing step at the end. Build the stated requirement and you automate the part that was never the problem.

Step 2: Shadow the workflow, don't survey it

Managers describe the workflow as designed. Operators run the workflow as it survived contact with reality. You need the second one, and you only get it by watching.

Concretely: get thirty minutes of screen-share with the person who does the job daily. Watch them handle five real items. You are looking for:

The invisible tabs. Every workflow has a spreadsheet, a Slack channel, or someone's memory that the process doc doesn't mention. That's real state your agent will need — or break.
The judgment calls. Where does the operator pause and think? Those moments are your hardest requirements and your best eval cases.
The exceptions bin. What do they do with the weird ones? However humans escalate today is the template for how your agent should escalate tomorrow.
The unofficial SLA. Not the contractual one — the one where a certain customer gets answered within the hour because everyone knows they're difficult.

Five shadowed cases beat fifty survey responses. Every time.

Step 3: Ask the questions specific to agents

Normal requirements gathering covers inputs, outputs, and volume. Agent deployments need a second layer of questions that customers have never been asked before, so budget real time for them:

"What is this agent allowed to do without a human looking?" The answer defines your autonomy boundary and, usually, your entire v1 architecture. Most enterprises say "nothing" at first — which is fine, because draft-and-approve is a great v1.
"What's the cost of a wrong answer here?" A wrong internal knowledge-base summary is annoying. A wrong quote to a customer is a lawsuit. Same agent skeleton, completely different eval bar and guardrails.
"Where does the truth live, and how stale is it?" Agents amplify data problems. If the knowledge base is 40% outdated, the agent will be confidently wrong 40% of the time, and it will be blamed — not the data.
"Who gets paged when it misbehaves?" If the answer is "nobody thought about that," you've just found your real project risk.
"How will you know it's working?" Push past "it saves time" to a number someone already tracks: handle time, backlog size, escalation rate. If no such number exists, define one now — it becomes your acceptance criterion and your renewal story.

Step 4: Translate into an agent spec

Discovery output should be an artifact the customer signs, not vibes in your notebook. My spec for any agent workflow fits on roughly a page:

agent: support-triage-v1
trigger: new email in shared inbox
inputs:
  - email thread (full history)
  - customer record (read-only, via MCP server)
  - order status (read-only, via MCP server)
allowed_actions:
  - classify: [billing, shipping, technical, other]
  - draft_reply           # never sends
  - escalate_to_human     # with a stated reason
forbidden:
  - sending anything externally
  - promising refunds or dates
  - touching accounts flagged "escalated"
escalation: anything below 0.8 classification confidence,
            or any mention of legal/cancellation
success_metric: median first-response time, measured weekly
review: human approves every draft for the first 4 weeks

Three things about this format matter more than the syntax. The forbidden list is where trust is won — customers relax visibly when they see you thought about what the agent must never do. The escalation clause encodes the exceptions bin from step 2. And allowed_actions maps one-to-one to tools, which means the spec is also your tool-design document — and narrow, intention-revealing tools are the foundation of a reliable production agent and a clean MCP server.

While you're here, collect twenty real historical cases with known-good outcomes. That's your golden eval set, and it costs nothing to gather during discovery and a lot to reconstruct later.

Step 5: Scope v1 like your reputation depends on it

It does. The wishlist from discovery is always a v3. Your job is to extract the v1, and my cutting rules are:

One workflow, fully. An agent that handles all of triage beats an agent that half-handles triage, drafting, and follow-ups. Breadth in v1 is how you ship three mediocre demos instead of one production system.
Human approval on every consequential action. Not because the model can't handle it — because adoption can't. The approval step is how the ops team learns to trust the agent, and their approvals/edits become your training signal for widening autonomy in v2.
Read integrations before write integrations. Reading the CRM is a security conversation. Writing to it is a security committee. Ship value on reads while the write approvals grind through.
Weeks, not months. If v1 can't ship in a few weeks, cut until it can. A live system generating feedback beats a comprehensive system generating slides. Adopting tools like Claude Code across my team cut our own iteration loop from weeks to hours; the deployment cadence should match.

Then say the quiet part in the kickoff deck: here is what v1 deliberately does not do, and here's when we revisit. Scope cuts that are hidden get discovered as disappointments. Scope cuts that are stated become the roadmap.

The red flags worth naming

Signals from discovery that predict a doomed deployment — worth raising early, even when it's awkward:

No operator in the room, ever. If you can't get access to the people who do the work, you're building from the org chart's imagination.
"The AI should just know." A stakeholder who resists defining escalation rules is telling you they expect magic. Reset expectations now or fail later.
A metric vacuum. Nobody can say what number improves. That project has no finish line and no renewal case.
The workflow is secretly political. Sometimes the "inefficient process" exists because two departments don't trust each other. An agent doesn't fix that; it just automates the distrust.

Flagging these is exactly what being the customer's trusted technical advisor means — the discovery phase is where that trust is either established or never happens.

The takeaway

Requirements gathering for AI agents is workflow archaeology: the real process is always buried under the stated one, and the artifacts that matter — the invisible spreadsheet, the judgment calls, the exceptions bin — only surface when you watch real work happen. Shadow the operators, ask the agent-specific questions, write a one-page spec with a forbidden list, harvest your eval set while you're there, and scope a v1 that ships in weeks with a human in the loop.

Do the archaeology properly and the build is almost boring. Skip it and no amount of engineering will save the deployment — you'll just be very efficiently automating the wrong thing.

Related reading