Build AI Agents
That Do Real Work
So you've got a working agent. Great!
Give it one job
Limit its tools
Check the result
The agent does a defined job and records what happened.
Here are some questions to consider before you take your agent to production:
- What happens when something goes wrong? Who owns the agent?
- What happens in edge cases that weren't covered by the prototype?
- How do you know if it's working? How will you monitor it?
- How much oversight will it need?
- Is it compliant with your organisation's security and data requirements?
Prototypes are useful for proving out how well an agent performs when it's given known inputs and expected outputs.
Production agents deal with the messy, complicated, unknown.
What is a production AI agent?
Production agents have more structure and oversight than prototypes. They can be implemented by adding guardrails to your prototype. Guardrails are a way of structuring agent development, implementation, and usage to ensure agents are performing tasks the way that your organisation requires. We recommend the following five guardrails for building and deploying agents to production.
- Clear job definition: An agent should be implemented to do one thing. You should be able to succinctly define its job in a single sentence. Examples could be triage my inbox, reconcile payments, qualify inbound leads, etc. If you need more than one sentence to define what an agent will do, break it into multiple agents.
- A small number of simple tools: Tools are APIs that your agent can call. Each tool should provide information that informs its reasoning. Anthropic's Model Context Protocol can simplify the use of tools, where appropriate. The output of a tool should be easy to interpret by your agent, and it's best to limit the number of tools.
- Guardrails: There should be limits to what an agent can do, and those limits should be hard-coded, not just in the prompt. Examples might be an agent can't give a refund over a specified ceiling, shouldn't access certain tables.
- Evals: Agents should pass a suite of evals before they're promoted to production.
- Human-readable audit logs: Agents should provide logs detailing everything they did during a given agent run.
How to take your prototype to production
We've built agents that do many of these things, we know what works and what doesn't, and we're happy to help. When you hire Vu to build a new agent as a service, we:
- Test the integration and any API calls
- Iterate on the prompt until it's working
- Use real-world cases where we know what should happen to ensure that the agent is working.
Book a call to discuss building out guardrails around your prototype.
We'll dig into:
- What is the agent's job? What does success look like?
- How often will it be running?
- What other systems does it need to pull data from to perform this task?
- What is the cost of getting it wrong?
After our conversation, we'll be able to determine the best path forward for your project, including if your agent will require oversight from a human in the loop. We'll also provide recommendations for additional guardrails.