I've had a fleet of autonomous AI agents running in production since early 2025. They handle CRM email sync, document processing, analytics pipelines, LinkedIn inbox processing, and vault maintenance - running on a Quartz scheduler, backed by a Spring Boot MCP server, using Claude as the reasoning engine.
This article is the strategic companion to the technical MCP server architecture article. That article covers how to build the infrastructure. This one covers what you're actually building toward and what makes the difference between agents that work reliably and agents that are interesting demos.
What "Agent" Actually Means in This Context
There's a lot of loose usage of the word "agent" in AI right now. For this article, an agent is:
- A system prompt that tells an AI model what job it's doing
- A set of tools the model can call to take actions (read a file, send an email, update a database)
- A scheduler that fires the agent on a cadence
- Infrastructure that handles the tool calls and returns results to the model
The model reasons about what to do, decides which tools to call, and interprets the results. The infrastructure executes the tool calls. This is the key division: the AI reasons, the infrastructure acts.
This is meaningfully different from a chatbot (which responds to user input) and from a workflow system (which follows a fixed sequence of steps). An agent decides its own sequence of steps based on what it finds when it runs.
The Scheduler Is the Foundation
Agents need to fire on a schedule or in response to events. Without reliable scheduling, you don't have an autonomous agent - you have something you run manually.
I use Quartz Scheduler running inside the MCP server's Spring Boot application. Each agent definition maps to a Quartz job with a cron expression. The scheduler fires at the right time, reads the agent definition, builds the Claude API request, and submits it.
A few scheduling considerations that matter in production:
Jitter matters for high-frequency agents. If you have multiple agents that all fire at the top of the hour, you create a burst of load. Add a few minutes of jitter to spread them out. This is especially important if agents share downstream resources like databases or email servers.
Cron expressions are a liability. A cron expression that looks correct can have subtle issues (firing more or less frequently than expected, behaving differently across DST transitions). Test them carefully and log every firing.
Missed fire handling. Quartz has configurable misfire handling: what happens if the scheduler was down when a job was supposed to fire. For most CRM sync and analytics agents, "fire once when the scheduler comes back up" is the right behavior. For time-sensitive agents, "skip if missed" may be more appropriate.
The ${START_EPOCH} pattern. Each agent needs to know when it's running so it can calculate time-based cutoffs ("find emails received since the last run"). I interpolate the current Unix epoch into the system prompt at fire time. Agents append 000 to get milliseconds. This gives every agent a reliable "now" reference without a tool call.
Agent Definitions as Files
Rather than hard-coding agent logic in application code, each agent is a markdown file: YAML frontmatter for configuration, a system prompt in the body.
The frontmatter declares the agent ID, the Quartz cron schedule, and the list of tools the agent is allowed to use. The body is the system prompt that tells the model what job it's doing, what the expected output is, and any rules it should follow.
Defining agents as files pays off in three ways:
Version control. Every change to an agent's behavior is a git commit. When something goes wrong, git log tells you exactly what changed. git diff shows you the exact change.
No redeploy required. When an agent needs to be modified - new step added, behavior tweaked, tool access changed - you edit the file. The running application picks it up on the next fire without a deployment.
Easy inspection. When an agent misbehaves, you can read its definition and immediately understand what it's supposed to do. No code to trace, no compiled binary to decompile.
Tool Access Is Agent-Scoped
Each agent should have access to exactly the tools it needs and no more. Not the full tool catalog.
There are two reasons for this. The practical reason is that a large tool catalog creates more opportunity for wrong tool selection. An agent trying to sync CRM emails doesn't need the image-generation tools or the calendar tools. Giving it access to only the email and filesystem tools means it can't accidentally call something unrelated.
The governance reason is that it makes each agent's potential blast radius explicit and auditable. You can read an agent definition and know exactly what it can and can't do. An agent with access to every tool in the catalog can, in principle, do anything - including things you don't intend.
What Makes Agents Reliable
This is the most important part of the article.
Clear, specific system prompts. Vague instructions produce vague behavior. An agent that's told to "sync CRM contacts" will make different decisions in edge cases than an agent with a detailed spec of exactly what sync means, what constitutes a new email, how to handle duplicates, and what to write when done. Invest time in the system prompt. Treat it like a specification.
Structured outputs. Define the format of what the agent writes when it's done. If an agent produces a report, specify the format. If it updates a note, specify which fields. Unstructured output is hard to audit and harder to feed into downstream processes.
Idempotency. Agents will sometimes fire twice, or fire after a crash, or fire when the previous run left things in a partial state. Design agents to handle this gracefully: check what's already been done before doing it again, use cursors or cutoff dates to avoid reprocessing, write final state atomically.
Logging tool call chains. The most useful debugging information is the sequence of tool calls the agent made and what each one returned. Log this at the infrastructure level. When an agent does something unexpected, the tool call log tells you exactly where its reasoning diverged from what you intended.
Incremental scope. Start with agents that do one thing with a small blast radius. CRM sync that reads emails and writes notes is less risky than an agent that reads emails, writes notes, sends replies, and updates external systems. Add scope as you gain confidence in the agent's behavior.
What Makes Agents Unpredictable
Ambiguous tool descriptions. The model uses tool descriptions to decide when to call them. If two tools have similar descriptions, the model will sometimes call the wrong one. If a tool's description doesn't clearly specify its side effects, the model may call it when you don't intend. Write descriptions that are explicit about what a tool does and when to use it.
Too many tools. More tools means more decisions, more potential for wrong selection, and more cognitive load on the model. Keep the tool catalog focused.
Insufficient context about state. An agent that doesn't know what was done in previous runs will repeat work or skip work depending on what it finds. Give agents a clear "starting point" - a cursor, a timestamp, a last-processed marker - so they know where to begin.
Nondeterministic environments. If the data the agent reads changes between runs in unpredictable ways, the agent's behavior will appear to change even when the agent hasn't changed. Understand what your agents depend on and make sure that data is reliable.
Prompt Caching Is Worth the Setup Overhead
Anthropic's prompt caching caches system prompts across API calls with a 5-minute TTL. For agents that fire every 10-15 minutes (check-in agents, event processors), this means the system prompt is usually cached when the agent fires. Cache reads cost roughly 10% of a fresh write.
For long system prompts that fire frequently, this is meaningful cost reduction. Mark system prompts as cacheable in your API requests. Track your cache hit rate and adjust agent firing intervals to stay within the TTL when possible.
For agents that fire once an hour or once a day, the cache will usually be cold. The savings are smaller but still real for long prompts.
The Human-in-the-Loop Question
Autonomous agents that take actions without human review are appropriate for low-stakes, reversible operations: reading data, writing notes, generating reports. For higher-stakes operations - sending emails, modifying records that others depend on, triggering financial transactions - consider a human review step before the action executes.
The practical pattern is a "proposed actions" queue: the agent generates a list of proposed actions, writes them to a review file, and a human approves or rejects before execution. This preserves the efficiency of automation while keeping a human in the loop for consequential decisions.
As you gain confidence in an agent's behavior over time, you can reduce or eliminate the review step for specific action types. Start with review and relax it based on evidence, rather than starting without review and adding it after something goes wrong.