Human in the Loop: Why Agents Should Draft and Wait

This morning, while checking my local AI setup for my studio in Munich, I noticed an email draft sitting in my queue. It was a proposal I'd asked my agent to draft based on a client's initial inquiry. The draft wasn't perfect, but it captured the essence and the key points we needed to discuss. As I read through it, I felt grateful for the human-in-the-loop pattern that keeps my AI setup safe and effective. This simple yet powerful rule ensures that every action taken by an agent is first vetted by me, preventing any costly mistakes from happening.

The core of this pattern is straightforward: agents in my system draft responses or actions but never send them without human approval. When I receive a new input, like an email or order, I instruct my reader component to pull it into the system. Once inside, the drafter analyzes the input and generates a draft response or action based on predefined rules and guidelines. These drafts are then placed in a queue where they wait until I review them.

Why do I insist on this process? Because the cost of a mistake can be enormous when it comes to external actions. A wrong tweet can damage my reputation, an erroneous order can result in financial loss, and an ill-timed email can jeopardize relationships. Drafts, however, are cheap. They take seconds to read and discard if they're not up to par. This means that by reviewing drafts before taking any action, I'm able to catch errors early and correct them without significant consequences.

For instance, when my agent generates a draft for an email reply, it includes all the necessary details but doesn't hit send. Instead, the draft is stored in the queue until I log in and review it. If everything looks good, I approve the draft, and the separate sending mechanism takes over to ensure it's dispatched correctly. This process might seem cumbersome at first, but it significantly reduces the risk of errors while still providing a level of automation that simplify my workflow.

Some people might argue that this approach isn't true automation because it requires human intervention. However, I see it differently. My role has shifted from writing entire responses from scratch to simply reviewing and approving drafts. This change saves me time and effort while maintaining control over the final outcome. The initial setup of training my agents to generate high-quality drafts also pays off in the long run, as these drafts often require minimal editing.

Another common objection is that agents should eventually become trusted enough to act autonomously. In theory, this might sound appealing, but in practice, it's a risky proposition. Even if an agent performs flawlessly for weeks or months, there's always the chance of a sudden error or unexpected situation leading to a costly mistake. By keeping drafts separate from actions, I maintain a safety net that protects me from such unforeseen issues.

There is one exception to this rule: internal actions that have no external impact and can be easily reversed if necessary. For example, my backup agent automatically saves my database without needing approval because the worst-case scenario would be recovering data manually, which isn't too difficult. Similarly, the summary of daily emails generated for me doesn't need vetting since it's purely informational.

In contrast, any action that involves sending information to external parties or making changes that could affect others, like posting on social media or deploying code, goes through my review queue first. This ensures that I'm always in control and can prevent potential disasters before they occur.

By adhering to this human-in-the-loop pattern, I've found a balance between using the power of AI for efficiency and maintaining full oversight to avoid costly errors. It's not about eliminating mistakes entirely but rather minimizing their impact through careful planning and review processes. This approach has allowed me to trust my local AI agents while ensuring they remain tools that serve my needs without causing harm.

As I continue to refine my setup, I find myself reflecting on the simplicity of this pattern. In a world where complex solutions often dominate discussions about technology and automation, it's refreshing to rely on a straightforward yet effective principle. The human-in-the-loop pattern is not just a technical solution but a mindset that values caution and control in our increasingly automated lives.