Private Email Triage with IMAP and a Local AI Agent

Your email stays where it is. Gmail, Outlook, Fastmail, a self-hosted server: whatever you use today continues to work exactly as it does. The system connects over IMAP (the same protocol every email client uses) and adds an intelligence layer on top. That layer does three things. It classifies incoming messages so you know what matters before you open your inbox. It applies per-sender rules that handle routine mail automatically: archiving, flagging, deferring, or drafting a reply. And it gives you a morning briefing that surfaces only the messages that need your attention, delivered through whatever channel you already check first: Telegram, Signal, Discord, or email itself. The point is not to fire off autonomous replies. The point is to reduce the cognitive overhead of email while keeping you in the loop. Every draft waits for your approval. Every action is logged. The system makes email manageable without taking control away from you.

The morning starts with a summary. The agent checked your inbox overnight and grouped what arrived into categories: messages that need a reply, messages that are informational, messages that match your archive rules, and anything flagged as unusual. The summary arrives in your messaging app. You read it over coffee instead of opening your inbox cold. Sender rules handle the predictable stuff. Your bank sends transaction alerts: archived automatically. A specific client always needs a response within 24 hours: flagged and moved to a priority folder. Newsletters you actually read: left alone. Newsletters you don't: archived. You set these rules by telling the agent in plain language: "archive anything from notifications@bank.com" or "flag emails from sarah@ and draft a reply." The rules live in a configuration file the agent maintains. When you need something specific, you ask. "What did the accountant send last week?" pulls the relevant messages. "Draft a reply to the permit office saying we'll have the documents by Friday" produces a draft you can review and send. "Move everything from that recruiter to trash" does what it says. The agent has access to the same inbox you do, through the same protocol, so it can search, read, move, flag, and draft without you switching to a different interface. Drafts are the most useful daily feature. The agent learns your voice from your sent mail. When it drafts a reply to a routine request, it sounds like you wrote it, not like a template. You review, edit if needed, and approve. The agent sends it through your existing SMTP server.

The system connects to your mail server over IMAP using standard libraries ( in Node.js, in Python). It authenticates with your credentials, which stay on your machine. No OAuth redirect to a third-party service. No API key shared with a cloud provider. On each check (typically every 15 to 60 minutes via an OpenClaw cron job), the agent fetches new message envelopes: sender, subject, date, flags. It does not download full message bodies unless classification or drafting requires it. This keeps bandwidth and storage minimal. Classification runs in two stages. First, a deterministic pass against the sender rules file: exact domain matches, regex patterns, known contacts. This handles 60 to 80 percent of incoming mail without involving a language model at all. Messages that don't match any rule go through a second pass where the LLM reads the subject and (if needed) the body to classify intent: needs reply, informational, promotional, transactional, or unknown. State lives in SQLite. Every message the system has seen gets a row: message ID, classification, actions taken, draft status. This makes deduplication trivial (IMAP message IDs are unique) and gives you a queryable history of what the agent did and why. The CLI returns structured JSON for every query: , , , , , , . The sender rules file is the core configuration. It maps senders (by exact address or domain pattern) to actions: archive, flag, defer, draft-reply, ignore, or a custom handler. Rules are ordered by specificity. The agent can update this file when you tell it to add or change a rule. The format is human-readable so you can audit it directly. Draft generation uses the language model. The agent reads the incoming message, pulls relevant context (previous thread, sender history, any related calendar events or documents), and produces a draft. The draft is stored locally and presented for approval. Only after explicit approval does the agent send via SMTP. This is the one place where the LLM does substantive work: understanding intent and composing a response. OpenClaw ties it together. The email CLI is wrapped as an OpenClaw plugin with a typed schema. Cron jobs run the inbox check and morning briefing on schedule. The messaging layer delivers summaries and draft notifications to whichever channel you prefer. Cross-tool orchestration means an email about a meeting can trigger a calendar lookup, an email about a lab result can trigger the health pipeline, and an email about a deadline can create a reminder, all in the same agent turn. The integration point matters. Email is not isolated. An email from your clinic with a lab report attachment gets parsed by the health ingestion pipeline. An email confirming a flight gets cross-referenced with your calendar. An email from a client mentioning a contract gets linked to the document search system. The email agent is one surface into the same connected system that handles everything else.

The sender rules engine comes first. Before any AI is involved, build the deterministic classification layer. A config file that maps sender patterns to actions, a function that evaluates rules in order, and tests that verify the right action fires for each pattern. This handles the majority of email volume and gives you a working system before the language model enters the picture. IMAP connectivity is the second step, and it's where most people underestimate the complexity. IMAP is a stateful protocol with quirks: folder naming varies by provider (Gmail uses labels, others use folders), message IDs are not always sequential, and connection handling needs retry logic for flaky networks. Use a library that handles the protocol layer ( is the most reliable Node.js option) and write integration tests against a real mailbox, not mocks. If you skip this, you'll debug protocol issues in production. The CLI comes third. Subcommands for (list envelopes), (by sender, subject, date), (full message), (flags), (between folders), , and . Every subcommand returns structured JSON. The agent never parses human-readable email output. This is the same pattern every Siam AI Lab system follows: deterministic CLI with JSON output, wrapped by a plugin, called by the agent. LLM classification and drafting layer on top of the deterministic base. Messages that pass through sender rules without a match get classified by the model. Draft generation reads thread context and produces a response. Both of these are the last thing you build, not the first. The system is useful before the LLM is involved. The LLM makes it better. The OpenClaw plugin wrapper registers the CLI as a tool with typed parameters and return schemas. Cron jobs schedule the inbox check (every 15 to 60 minutes) and the morning briefing (once daily at a configured time). The messaging integration delivers summaries and draft approvals to your preferred channel. Testing matters more here than in most systems because email is high-stakes. A misclassified health alert or an auto-archived client email is a real problem. Test sender rules exhaustively. Test classification against a labeled sample of your own email. Test draft quality by comparing agent drafts to your actual sent replies. If the drafts don't sound like you after tuning, the model needs more context from your sent mail history.

Most of the email system is deterministic. Sender rule matching, IMAP operations, folder management, flag updates, search queries: none of these involve a language model. The model handles two things: classifying messages that don't match any sender rule, and generating reply drafts. For classification, the model needs to read a subject line and sometimes a message body and assign a category. This is a structured output task. It doesn't require a large model or vision capabilities. A smaller, faster model works well here because latency on classification directly affects how quickly the morning briefing is ready. For drafting, the model needs to understand conversational context, match your writing style, and produce a coherent reply. This benefits from a more capable model, especially for nuanced business correspondence. The system can use different models for different tasks: a fast model for classification, a more capable one for drafting. See also: Choosing Models for Private AI Systems for the full reference.

Email is the primary vector for phishing, spam, and social engineering. Any system that reads email programmatically inherits that attack surface. The mitigation is proportional, not paranoid. Sender filtering is the first line. The deterministic rules engine processes known senders before the LLM ever sees a message. Unknown senders get classified but the agent does not auto-draft replies to them. This prevents the most obvious prompt injection vector: a crafted email designed to manipulate the agent's context. Message bodies from unknown senders should be treated as untrusted content entering the agent's context. If the system uses the message body for classification, it should extract text only (no HTML rendering, no remote image loading, no link following). The LLM reads plain text, not rendered email. Credentials stay local. IMAP and SMTP passwords or OAuth tokens live in the system's config file on the local machine. They are never sent to an external service. Disk encryption and filesystem permissions protect them at rest. Draft approval is a security feature, not just a UX choice. The agent never sends email autonomously by default. Every outgoing message requires explicit human approval. This prevents both accidental sends and any scenario where a manipulated context produces an unintended reply. Audit logging covers every action: messages read, classifications applied, rules triggered, drafts generated, sends approved. The log lives in SQLite alongside the message state. If something goes wrong, you can trace exactly what happened. See also: Security Considerations for Private AI Systems for the full reference.

The minimum useful version is sender rules and a morning briefing. No LLM, no drafting. Just deterministic classification of known senders and a daily summary delivered to your messaging app. This is already valuable because it means you never open your inbox cold. After using that for a week, you know which messages fall through the rules. That's when LLM classification earns its place: handling the 20 to 40 percent of mail that doesn't match a pattern. The model classifies by intent and the briefing gets smarter. Draft assistance comes next. Start with the senders you reply to most often with similar messages. Tune the drafts against your actual sent mail until they sound right. Expand to more senders as confidence grows. The plugin and cron jobs come in when the system is stable. Wrapping the CLI in an OpenClaw plugin means other parts of the system can trigger email actions: a calendar event can prompt a meeting prep email, a completed task can trigger a status update. Cron scheduling automates the check cycle. Later refinements: thread summarization for long email chains, attachment handling (download and route to the appropriate pipeline), priority scoring based on response time history, and integration with calendar and document systems for richer context in drafts.

A freelance consultant in Bangkok starts every day with 80 to 120 emails. About half are transactional notifications, a quarter are newsletters, and the rest need actual attention. The morning briefing cuts through all of it: three client emails need replies, one invoice arrived, and a flight confirmation for next week. Everything else is already archived or filed. The inbox that used to take 45 minutes to process takes five. Someone managing a property search across multiple real estate agents gets listing notifications from six different platforms. The agent files them by source, flags listings that match saved criteria (price range, location, size), and surfaces only the matches in the morning summary. Listings that don't match get archived without cluttering the inbox. A person coordinating medical care across three providers in different cities gets appointment confirmations, lab result notifications, and insurance correspondence. The agent routes lab results to the health pipeline for parsing, creates calendar events from appointment confirmations, and flags insurance emails that need a response. The email becomes an ingestion surface for the health system rather than a dead-end archive. An OpenClaw user who already has health tracking, calendar management, and document search running adds email as another connected surface. The agent now has the full picture: an email about a deadline triggers a calendar check, an email with an attachment triggers document ingestion, and the morning briefing includes email alongside sleep data and today's schedule. One system, one briefing, no app-switching.

A law firm's managing partner receives 200 emails a day. Half are internal, a quarter are from opposing counsel, and the rest are administrative. The system classifies by sender and routes: internal emails get filed by matter, opposing counsel emails get flagged for immediate attention and linked to the relevant case in the document system, and administrative emails get handled by sender rules. Drafts for routine responses (scheduling, acknowledgments, document requests) are generated and queued for approval. Client communications never leave the firm's server. A dental practice in Bangkok receives appointment requests, insurance verifications, and patient questions by email. The agent classifies each type, routes appointment requests to the scheduling system, flags insurance emails that need manual verification, and drafts acknowledgment replies to patients. The practice's email address becomes a functional intake surface instead of a pile that someone triages manually every morning. A consulting firm's research team receives industry reports, news digests, and client correspondence. The agent files research content by topic, summarizes long reports into digest form, and flags client emails that reference active engagements. When a consultant asks "what did we get from the energy sector this week?" the agent searches across email, filed documents, and ingested reports to give a complete answer. A small business owner who handles sales, support, and operations from one inbox uses sender rules to separate the three streams. Sales inquiries get a draft response with availability and pricing template. Support requests get classified by urgency and routed to the appropriate person. Operations emails (invoices, shipping confirmations, vendor communications) get filed automatically. The morning briefing shows the business owner exactly which emails need their attention and in what order.