Smart Email Labeling Agent
Overview
Learn how to build an intelligent email classification agent that uses AI to automatically analyze and label incoming emails. This intermediate example showcases AgentMail’s powerful labeling feature combined with OpenAI’s GPT-4o-mini to create a sophisticated inbox automation system.
What You’ll Build
By the end of this guide, you’ll have a working smart labeling agent that:
- Receives incoming emails to a dedicated AgentMail inbox
- Analyzes each email with AI across 4 dimensions:
- Sentiment: positive, neutral, or negative
- Category: question, complaint, feature-request, bug-report, or praise
- Priority: urgent, high, normal, or low
- Department: sales, support, billing, or technical
- Automatically applies labels to each email for easy filtering
- Handles failures gracefully with retry logic and validation
Here’s what happens when an email arrives:
Prerequisites
Before you begin, make sure you have:
Required:
- Python 3.8 or higher installed
- An AgentMail account and API key
- An OpenAI API key (for AI classification)
- An ngrok account
Project Setup
Step 1: Create Project Directory
Create a new directory for your agent:
Step 2: Create the Agent Code
Create a file named agent.py and paste the following code:
Click to view full agent.py code
Step 3: Create Requirements File
Create a file named requirements.txt:
Step 4: Install Dependencies
Install the required Python packages:
Step 5: Configure Environment Variables
Create a .env file with your credentials:
Code Walkthrough
Let’s understand how the agent works by breaking down the key components.
Architecture Overview
1. Initialization
2. Setting Up Infrastructure
The setup_agentmail() function creates your inbox and webhook:
Idempotency with client_id:
Using client_id ensures you can safely restart the agent without creating duplicate inboxes or webhooks. If a resource already exists, AgentMail returns the existing one.
3. AI-Powered Email Analysis
The analyze_email() function is the core of the agent:
Key features:
- Structured prompt: Clearly defines the 4 classification dimensions
- JSON mode: Forces OpenAI to return valid JSON
- Retry logic: Automatically retries up to 3 times on failures
- Strict validation: Ensures classifications match expected values
- Low temperature (0.3): Consistent, predictable classifications
Example OpenAI prompt:
AI Response:
4. Applying Labels
The apply_labels() function applies labels with a two-tier strategy:
Why two tiers?
- Batch first: Fastest approach (1 API call for all labels)
- Individual fallback: If batch fails, try each label separately to save what we can
- Resilient: Won’t fail completely if one label has an issue
5. Processing Webhooks
The receive_webhook() function orchestrates everything:
Why always return 200?
Even if classification or labeling fails, we return 200 to tell AgentMail: “I received this webhook.” If we returned an error (400/500), AgentMail would retry sending the webhook, which doesn’t help with application errors.
Running the Agent
Start the agent:
You should see output like this:
Success! Your agent is now running and ready to classify emails.
Leave this terminal window open - closing it will stop the agent.
Testing Your Agent
Let’s test the agent with different types of emails to see how it classifies them.
Example 1: Urgent Complaint
Send this email:
Console Output:
Why this classification?
- Sentiment: negative (words: “terrible”, “lost work”)
- Category: complaint (expressing dissatisfaction)
- Priority: urgent (keywords: “IMMEDIATELY”, “crashed 3 times”)
- Department: support (product issue)
Example 2: Feature Request
Send this email:
Console Output:
Why this classification?
- Sentiment: positive (words: “love”, “amazing”, “great work”)
- Category: feature-request (suggesting new functionality)
- Priority: normal (no urgency indicators)
- Department: technical (feature implementation)
What Happens Next?
View in Dashboard
Go to your AgentMail inbox and filter by labels to organize your emails:
Filter by sentiment:
- Search
negativeto see all unhappy customers - Search
positiveto find praise and testimonials - Search
neutralto review informational emails
Filter by priority, by department…
Combine filters for powerful queries:
urgent+negative→ Critical customer issuessales+high→ Hot leads requiring fast responsetechnical+bug-report→ Engineering backlog
Pro tip: You can use AgentMail’s API to programmatically fetch emails by labels and build custom workflows, dashboards, and analytics.
Building on Labels
Once your emails are automatically labeled, you can build powerful automation on top:
Example 1: Priority Notifications
Scenario: Alert your team instantly when urgent issues arrive.
When an email is labeled urgent + negative, automatically send a Slack notification to your support channel. Include the sender’s email and subject line so your team can respond immediately. This ensures critical customer issues never slip through the cracks.
Example 2: Sentiment Escalation
Scenario: Escalate negative sentiment to management.
Track all emails labeled negative and automatically notify customer success managers when sentiment trends downward. If a single customer sends 3+ negative emails in a week, trigger a personal outreach from leadership to address their concerns proactively.
Example 3: Department Routing
Scenario: Auto-forward emails to the right team.
Create rules that automatically forward emails based on department labels:
sales→ Forward to sales@yourcompany.combilling→ Create a ticket in your billing systemtechnical→ Post to #engineering Slack channelsupport→ Add to support queue with appropriate SLA
Example 4: Smart Auto-Reply
Scenario: Send contextual automated responses.
Instead of generic auto-replies, craft responses based on classification:
question→ “Thanks for your question! We’ll respond within 24 hours.”bug-report→ “Thanks for reporting this. Our engineering team has been notified.”complaint→ “We’re sorry to hear that. A senior support agent will contact you within 4 hours.”feature-request→ “Great idea! We’ve added this to our product roadmap.”
Example 5: Analytics Dashboard
Scenario: Track email metrics and trends.
Build a dashboard that queries emails by labels to analyze:
- Sentiment trends: Are customers getting happier or more frustrated?
- Volume by department: Which team is handling the most emails?
- Response times by priority: Are you meeting SLAs for urgent issues?
- Common categories: What are customers asking about most?
Use this data to identify bottlenecks, improve processes, and make data-driven decisions about staffing and product priorities.
Congratulations! You’ve built an AI-powered email classification system. This agent showcases how AgentMail’s labeling feature can power sophisticated inbox automation and analytics.
