> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.agentmail.to/llms.txt. For full content including API reference and SDK examples, see https://docs.agentmail.to/llms-full.txt.

# Smart Email Labeling Agent

> Build an AI-powered agent that automatically classifies and labels incoming emails across multiple dimensions

## Overview

Learn how to build an intelligent email classification agent that uses AI to automatically analyze and label incoming emails. This intermediate example showcases AgentMail's powerful labeling feature combined with OpenAI's GPT-4o-mini to create a sophisticated inbox automation system.

## What You'll Build

By the end of this guide, you'll have a working smart labeling agent that:

1. **Receives incoming emails** to a dedicated AgentMail inbox
2. **Analyzes each email with AI** across 4 dimensions:
   * **Sentiment**: positive, neutral, or negative
   * **Category**: question, complaint, feature-request, bug-report, or praise
   * **Priority**: urgent, high, normal, or low
   * **Department**: sales, support, billing, or technical
3. **Automatically applies labels** to each email for easy filtering
4. **Handles failures gracefully** with retry logic and validation

Here's what happens when an email arrives:

```
Email: "Your product crashed! I need help ASAP!"
         ↓
   AI Analysis
         ↓
   Applied Labels:
   • negative
   • complaint
   • urgent
   • support
```

## Prerequisites

Before you begin, make sure you have:

**Required:**

* Python 3.8 or higher installed
* An [AgentMail account](https://agentmail.to) and API key
* An [OpenAI API key](https://platform.openai.com/api-keys) (for AI classification)
* An [ngrok account](https://ngrok.com)

## Project Setup

### Step 1: Create Project Directory

Create a new directory for your agent:

```bash
mkdir smart-labeling-agent
cd smart-labeling-agent
```

### Step 2: Create the Agent Code

Create a file named `agent.py` and paste the following code:

#### Click to view full agent.py code

```python
"""
Smart Email Labeling Agent

An AI-powered email classification agent that automatically analyzes incoming
emails across multiple dimensions and applies appropriate labels.
"""

import os
import json
import time
from dotenv import load_dotenv

load_dotenv()

from flask import Flask, request, Response
import ngrok
from agentmail import AgentMail
from openai import OpenAI

# Configuration
PORT = int(os.getenv("PORT", "8080"))
INBOX_USERNAME = os.getenv("INBOX_USERNAME", "smart-labels")
WEBHOOK_DOMAIN = os.getenv("WEBHOOK_DOMAIN")

# Initialize
app = Flask(__name__)
client = AgentMail()
openai_client = OpenAI()


def setup_agentmail():
    """Create inbox and webhook with idempotency."""
    # Create inbox
    try:
        inbox = client.inboxes.create(
            username=INBOX_USERNAME,
            client_id=f"{INBOX_USERNAME}-inbox"
        )
    except Exception as e:
        if "already exists" in str(e).lower():
            inbox_id = f"{INBOX_USERNAME}@agentmail.to"
            class SimpleInbox:
                def __init__(self, inbox_id):
                    self.inbox_id = inbox_id
            inbox = SimpleInbox(inbox_id)
        else:
            raise

    # Start ngrok
    listener = ngrok.forward(PORT, domain=WEBHOOK_DOMAIN, authtoken_from_env=True)

    # Create webhook
    try:
        client.webhooks.create(
            url=f"{listener.url()}/webhook/agentmail",
            event_types=["message.received"],
            client_id=f"{INBOX_USERNAME}-webhook"
        )
    except Exception as e:
        if "already exists" not in str(e).lower():
            raise

    print(f"Ready: {inbox.inbox_id}\n")
    return inbox, listener


def analyze_email(subject, content):
    """Use AI to classify email across multiple dimensions with retry logic."""
    valid_values = {
        "sentiment": {"positive", "neutral", "negative"},
        "category": {"question", "complaint", "feature-request", "bug-report", "praise"},
        "priority": {"urgent", "high", "normal", "low"},
        "department": {"sales", "support", "billing", "technical"}
    }

    for attempt in range(1, 4):
        try:
            if attempt > 1:
                time.sleep(1)

            response = openai_client.chat.completions.create(
                model="gpt-4o-mini",
                messages=[
                    {
                        "role": "system",
                        "content": "You are an expert email classifier. Analyze emails and return structured classifications."
                    },
                    {
                        "role": "user",
                        "content": f"""Analyze this email across 4 dimensions:

                        Subject: {subject}
                        Content: {content}

                        Classify into:
                        1. sentiment: positive | neutral | negative
                        2. category: question | complaint | feature-request | bug-report | praise
                        3. priority: urgent | high | normal | low
                        4. department: sales | support | billing | technical

                        Consider:
                        - Sentiment: Overall tone and emotion
                        - Category: Primary intent of the email
                        - Priority: Urgency indicators (ASAP, urgent, immediately, deadline mentions, emergency)
                        - Department: Best team to handle this

                        Return ONLY valid JSON with these exact keys: sentiment, category, priority, department.
                        Example: {{"sentiment": "positive", "category": "question", "priority": "normal", "department": "sales"}}
                        """
                    }
                ],
                response_format={"type": "json_object"},
                temperature=0.3
            )

            # Parse and validate
            result = json.loads(response.choices[0].message.content)

            required_keys = ["sentiment", "category", "priority", "department"]
            missing_keys = [key for key in required_keys if key not in result]
            if missing_keys:
                raise ValueError(f"Missing keys: {missing_keys}")

            invalid_values = []
            for dimension, value in result.items():
                if dimension in valid_values and value not in valid_values[dimension]:
                    invalid_values.append(f"{dimension}={value}")

            if invalid_values:
                raise ValueError(f"Invalid values: {', '.join(invalid_values)}")

            return result

        except Exception as e:
            if attempt == 3:
                raise Exception(f"AI classification failed: {e}")


def apply_labels(inbox_id, message_id, classifications):
    """Apply labels based on classification results."""
    labels = [
        f"{classifications['sentiment']}",
        f"{classifications['category']}",
        f"{classifications['priority']}",
        f"{classifications['department']}"
    ]

    # Try batch first
    try:
        client.inboxes.messages.update(
            inbox_id=inbox_id,
            message_id=message_id,
            add_labels=labels
        )
        for label in labels:
            print(f"  ✓ {label}")
        return
    except Exception:
        pass

    # Try individually
    successful = []
    for label in labels:
        try:
            client.inboxes.messages.update(
                inbox_id=inbox_id,
                message_id=message_id,
                add_labels=[label]
            )
            successful.append(label)
            print(f"  ✓ {label}")
        except Exception:
            print(f"  ✗ {label}")

    if not successful:
        raise Exception("Failed to apply labels")


@app.route('/webhook/agentmail', methods=['POST'])
def receive_webhook():
    """Webhook endpoint to receive incoming email notifications."""
    try:
        payload = request.json
        event_type = payload.get('type') or payload.get('event_type')

        # Ignore outgoing messages
        if event_type == 'message.sent':
            return Response(status=200)

        message = payload.get('message', {})
        message_id = message.get('message_id')
        inbox_id = message.get('inbox_id')
        from_field = message.get('from_', '') or message.get('from', '')

        # Validate required fields
        if not message_id or not inbox_id or not from_field:
            return Response(status=200)

        # Extract sender email
        if '<' in from_field and '>' in from_field:
            sender_email = from_field.split('<')[1].split('>')[0].strip()
        else:
            sender_email = from_field.strip()

        subject = message.get('subject', '(no subject)')
        email_body = message.get('text', '') or message.get('body', '') or message.get('html', '')

        # Log
        print(f"\n📧 {sender_email}: {subject}")

        # Analyze
        classifications = analyze_email(subject, email_body)

        print(f"  Sentiment: {classifications['sentiment']}")
        print(f"  Category: {classifications['category']}")
        print(f"  Priority: {classifications['priority']}")
        print(f"  Department: {classifications['department']}")

        # Apply labels
        apply_labels(inbox_id, message_id, classifications)
        print("Done\n")

    except Exception as e:
        print(f"Error: {e}\n")

    return Response(status=200)


if __name__ == '__main__':
    print("SMART EMAIL LABELING AGENT\n")
    inbox, listener = setup_agentmail()
    print("Waiting for emails...\n")
    app.run(port=PORT)
```

### Step 3: Create Requirements File

Create a file named `requirements.txt`:

```txt
agentmail
flask>=3.0.0
ngrok>=1.0.0
python-dotenv>=1.0.0
openai>=1.0.0
```

### Step 4: Install Dependencies

Install the required Python packages:

```bash
pip install -r requirements.txt
```

### Step 5: Configure Environment Variables

Create a `.env` file with your credentials:

```env
# AgentMail Configuration
AGENTMAIL_API_KEY=your_agentmail_api_key_here

# OpenAI Configuration
OPENAI_API_KEY=your_openai_api_key_here

# Ngrok Configuration
NGROK_AUTHTOKEN=your_ngrok_authtoken_here
WEBHOOK_DOMAIN=your-domain.ngrok-free.app

# Agent Settings
INBOX_USERNAME=smart-labels
PORT=8080
```

## Code Walkthrough

Let's understand how the agent works by breaking down the key components.

### Architecture Overview

```
Email arrives → AgentMail → Webhook → ngrok → Flask → AI Analysis → Apply Labels
```

### 1. Initialization

```python
# Load environment variables first
load_dotenv()

# Initialize three clients
app = Flask(__name__)           # Web server for webhooks
client = AgentMail()            # AgentMail SDK
openai_client = OpenAI()        # OpenAI for AI classification
```

### 2. Setting Up Infrastructure

The `setup_agentmail()` function creates your inbox and webhook:

```python
# Create inbox with idempotency
inbox = client.inboxes.create(
    username=INBOX_USERNAME,
    client_id=f"{INBOX_USERNAME}-inbox"  # Prevents duplicates
)

# Start ngrok tunnel (localhost → public URL)
listener = ngrok.forward(PORT, domain=WEBHOOK_DOMAIN)

# Register webhook with AgentMail
client.webhooks.create(
    url=f"{listener.url()}/webhook/agentmail",
    event_types=["message.received"],
    client_id=f"{INBOX_USERNAME}-webhook"
)
```

**Idempotency with `client_id`:**

Using `client_id` ensures you can safely restart the agent without creating duplicate inboxes or webhooks. If a resource already exists, AgentMail returns the existing one.

### 3. AI-Powered Email Analysis

The `analyze_email()` function is the core of the agent:

```python
def analyze_email(subject, content):
    # Define valid classification values
    valid_values = {
        "sentiment": {"positive", "neutral", "negative"},
        "category": {"question", "complaint", "feature-request", ...},
        "priority": {"urgent", "high", "normal", "low"},
        "department": {"sales", "support", "billing", "technical"}
    }

    # Retry up to 3 times
    for attempt in range(1, 4):
        try:
            # Call OpenAI API
            response = openai_client.chat.completions.create(...)

            # Parse JSON response
            result = json.loads(response.choices[0].message.content)

            # Validate keys and values
            # ... validation logic ...

            return result
        except Exception as e:
            if attempt == 3:
                raise
```

**Key features:**

1. **Structured prompt**: Clearly defines the 4 classification dimensions
2. **JSON mode**: Forces OpenAI to return valid JSON
3. **Retry logic**: Automatically retries up to 3 times on failures
4. **Strict validation**: Ensures classifications match expected values
5. **Low temperature (0.3)**: Consistent, predictable classifications

**Example OpenAI prompt:**

```
Analyze this email across 4 dimensions:

Subject: Product keeps crashing!
Content: Your software is terrible. Fix it ASAP!

Classify into:
1. sentiment: positive | neutral | negative
2. category: question | complaint | feature-request | bug-report | praise
3. priority: urgent | high | normal | low
4. department: sales | support | billing | technical

Return ONLY valid JSON...
```

**AI Response:**

```json
{
  "sentiment": "negative",
  "category": "complaint",
  "priority": "urgent",
  "department": "support"
}
```

### 4. Applying Labels

The `apply_labels()` function applies labels with a two-tier strategy:

```python
# Create labels from classification values
labels = [
    "negative",
    "complaint",
    "urgent",
    "support"
]

# Try batch application first (most efficient)
try:
    client.inboxes.messages.update(
        inbox_id=inbox_id,
        message_id=message_id,
        add_labels=labels  # Apply all at once
    )
    return
except Exception:
    pass  # If batch fails, try individually

# Apply labels one by one
for label in labels:
    try:
        client.inboxes.messages.update(
            inbox_id=inbox_id,
            message_id=message_id,
            add_labels=[label]  # One at a time
        )
    except Exception:
        pass  # Log failure but continue
```

**Why two tiers?**

1. **Batch first**: Fastest approach (1 API call for all labels)
2. **Individual fallback**: If batch fails, try each label separately to save what we can
3. **Resilient**: Won't fail completely if one label has an issue

### 5. Processing Webhooks

The `receive_webhook()` function orchestrates everything:

```python
@app.route('/webhook/agentmail', methods=['POST'])
def receive_webhook():
    payload = request.json

    # Ignore outgoing messages (prevents infinite loops)
    if event_type == 'message.sent':
        return Response(status=200)

    # Extract email data
    message_id = message.get('message_id')
    inbox_id = message.get('inbox_id')
    sender_email = extract_email(from_field)
    subject = message.get('subject')
    email_body = message.get('text')

    # Classify with AI
    classifications = analyze_email(subject, email_body)

    # Apply labels
    apply_labels(inbox_id, message_id, classifications)

    # Always return 200 (tells AgentMail we received it)
    return Response(status=200)
```

**Why always return 200?**

Even if classification or labeling fails, we return `200` to tell AgentMail: "I received this webhook." If we returned an error (400/500), AgentMail would retry sending the webhook, which doesn't help with application errors.

## Running the Agent

Start the agent:

```bash
python agent.py
```

You should see output like this:

```
SMART EMAIL LABELING AGENT

Ready: smart-labels@agentmail.to

Waiting for emails...

 * Running on http://127.0.0.1:8080
```

**Success!** Your agent is now running and ready to classify emails.

Leave this terminal window open - closing it will stop the agent.

## Testing Your Agent

Let's test the agent with different types of emails to see how it classifies them.

### Example 1: Urgent Complaint

**Send this email:**

```
To: smart-labels@agentmail.to
Subject: Product crashed - need immediate help!
Body: Your product is TERRIBLE! It crashed 3 times today and I lost all my work.
      I need this fixed IMMEDIATELY or I want a full refund!
```

**Console Output:**

```
you@example.com: Product crashed - need immediate help!
  Sentiment: negative
  Category: complaint
  Priority: urgent
  Department: support
  ✓ negative
  ✓ complaint
  ✓ urgent
  ✓ support
Done
```

**Why this classification?**

* **Sentiment**: negative (words: "terrible", "lost work")
* **Category**: complaint (expressing dissatisfaction)
* **Priority**: urgent (keywords: "IMMEDIATELY", "crashed 3 times")
* **Department**: support (product issue)

### Example 2: Feature Request

**Send this email:**

```
To: smart-labels@agentmail.to
Subject: Dark mode would be amazing!
Body: Hi! I absolutely love your product. I use it every day.
      One feature that would make it even better is dark mode support.
      Keep up the great work!
```

**Console Output:**

```
you@example.com: Dark mode would be amazing!
  Sentiment: positive
  Category: feature-request
  Priority: normal
  Department: technical
  ✓ positive
  ✓ feature-request
  ✓ normal
  ✓ technical
Done
```

**Why this classification?**

* **Sentiment**: positive (words: "love", "amazing", "great work")
* **Category**: feature-request (suggesting new functionality)
* **Priority**: normal (no urgency indicators)
* **Department**: technical (feature implementation)

## What Happens Next?

### View in Dashboard

Go to your AgentMail inbox and filter by labels to organize your emails:

![Test image](/_fern-img/58630b8f4fda508ac0e6baf8df2c5a3abb992f27f0c0ab1f0642bb8d76f355a7.webp)

**Filter by sentiment:**

* Search `negative` to see all unhappy customers
* Search `positive` to find praise and testimonials
* Search `neutral` to review informational emails

**Filter by priority, by department...**

**Combine filters for powerful queries:**

* `urgent` + `negative` → Critical customer issues
* `sales` + `high` → Hot leads requiring fast response
* `technical` + `bug-report` → Engineering backlog

**Pro tip:** You can use AgentMail's API to programmatically fetch emails by labels and build custom workflows, dashboards, and analytics.

## Building on Labels

Once your emails are automatically labeled, you can build powerful automation on top:

### Example 1: Priority Notifications

**Scenario**: Alert your team instantly when urgent issues arrive.

When an email is labeled `urgent` + `negative`, automatically send a Slack notification to your support channel. Include the sender's email and subject line so your team can respond immediately. This ensures critical customer issues never slip through the cracks.

### Example 2: Sentiment Escalation

**Scenario**: Escalate negative sentiment to management.

Track all emails labeled `negative` and automatically notify customer success managers when sentiment trends downward. If a single customer sends 3+ negative emails in a week, trigger a personal outreach from leadership to address their concerns proactively.

### Example 3: Department Routing

**Scenario**: Auto-forward emails to the right team.

Create rules that automatically forward emails based on department labels:

* `sales` → Forward to [sales@yourcompany.com](mailto:sales@yourcompany.com)
* `billing` → Create a ticket in your billing system
* `technical` → Post to #engineering Slack channel
* `support` → Add to support queue with appropriate SLA

### Example 4: Smart Auto-Reply

**Scenario**: Send contextual automated responses.

Instead of generic auto-replies, craft responses based on classification:

* `question` → "Thanks for your question! We'll respond within 24 hours."
* `bug-report` → "Thanks for reporting this. Our engineering team has been notified."
* `complaint` → "We're sorry to hear that. A senior support agent will contact you within 4 hours."
* `feature-request` → "Great idea! We've added this to our product roadmap."

### Example 5: Analytics Dashboard

**Scenario**: Track email metrics and trends.

Build a dashboard that queries emails by labels to analyze:

* **Sentiment trends**: Are customers getting happier or more frustrated?
* **Volume by department**: Which team is handling the most emails?
* **Response times by priority**: Are you meeting SLAs for urgent issues?
* **Common categories**: What are customers asking about most?

Use this data to identify bottlenecks, improve processes, and make data-driven decisions about staffing and product priorities.

---

Congratulations! You've built an AI-powered email classification system. This agent showcases how AgentMail's labeling feature can power sophisticated inbox automation and analytics.