Skip to content

AxmeAI/ai-agent-rate-limiting

Repository files navigation

AI Agent Rate Limiting

Your agent hit the API rate limit and started failing silently. AXME enforces rate and cost limits at the gateway - before the damage is done.

AI agents make API calls in loops. Without centralized rate control, one runaway agent can exhaust your API quota, burn through your budget, and take down the whole system. AXME cost policies let you set hard limits per agent: max intents per day, max intents per hour, max cost per day in USD.

Alpha - Built with AXME (AXP Intent Protocol). cloud.axme.ai - contact@axme.ai


The Problem

Agent "order-processor" in production:
  09:00 - processes 50 orders (normal)
  09:15 - retry loop hits a flaky upstream API
  09:16 - 200 retries/minute (exponential backoff broken)
  09:30 - 5,000 API calls made, $47 in LLM costs
  09:45 - 12,000 API calls, $130 in costs, rate-limited by upstream
  10:00 - all other agents sharing the API key are now blocked
  11:00 - someone finally notices

What should have happened:
  09:00 - processes 50 orders (normal)
  09:15 - retry loop starts
  09:16 - AXME: 200 intents/hour limit reached. 429 returned.
  09:16 - Agent stops. Alert fires. $0.80 spent, not $130.

The Solution

import httpx, os

api_key = os.environ["AXME_API_KEY"]
base = "https://cloud.axme.ai"
headers = {"x-api-key": api_key}

# Set a cost policy on the agent
address = "agent://myorg/production/order-processor"
httpx.put(
    f"{base}/v1/mesh/agents/{address}/policies/cost",
    headers=headers,
    json={
        "max_intents_per_hour": 200,
        "max_intents_per_day": 2000,
        "max_cost_per_day_usd": 10.00,
        "action_on_breach": "block",  # or "alert", "require_approval"
    },
)

When the agent exceeds any limit, AXME returns 429 Too Many Requests with a Retry-After header. The agent stops burning money immediately.


Quick Start

pip install axme httpx
export AXME_API_KEY="your-key"   # Get one: axme login

1. Set a cost policy

# set_policy.py
import httpx
import os

api_key = os.environ["AXME_API_KEY"]
base_url = os.environ.get("AXME_BASE_URL", "https://cloud.axme.ai")
headers = {"x-api-key": api_key}

agent_address = "agent://myorg/production/order-processor"

response = httpx.put(
    f"{base_url}/v1/mesh/agents/{agent_address}/policies/cost",
    headers=headers,
    json={
        "max_intents_per_hour": 200,
        "max_intents_per_day": 2000,
        "max_cost_per_day_usd": 10.00,
        "action_on_breach": "block",
    },
)
print(f"Policy set: {response.json()}")

2. Send intents until the limit is reached

# send_until_limited.py
from axme import AxmeClient, AxmeClientConfig
import os

client = AxmeClient(AxmeClientConfig(api_key=os.environ["AXME_API_KEY"]))

for i in range(250):
    try:
        intent_id = client.send_intent({
            "intent_type": "intent.orders.process.v1",
            "to_agent": "agent://myorg/production/order-processor",
            "payload": {"order_id": f"ORD-{i:04d}"},
        })
        print(f"[{i}] Sent: {intent_id}")
    except Exception as e:
        print(f"[{i}] Blocked: {e}")
        # Agent hit the rate limit - this is expected after 200 intents
        break

3. Check current usage

# check_usage.py
import httpx
import os

api_key = os.environ["AXME_API_KEY"]
base_url = os.environ.get("AXME_BASE_URL", "https://cloud.axme.ai")
headers = {"x-api-key": api_key}

agent_address = "agent://myorg/production/order-processor"

response = httpx.get(
    f"{base_url}/v1/mesh/agents/{agent_address}/policies/cost",
    headers=headers,
)
policy = response.json()
print(f"Policy: {policy}")

Before / After

Before: DIY Rate Limiting

import redis
import time

r = redis.Redis()

def rate_limited_call(agent_id, func, *args):
    key = f"rate:{agent_id}:{time.strftime('%Y%m%d%H')}"
    count = r.incr(key)
    r.expire(key, 3600)

    if count > 200:
        raise Exception("Rate limit exceeded")

    daily_key = f"rate:{agent_id}:{time.strftime('%Y%m%d')}"
    daily_count = r.incr(daily_key)
    r.expire(daily_key, 86400)

    if daily_count > 2000:
        raise Exception("Daily limit exceeded")

    # Still need: cost tracking, USD limits, per-agent policies,
    # alerting, approval workflows, audit trail, dashboard...
    return func(*args)

After: AXME Cost Policy (one API call)

httpx.put(
    f"{base_url}/v1/mesh/agents/{address}/policies/cost",
    headers={"x-api-key": api_key},
    json={
        "max_intents_per_hour": 200,
        "max_intents_per_day": 2000,
        "max_cost_per_day_usd": 10.00,
        "action_on_breach": "block",
    },
)

Rate limiting, cost caps, breach actions, audit trail, and dashboard - all built in.


How It Works

                        cost policy check
+-----------+  send_intent() +----------------+  deliver   +-----------+
|           | -------------> |                | ---------> |           |
| Initiator |                |   AXME Cloud   |            |   Agent   |
|           |                |   (gateway)    |            |           |
+-----------+                |                |            +-----------+
                             |  counters:     |
                             |  - hourly: 197 |
                             |  - daily: 1842 |
                             |  - cost: $7.20 |
                             |                |
                             |  limit hit?    |
                             |  YES -> 429    |
                             |  NO  -> deliver|
                             +----------------+
                                    |
                             mesh.axme.ai
                             (usage dashboard)

Dashboard

Agent Mesh Dashboard

View real-time usage counters, policy status, and breach history for every agent on your mesh:

Agent Mesh Policies

mesh.axme.ai


Policy Options

Field Type Description
max_intents_per_hour int Maximum intents allowed per rolling hour
max_intents_per_day int Maximum intents allowed per calendar day
max_cost_per_day_usd float Maximum USD cost per calendar day
action_on_breach string "block" (429), "alert" (allow + notify), "require_approval" (hold for human)

Run the Full Example

Prerequisites

curl -fsSL https://raw.githubusercontent.com/AxmeAI/axme-cli/main/install.sh | sh
axme login
pip install axme httpx

Terminal 1: Set policy and register agent

python set_policy.py
axme scenarios apply scenario.json

Terminal 2: Run agent

AXME_API_KEY=<agent-key> python agent.py

Terminal 1: Blast intents until rate limited

python send_until_limited.py

Verify

python check_usage.py
# Shows current counters vs policy limits

Related


License

MIT - see LICENSE.


Built with AXME (AXP Intent Protocol).

About

Your agent hit the API rate limit and started failing silently. AXME enforces rate and cost limits at the gateway.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages