AI Gateway Community Edition

🌐 Official Website • 中文文档 • Report Issues

中文 | English

A lightweight, production-ready AI Gateway built with Go and featuring intelligent request routing, tool call validation, and automatic error correction.

Official Website: bettercall.cn

✨ Features

🎯 Model Group Management

• Intelligent Load Balancing: Round-robin, weighted, random, and least-latency strategies
• Multi-Channel Support: Route requests across multiple AI providers
• Automatic Failover: Timeout and error handling with fallback strategies
• Cost Optimization: Token-based, request-based, and cost-multiplier billing modes

🔧 Tool Call Validation & AutoFix

• Automatic Error Detection: Validates AI tool calls against JSON schemas
• Smart AutoFix: Automatically corrects common errors:
  • JSON syntax errors (missing commas, unescaped quotes, trailing commas)
  • Type conversion (string to number, boolean conversion)
  • Single quote to double quote conversion
  • Object format validation
• Three Validation Modes:
  • disabled: No validation
  • validate_only: Detect errors without fixing
  • auto_fix: Automatically correct errors

📊 Request Logging

• Detailed Request Tracking: Full request/response logging with streaming support
• Chunk-Level Analysis: View individual SSE chunks for streaming responses
• Performance Metrics: Token usage, latency, and error statistics
• AutoFix Reporting: Track validation results and corrections

🗜️ TOON Optimization

• Token Compression: Reduce token consumption using TOON format
• Flexible Modes: disabled, tool_results, or full compression
• Transparent Integration: Works seamlessly with existing tools

🔑 API Key Management

• Multiple Model Groups: Single API key can access multiple model groups
• Rate Limiting: Configurable RPM and token limits
• Auto-Detection: Automatically detect and save new tool definitions

🚀 Quick Start

Option 1: Docker (Recommended)

Pull and run the pre-built Docker image from GitHub Container Registry:

# Pull the latest image
docker pull ghcr.io/agentofreef/bettercall-community:latest

# Run the container
docker run -d \
  --name bettercall-gateway \
  -p 8080:8080 \
  -v $(pwd)/data:/root/data \
  ghcr.io/agentofreef/bettercall-community:latest

The server will start on http://localhost:8080.

Available tags:

• latest - Latest stable release
• main - Latest commit from main branch
• v1.x.x - Specific version releases

Option 2: Build from Source

Prerequisites:

• Go 1.23 or later
• SQLite (embedded, no installation required)

Installation:

# Clone the repository
git clone https://github.com/agentofreef/bettercall-community.git
cd bettercall-community

# Install dependencies
go mod download

# Run the server
go run cmd/server/main.go

The server will start on http://localhost:8080.

Configuration

Create a .env file in the project root:

DB_TYPE=sqlite
DB_NAME=gateway.db
DB_FILE_PATH=./

SERVER_PORT=8080
SERVER_MODE=debug

First Steps

1. Access the Dashboard: Open http://localhost:8080 in your browser
2. Add Channels: Configure AI provider connections (OpenAI, Anthropic, etc.)
3. Create Model Groups: Set up load balancing and routing rules
4. Generate API Keys: Create keys for your applications
5. Define Tools: Add tool definitions for function calling

📖 Usage

Making Requests

Use the OpenAI-compatible API:

curl http://localhost:8080/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4",
    "messages": [{"role": "user", "content": "Hello!"}],
    "tools": [...]
  }'

Tool Call AutoFix Example

Original (with errors):

{
  "name": "get_weather",
  "arguments": "{'city': 'New York', max_items: '10'}"
}

After AutoFix:

{
  "name": "get_weather",
  "arguments": "{\"city\": \"New York\", \"max_items\": 10}"
}

The gateway automatically:

• Converted single quotes to double quotes
• Changed max_items from string "10" to number 10
• Fixed JSON object format

Viewing Logs

Navigate to http://localhost:8080/logs to:

• View all requests with filtering options
• Inspect individual requests with full details
• See streaming chunks for SSE responses
• Review AutoFix results and corrections

🏗️ Architecture

┌─────────────────────────────────────────────────────────────────┐
│                         Client Application                       │
└───────────────────────────────┬─────────────────────────────────┘
                                │ OpenAI-compatible API
                                │
┌───────────────────────────────▼─────────────────────────────────┐
│                         AI Gateway                               │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐          │
│  │   API Key    │  │  Tool Call   │  │   Request    │          │
│  │   Manager    │  │  Validator   │  │   Logger     │          │
│  └──────────────┘  └──────────────┘  └──────────────┘          │
│                                                                  │
│  ┌──────────────────────────────────────────────────────────┐  │
│  │              Model Group Router                           │  │
│  │  (Load Balancing, Failover, TOON Optimization)           │  │
│  └──────────────────────────────────────────────────────────┘  │
└───────────────────────────────┬─────────────────────────────────┘
                                │
        ┌───────────────────────┼───────────────────────┐
        │                       │                       │
┌───────▼────────┐    ┌─────────▼──────┐    ┌─────────▼──────┐
│   OpenAI       │    │   Anthropic    │    │   Other AI     │
│   Provider     │    │   Provider     │    │   Providers    │
└────────────────┘    └────────────────┘    └────────────────┘

🔧 Advanced Configuration

Model Group Settings

{
  "name": "production-gpt4",
  "load_balance_strategy": "weighted",
  "timeout_seconds": 60,
  "enable_tool_validation": true,
  "tool_validation_mode": "auto_fix",
  "toon_optimization": "tool_results",
  "billing_mode": "token_based"
}

Channel Configuration

Each channel represents a connection to an AI provider:

{
  "name": "OpenAI Primary",
  "type": "openai",
  "base_url": "https://api.openai.com/v1",
  "api_key": "sk-...",
  "models": ["gpt-4", "gpt-3.5-turbo"],
  "weight": 10,
  "priority": 1
}

📊 Dashboard Features

• Real-time Monitoring: View active requests and performance metrics
• Tool Management: Define and manage function calling tools
• Log Analysis: Search and filter request logs
• Settings: Configure auto-detection and system preferences

🤝 Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

1. Fork the repository
2. Create your feature branch (git checkout -b feature/AmazingFeature)
3. Commit your changes (git commit -m 'Add some AmazingFeature')
4. Push to the branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

📝 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

• Built with Gin - Fast HTTP web framework
• Database management with GORM
• Inspired by the need for intelligent AI request routing and error correction

📞 Support

• Issues: GitHub Issues
• Discussions: GitHub Discussions

---

Made with ❤️ for the AI community