AI Setup

AI Setup

🌍 English · Polski · Français · Deutsch · 中文

ECLI ships an AI assistant panel (Ctrl+A) that integrates with four providers. None are required — ECLI works fully without AI.

Supported providers

Provider	Best for	Cost model
Anthropic	High-quality reasoning, long context, code refactoring	Pay-per-token, free trial
OpenAI	Broad ecosystem, GPT-4 family	Pay-per-token
Ollama	Offline / on-prem, no API costs	Free (local compute)
HuggingFace	Open-source models, experimentation	Free tier + paid Inference API

You can configure multiple providers and switch between them with Tab inside the AI panel.

---

Anthropic (Claude)

Best balance of quality and cost for code-heavy work.

1. Get an API key

• Sign up at console.anthropic.com
• Generate a key under "API Keys"
• Keys start with sk-ant-...

2. Configure ECLI

Add to ~/.config/ecli/config.toml:

[ai]
default_provider = "anthropic"

[ai.anthropic]
api_key = "sk-ant-..."          # or use ANTHROPIC_API_KEY env var
model = "claude-sonnet-4-5"     # or claude-opus-4-5, claude-haiku-4-5
max_tokens = 4096
temperature = 0.2
timeout_seconds = 60

3. Or use environment variables

export ANTHROPIC_API_KEY="sk-ant-..."
export ECLI_AI_PROVIDER="anthropic"

Env vars take precedence over the config file — useful for shared machines.

Model choice

• claude-haiku-4-5 — fast, cheap, good for short edits and quick questions
• claude-sonnet-4-5 — balanced, recommended default
• claude-opus-4-5 — most capable, slower and more expensive

---

OpenAI

1. Get an API key

• Sign up at platform.openai.com
• Add billing (required for API access)
• Generate a key under "API keys"
• Keys start with sk-proj-... or sk-...

2. Configure ECLI

[ai]
default_provider = "openai"

[ai.openai]
api_key = "sk-..."              # or OPENAI_API_KEY env var
model = "gpt-4o"                # or gpt-4o-mini, gpt-4-turbo, o1
max_tokens = 4096
temperature = 0.2
timeout_seconds = 60

3. Or use environment variables

export OPENAI_API_KEY="sk-..."
export ECLI_AI_PROVIDER="openai"

Model choice

• gpt-4o-mini — fast and cheap
• gpt-4o — recommended default
• o1 / o1-mini — reasoning models, slower but stronger on complex problems

---

Ollama (local)

Run AI models locally — no API key, no cloud, no costs.

1. Install Ollama

# Linux / macOS
curl -fsSL https://ollama.com/install.sh | sh

# Or via package manager
brew install ollama       # macOS

Start the daemon:

ollama serve              # runs on localhost:11434

2. Pull a model

ollama pull llama3.1:8b
ollama pull codellama:13b
ollama pull qwen2.5-coder:7b   # code-tuned

3. Configure ECLI

[ai]
default_provider = "ollama"

[ai.ollama]
endpoint = "http://localhost:11434"
model = "qwen2.5-coder:7b"
timeout_seconds = 120          # local models can be slower

Hardware notes

• 7B-8B models — runs on a laptop with 16GB RAM
• 13B models — needs ~16GB RAM, faster with GPU
• 70B models — needs dedicated GPU (24GB+ VRAM)

---

HuggingFace Inference

1. Get a token

• Sign up at huggingface.co
• Create an access token under Settings → Access Tokens
• Tokens start with hf_...

2. Configure ECLI

[ai]
default_provider = "huggingface"

[ai.huggingface]
api_key = "hf_..."             # or HF_TOKEN env var
model = "meta-llama/Meta-Llama-3.1-8B-Instruct"
endpoint = "https://api-inference.huggingface.co"
timeout_seconds = 120

Free tier

The free Inference API has rate limits and may have cold-start delays. For production use, switch to Inference Endpoints (paid, dedicated capacity).

---

Switching providers at runtime

Inside the AI panel, press Tab to cycle through all providers configured in config.toml. The current provider shows in the panel header.

You can also override per-launch:

ECLI_AI_PROVIDER=ollama ecli

---

Disabling AI

[ai]
default_provider = "none"

The Ctrl+A panel will still open but display a message instead of taking input. To remove the keybinding entirely, see Keybindings › Customization.

---

Privacy

• Anthropic / OpenAI / HuggingFace — your prompts (and any code you include) are sent to the provider's servers per their privacy policies.
• Ollama — runs entirely locally. No data leaves your machine.

If you handle sensitive code or proprietary IP, prefer Ollama or disable AI entirely.

Cost monitoring

Both Anthropic and OpenAI provide usage dashboards in their consoles. ECLI does not currently display cost estimates inside the editor (planned for v0.2).

Troubleshooting AI

Symptom	Likely cause	Fix
"API key not configured"	Key missing in config and env	Set api_key in config or export env var
"Connection refused" (Ollama)	Daemon not running	ollama serve
"Rate limit exceeded"	Too many requests	Wait or switch provider
"Context length exceeded"	Prompt too long	Reduce input or use a model with bigger context
Slow responses	Provider latency or local model size	Switch model, check network

See Troubleshooting › AI for more.