Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
e67f204
Add OAuth M2M authentication as alternative to PAT
dgokeeffe Feb 26, 2026
c94d488
feat: Add multi-terminal support and git credential helper
dgokeeffe Mar 5, 2026
d3cd526
feat: Add Makefile for automated deployment with PAT provisioning
dgokeeffe Mar 5, 2026
c5fe930
feat: Switch OpenCode to Databricks fork with native provider
dgokeeffe Mar 6, 2026
7eebc33
feat: Enterprise git support with host-aware credentials and auto-clone
dgokeeffe Mar 6, 2026
7d13ebb
feat: Install GitHub CLI (gh) for interactive git authentication
dgokeeffe Mar 6, 2026
e62d5a7
fix: Wrap gh auth login to skip interactive prompts in xterm.js
dgokeeffe Mar 6, 2026
cf0ace5
fix: Pipe answer to gh auth login to avoid OSC escape sequence errors
dgokeeffe Mar 6, 2026
aa7e7c3
fix: Filter OSC escape sequence responses from xterm.js input
dgokeeffe Mar 6, 2026
cab43fe
feat: Add colored prompt, aliases, and login shell for terminal
dgokeeffe Mar 6, 2026
bd60f26
feat: Add tmux for terminal session persistence across refreshes
dgokeeffe Mar 6, 2026
881cbea
feat: Scale architecture for 20 concurrent terminals with persistence
dgokeeffe Mar 6, 2026
20f0fee
feat: Persist Claude Code state across container restarts
dgokeeffe Mar 6, 2026
bb49bcc
fix: Install tmux via AppImage instead of apt-get
dgokeeffe Mar 6, 2026
40bbf42
fix: Use AppRun wrapper for tmux to include bundled libraries
dgokeeffe Mar 6, 2026
624d83b
fix: Set APPDIR for tmux AppRun so terminfo is found
dgokeeffe Mar 7, 2026
af89458
feat: Scale architecture for 20 concurrent terminals with persistence
dgokeeffe Mar 7, 2026
2867542
fix: Add ~/.local/bin to server PATH so tmux is found
dgokeeffe Mar 7, 2026
46eac40
feat: Enterprise hardening — security, auth, CI/CD, observability
dgokeeffe Mar 7, 2026
1f1f9c2
fix: Resolve ruff lint and format errors for CI
dgokeeffe Mar 7, 2026
99dd39d
fix: Remove rate limiter causing terminal input lag
dgokeeffe Mar 7, 2026
92231bc
fix: Guard against stale CWD in terminal sessions
dgokeeffe Mar 7, 2026
cf100df
Merge remote-tracking branch 'origin/main' into feat/enterprise-git-s…
dgokeeffe Mar 8, 2026
ffed0d0
fix: Strip OAuth M2M vars when PAT is set for OpenCode
dgokeeffe Mar 8, 2026
11068d3
fix: Strip OAuth M2M vars in bashrc for OpenCode auth
dgokeeffe Mar 8, 2026
5f3991a
fix: Wrapper script strips OAuth vars for OpenCode
dgokeeffe Mar 8, 2026
3f264e5
fix: Guard against stale CWD in OpenCode wrapper
dgokeeffe Mar 8, 2026
daf0c5b
feat: Parallel setup + WebSocket terminal I/O
dgokeeffe Mar 9, 2026
4f66e57
feat: Add Databricks MCP server to Claude + OpenCode
dgokeeffe Mar 9, 2026
f97b9a0
refactor: Drop databricks MCP, keep deepwiki+exa for Claude+OpenCode
dgokeeffe Mar 9, 2026
10a3db4
fix: Prefer HTTP POST over SocketIO long-polling for input
dgokeeffe Mar 9, 2026
f374599
perf: Combine input+output in single HTTP round trip
dgokeeffe Mar 9, 2026
ed1c930
perf: Bundle Socket.IO client locally, add input batching
dgokeeffe Mar 9, 2026
e292b74
fix: Prevent dual output delivery causing character duplication
dgokeeffe Mar 9, 2026
655353e
feat: Add /api/active-sessions endpoint, gate tmux on TMUX_ENABLED
dgokeeffe Mar 10, 2026
8bcc2fa
feat: Add TabManager class, tab bar, mode toggle to terminal UI
dgokeeffe Mar 10, 2026
e55cda5
feat: Add TERMINAL_MODE and TMUX_ENABLED to app.yaml.template
dgokeeffe Mar 10, 2026
f75c624
feat: Add test_tab_ui.py and test_active_sessions.py
dgokeeffe Mar 10, 2026
74c795e
fix: Move mode toggle button outside toolbar for tabs-mode visibility
dgokeeffe Mar 10, 2026
2c00753
feat: Default TMUX_ENABLED to false for lighter-weight sessions
dgokeeffe Mar 10, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
name: CI

on:
push:
branches: [main]
pull_request:

jobs:
lint-and-security:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- uses: actions/setup-python@v5
with:
python-version: "3.11"

- name: Install uv
uses: astral-sh/setup-uv@v4

- name: Install tools
run: uv tool install ruff && uv tool install bandit

- name: Ruff lint
run: ruff check .

- name: Ruff format check
run: ruff format --check .

- name: Bandit security scan
run: bandit -r . -x ./tests,./static,./.claude -ll

- name: Validate pinned dependencies
run: |
if grep -qE '>=|<=|~=|[^=]>[^=]|[^=]<[^=]' requirements.txt; then
echo "ERROR: requirements.txt contains unpinned dependencies"
grep -nE '>=|<=|~=|[^=]>[^=]|[^=]<[^=]' requirements.txt
exit 1
fi
132 changes: 64 additions & 68 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -1,91 +1,87 @@
# Claude Code on Databricks
# CLAUDE.md

Welcome! This environment comes pre-configured with 39 skills and 2 MCP servers.
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Skills (30 total)
## What This Is

### Databricks Skills (16)
A browser-based terminal app (Databricks App) that gives Databricks users access to AI coding agents (Claude Code, Gemini CLI, Codex CLI, OpenCode) via xterm.js. No local IDE needed — models route through Databricks AI Gateway or Model Serving endpoints.

| Category | Skills |
|----------|--------|
| AI & Agents | agent-bricks, databricks-genie, mlflow-evaluation, model-serving |
| Analytics | aibi-dashboards, databricks-unity-catalog |
| Data Engineering | spark-declarative-pipelines, databricks-jobs, synthetic-data-generation |
| Development | asset-bundles, databricks-app-apx, databricks-app-python, databricks-python-sdk, databricks-config |
| Reference | databricks-docs, unstructured-pdf-generation |
## Development Commands

### Development Workflow Skills (14)
```bash
# Run locally (Flask dev server)
uv run python app.py
# Open http://localhost:8000

From [obra/superpowers](https://github.com/obra/superpowers):
# Production (Gunicorn, used by Databricks Apps)
uv run gunicorn app:app

| Skill | Purpose |
|-------|---------|
| brainstorming | Design features through collaborative dialogue |
| test-driven-development | RED-GREEN-REFACTOR cycle |
| systematic-debugging | 4-phase root cause analysis |
| writing-plans | Create detailed implementation plans |
| verification-before-completion | Verify before claiming done |
| executing-plans | Batch execution with checkpoints |
| dispatching-parallel-agents | Concurrent subagent workflows |
| subagent-driven-development | Fast iteration with two-stage review |
| using-git-worktrees | Parallel development branches |
| requesting-code-review | Pre-review checklist |
| receiving-code-review | Responding to feedback |
| finishing-a-development-branch | Merge/PR decision workflow |
| writing-skills | Create new skills |
| using-superpowers | Introduction to available skills |
# Deploy to Databricks Apps
databricks sync . /Workspace/Users/<email>/apps/<app-name> --watch=false
databricks apps deploy <app-name> --source-code-path /Workspace/Users/<email>/apps/<app-name>

## MCP Servers
# No test suite exists — skip test discovery
```

- **DeepWiki** - AI-powered documentation for any GitHub repository
- **Exa** - Web search and code context retrieval
## Architecture

## Databricks CLI
**Single-process Flask app** with PTY-based terminal sessions, served by Gunicorn (1 worker, 8 threads via gthread).

The Databricks CLI is pre-configured with your credentials. Test it:
```bash
databricks current-user me
```
### Startup Flow
1. `gunicorn.conf.py` → `post_worker_init` → `app.initialize_app()`
2. `initialize_app()` resolves auth (PAT or OAuth M2M via `utils.resolve_auth()`), determines app owner, starts cleanup thread, launches setup in background thread
3. Setup runs sequentially: git config (Python), micro editor (bash), GitHub CLI (`gh`), then `setup_claude.py`, `setup_codex.py`, `setup_opencode.py`, `setup_gemini.py`, `setup_databricks.py` — each installs a CLI and writes its config files. Each step has a 300s timeout. If `GIT_REPOS` is set, repos are auto-cloned into `~/projects/` after setup.
4. **State restore**: if `STATE_SYNC=true` (default), downloads saved state (Claude Code auto-memory, shell history) from `/Workspace/Users/{email}/.state/`
5. During setup, `/` serves `static/loading.html` (snake game); after setup, serves `static/index.html` (xterm.js terminal)
6. New terminal sessions start in `~/projects/` directory

Databricks can only authenticate with a PAT or CLIENT_ID and CLIENT_SECRET pair. If you have trouble logging in, remove the CLIENT_SECRET and CLIENT_ID from your environment, then try again. We want access to only be based on the app owner's credentials.
### Key Files
- **`app.py`** — Flask server, PTY session management (create/input/output/resize/close), authorization, setup orchestration
- **`utils.py`** — Auth resolution (PAT → OAuth M2M → SDK fallback), `TokenRefresher` for OAuth, `adapt_instructions_file()` for cross-CLI instruction sharing, `ensure_https()`
- **`setup_*.py`** — Per-agent setup scripts. Each resolves gateway vs direct endpoint, installs CLI binary, writes config files. Claude uses `~/.claude/settings.json`, Gemini uses `~/.gemini/.env`, OpenCode is built from fork (`dgokeeffe/opencode#feat/databricks-ai-sdk-provider`) with native Databricks provider — auto-discovers models and handles auth via `@databricks/sdk-experimental`, config at `~/.config/opencode/opencode.json`, Codex uses `~/.codex/config.toml` + `~/.codex/.env`, Databricks CLI uses `~/.databrickscfg`
- **`state_sync.py`** — Bidirectional state sync: `restore_state()` on startup, `save_state()` every 5 min + on shutdown. Syncs `~/.claude/projects/*/memory/` and `~/.bash_history` to `/Workspace/Users/{email}/.state/`
- **`sync_to_workspace.py`** — Post-commit hook target: syncs `~/projects/*` repos to `/Workspace/Users/{email}/projects/` via `databricks sync`
- **`gunicorn.conf.py`** — Must use `workers=1` (PTY fds and session state are process-local)

Common commands:
```bash
databricks workspace list /Workspace/Users/
databricks jobs list
databricks clusters list
```
### Authentication Model
`utils.resolve_auth()` tries in order: explicit `DATABRICKS_TOKEN` (PAT), `DATABRICKS_CLIENT_ID`+`SECRET` (OAuth M2M with token refresh), SDK auto-detect. The `TokenRefresher` class runs a background thread (every 30min) to refresh OAuth tokens and update all agent config files in-place.

**Git credentials** are handled by a host-aware credential helper (`git-credential-databricks`). It checks `GIT_TOKEN` first (scoped to `GIT_TOKEN_HOST` if set), then falls back to `DATABRICKS_TOKEN`. Users can also authenticate interactively via `gh auth login` (GitHub CLI is pre-installed). Workspace file sync is opt-in via `WORKSPACE_SYNC=true`.

### State Persistence
With `STATE_SYNC=true` (default), the following survives container restarts:
- **Claude Code auto-memory** (`~/.claude/projects/*/memory/`) — synced every 5 min + on shutdown
- **Shell history** (`~/.bash_history`) — synced every 5 min + on shutdown
- **Git repos** (`~/projects/`) — synced on commit if `WORKSPACE_SYNC=true`

## Project Setup
**Not persisted** (by design): tmux sessions (process state), CLI binaries (rebuilt on startup), gh auth tokens (security risk).

Before starting any new project or documentation:
### Security
Single-user app: the PAT owner is determined at startup, and `@app.before_request` checks `X-Forwarded-Email` against the owner. In OAuth M2M mode, authorization is delegated to the Databricks Apps proxy.

1. **Always initialize a git repo first:**
```bash
mkdir my-project && cd my-project
git init
```
Or clone an existing repo:
```bash
git clone https://github.com/user/repo.git
cd repo
```
### Session Management
PTY sessions use `pty.openpty()` + background reader threads. A cleanup thread kills sessions with no poll activity for 60s (SIGHUP → wait 3s → SIGKILL).

2. **Why?** Git commits automatically sync your work to Databricks Workspace at `/Workspace/Users/{your-email}/projects/{project-name}/`
### API Endpoints
- `GET /` — Loading screen (during setup) or terminal UI
- `GET /health` — Health check (no auth required)
- `GET /api/setup-status` — Setup progress (no auth required)
- `POST /api/session` — Create new PTY session
- `POST /api/input` — Send keystrokes to terminal (`{session_id, input}`)
- `POST /api/output` — Poll for terminal output (`{session_id}`) — also updates `last_poll_time`
- `POST /api/resize` — Resize terminal (`{session_id, cols, rows}`)
- `POST /api/session/close` — Close terminal session

3. **Then start working** - your commits will be backed up to Workspace
## Deployment Config

## Quick Start
- `app.yaml.template` — Template to copy to `app.yaml`. Set `DATABRICKS_GATEWAY_HOST` or remove it to fall back to direct Model Serving.
- Use `databricks sync` (not `workspace import-dir`) to upload — it respects `.gitignore` and handles `.git` correctly.
- **Never move the `.git` folder** to the workspace when running workspace import.

- Projects sync to Databricks Workspace on git commit
- Use `/commit` for guided commits
- Ask "help me create a dashboard" to see skills in action
- Ask about any GitHub repo with DeepWiki MCP
## Skills

## Credits
39 pre-installed skills live in `.claude/skills/`. Databricks skills come from [databricks-solutions/ai-dev-kit](https://github.com/databricks-solutions/ai-dev-kit), workflow skills from [obra/superpowers](https://github.com/obra/superpowers). Use `/refresh-databricks-skills` to pull latest.

- Databricks skills from [databricks-solutions/ai-dev-kit](https://github.com/databricks-solutions/ai-dev-kit)
- Development workflow skills from [obra/superpowers](https://github.com/obra/superpowers)
## Dependencies

# things to remember
Remember to never move .git folder to the workspace if you're running workspace import.
`requirements.txt`: flask, claude-agent-sdk, databricks-sdk. No pyproject.toml — no build system.
87 changes: 87 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
# Makefile for deploying Coding Agents to Databricks Apps
#
# Usage:
# make deploy PROFILE=daveok PAT=dapi...
# make deploy PROFILE=daveok # prompts for PAT interactively
# make redeploy PROFILE=daveok # skip secret setup, just sync + deploy
# make status PROFILE=daveok # check app status
# make logs PROFILE=daveok # tail app logs

# Configuration
PROFILE ?= DEFAULT
APP_NAME ?= coding-agents
SECRET_SCOPE ?= $(APP_NAME)-secrets
SECRET_KEY ?= databricks-token

# Resolve user email and workspace path from the profile
USER_EMAIL = $(shell databricks current-user me --profile $(PROFILE) --output json 2>/dev/null | python3 -c "import sys,json; print(json.load(sys.stdin).get('userName',''))")
WORKSPACE_PATH = /Workspace/Users/$(USER_EMAIL)/apps/$(APP_NAME)

.PHONY: help deploy redeploy create-app setup-secret sync deploy-app status logs clean-secret

help: ## Show this help
@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | awk 'BEGIN {FS = ":.*?## "}; {printf " \033[36m%-18s\033[0m %s\n", $$1, $$2}'

deploy: create-app setup-secret sync deploy-app ## Full deploy: create app, set secret, sync, deploy
@echo ""
@echo "Deployment complete! App URL:"
@databricks apps get $(APP_NAME) --profile $(PROFILE) --output json 2>/dev/null | python3 -c "import sys,json; print(json.load(sys.stdin).get('url','(pending)'))"

redeploy: sync deploy-app ## Redeploy: sync + deploy (skip secret setup)
@echo ""
@echo "Redeployment complete!"

create-app: ## Create the Databricks App (idempotent)
@echo "==> Checking if app '$(APP_NAME)' exists..."
@if databricks apps get $(APP_NAME) --profile $(PROFILE) >/dev/null 2>&1; then \
echo " App '$(APP_NAME)' already exists, skipping create."; \
else \
echo " Creating app '$(APP_NAME)'..."; \
databricks apps create $(APP_NAME) --profile $(PROFILE); \
fi

setup-secret: ## Create secret scope and store PAT
@echo "==> Setting up DATABRICKS_TOKEN secret..."
@# Create scope if it doesn't exist
@if databricks secrets list-scopes --profile $(PROFILE) --output json 2>/dev/null | python3 -c "import sys,json; scopes=[s['name'] for s in json.load(sys.stdin).get('scopes',[])]; exit(0 if '$(SECRET_SCOPE)' in scopes else 1)" 2>/dev/null; then \
echo " Secret scope '$(SECRET_SCOPE)' already exists."; \
else \
echo " Creating secret scope '$(SECRET_SCOPE)'..."; \
databricks secrets create-scope $(SECRET_SCOPE) --profile $(PROFILE); \
fi
@# Store the PAT - prompt if not provided
@if [ -z "$(PAT)" ]; then \
echo " Enter your Databricks PAT (will not echo):"; \
read -s pat_value && \
echo "$$pat_value" | databricks secrets put-secret $(SECRET_SCOPE) $(SECRET_KEY) --profile $(PROFILE); \
else \
echo "$(PAT)" | databricks secrets put-secret $(SECRET_SCOPE) $(SECRET_KEY) --profile $(PROFILE); \
fi
@echo " Secret stored in $(SECRET_SCOPE)/$(SECRET_KEY)"
@# Link secret to app resource
@echo " Linking secret to app resource 'DATABRICKS_TOKEN'..."
@curl -s -X PATCH \
"$$(databricks auth env --profile $(PROFILE) 2>/dev/null | python3 -c "import sys,json; print(json.load(sys.stdin)['env']['DATABRICKS_HOST'])")/api/2.0/apps/$(APP_NAME)" \
-H "Authorization: Bearer $$(databricks auth token --profile $(PROFILE) 2>/dev/null | python3 -c "import sys,json; print(json.load(sys.stdin)['access_token'])")" \
-H "Content-Type: application/json" \
-d '{"resources":[{"name":"DATABRICKS_TOKEN","description":"PAT for model serving access","secret":{"scope":"$(SECRET_SCOPE)","key":"$(SECRET_KEY)","permission":"READ"}}]}' \
>/dev/null
@echo " App resource linked."

sync: ## Sync local files to Databricks workspace
@echo "==> Syncing to $(WORKSPACE_PATH)..."
databricks sync . $(WORKSPACE_PATH) --watch=false --profile $(PROFILE)

deploy-app: ## Deploy the app from workspace
@echo "==> Deploying app '$(APP_NAME)'..."
databricks apps deploy $(APP_NAME) --source-code-path $(WORKSPACE_PATH) --profile $(PROFILE) --no-wait

status: ## Check app status
@databricks apps get $(APP_NAME) --profile $(PROFILE)

logs: ## Tail app logs
databricks apps logs $(APP_NAME) --profile $(PROFILE)

clean-secret: ## Remove secret scope (destructive)
@echo "==> Removing secret scope '$(SECRET_SCOPE)'..."
databricks secrets delete-scope $(SECRET_SCOPE) --profile $(PROFILE)
1 change: 1 addition & 0 deletions VERSION
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
0.1.0
Loading