Skip to content

feat: add spawner app for multi-user provisioning#71

Closed
dgokeeffe wants to merge 19 commits intodatasciencemonkey:mainfrom
dgokeeffe:pr/spawner-app
Closed

feat: add spawner app for multi-user provisioning#71
dgokeeffe wants to merge 19 commits intodatasciencemonkey:mainfrom
dgokeeffe:pr/spawner-app

Conversation

@dgokeeffe
Copy link
Copy Markdown
Contributor

Summary

Adds a Spawner — a separate Databricks App that provisions per-user coding-agents instances with one click.

Key features:

  • Users paste their PAT → spawner creates a secret scope, Databricks App, grants SP access, deploys, and waits for RUNNING
  • Async provisioning with live progress tracking (7-step pipeline with WebSocket polling)
  • Admin dashboard showing all spawned apps with state (RUNNING, DEPLOYING, PROVISIONING)
  • App names derived from user email (coding-agents-david-okeeffe)
  • 63-char app name limit handling with hash suffix for long names
  • Redeploy All button to push latest template to all spawned apps at once
  • Idempotent: re-provisioning an existing RUNNING app just refreshes the token

Architecture:

  • spawner/app.py — Flask backend with background threading for async provisioning
  • spawner/static/index.html — Databricks-themed UI with real-time progress
  • spawner/Makefile — Deploy targets (make deploy, make sync-template, make redeploy)
  • Deploys from shared template at /Workspace/Shared/apps/coding-agents

Test plan

  • Provision a new user app via PAT submission
  • Verify progress tracking shows all 7 steps
  • Re-provision existing RUNNING app (should just refresh token)
  • Test Redeploy All button updates all spawned apps
  • Verify app name truncation for long email addresses
  • Check admin dashboard shows correct app states

This pull request was AI-assisted by Isaac.

dgokeeffe and others added 19 commits March 17, 2026 22:47
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Admin token handles privileged ops (secret scopes, ACLs, deploy)
- User PAT creates app (ownership) + stored as runtime secret
- SCIM /Me resolves PAT owner to derive app name
- Secret resource included in app creation (no separate PATCH)
- Each PAT stored with unique UUID key
- /api/apps endpoint lists all spawned coding-agents apps
- Makefile for deploy/redeploy with run polling
- README documenting architecture and token model

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Pipe-based workspace import writes empty content. Write to temp file
first, then import with --file flag.

Co-authored-by: Isaac
New apps are UNAVAILABLE until first deploy, so waiting for
RUNNING causes a deadlock. Retry the deploy call with backoff.

Co-authored-by: Isaac
Deploy API requires compute_status=ACTIVE (~80s after app creation).
Gunicorn timeout bumped to 300s to handle the full provision flow.

Co-authored-by: Isaac
Random UUID secret keys caused re-provisions to store the PAT under a
new key while the app still referenced the old one. Users had to enter
their PAT twice because the first attempt's secret was orphaned.

Co-authored-by: Isaac
Secret values stored via `echo | databricks secrets put-secret`
include a trailing newline, causing invalid Authorization headers.

Co-authored-by: Isaac
The Databricks Apps API returns state under `app_status`, not `status`.
This caused the early-exit check to never detect running apps, and the
spawned apps table to always show UNKNOWN.

Co-authored-by: Isaac
Provision runs in a background thread so the endpoint returns
immediately. UI polls /api/provision-status every 3s showing
step-by-step progress with checkmarks. Apps table auto-refreshes
every 10s and shows in-flight provisions. Supports multiple
concurrent provisions.

Co-authored-by: Isaac
The list apps endpoint doesn't return app_status. Derive state from
compute_status and active_deployment.status instead.

Co-authored-by: Isaac
@mpkrass7
Copy link
Copy Markdown
Collaborator

mpkrass7 commented Apr 1, 2026

More up to date version here #92

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants