Restructure

[igorbenav]

Restructure: pluggable template, three-layer architecture, bp CLI

This branch turns FastAPI-boilerplate from a flat src/app/ Python package into a uv workspace with two members — backend/ for the deployable application and cli/ for the developer/operator tool — while moving the application onto a three-layer architecture (interfaces/, infrastructure/, modules/) with vertical-slice modules per domain. JWT access/refresh is replaced with server-side sessions and CSRF; CRUDAdmin is replaced with SQLAdmin; ARQ workers are replaced with Taskiq. A new plugin-aware bp CLI ships in its own package and exposes two extension points (bp.commands and bp.features) so third-party packages can mount Typer sub-apps and feature generators via Python entry points. The documentation site is rewritten end-to-end and the generator is swapped from MkDocs to Zensical.

---

Three-layer architecture and vertical-slice modules

The pre-rebase layout had a flat src/app/{admin,api,core,crud,middleware,models,schemas}/ structure. Each horizontal slice (models, schemas, CRUD) was a separate top-level directory, so adding a feature meant editing seven files in seven different places, and unrelated features ended up importing from the same models/__init__.py because that's where models lived.

Replaced with a three-layer split. backend/src/interfaces/ holds HTTP entry points — the FastAPI app factory, route includes, the SQLAdmin sub-app, main.py. backend/src/infrastructure/ holds cross-cutting concerns that multiple modules consume the same way — cache, sessions, rate limit, taskiq, security, logging, db, config. backend/src/modules/ holds one directory per domain feature, and each module owns its full vertical: models.py, schemas.py, crud.py, service.py, routes.py, enums.py. The current modules are user, tier, api_keys, rate_limit, plus common for shared exceptions and helpers.

Co-locating the vertical means that most feature work touches one directory, and the cross-feature surface stays narrow. Cross-cutting infrastructure is separate because the alternative — duplicating cache/rate-limit setup inside each module — would re-introduce the horizontal sprawl this layout exists to avoid.

---

Server-side sessions and CSRF, replacing JWT access/refresh

The old flow issued JWT access + refresh tokens, kept a token_blacklist table for revocation, and read tokens from the Authorization header. Token revocation through a blacklist is the standard JWT workaround and the standard JWT footgun: in practice, services either skip the blacklist check on hot paths (silently accepting revoked tokens) or pay a DB read per request.

The new flow uses opaque session IDs in HttpOnly cookies, backed by Redis (default), Memcached, or in-memory storage behind a single AbstractSessionStorage interface. Every authenticated request looks up the session by ID; revocation is just a delete. CSRF is enforced by default (CSRF_ENABLED=true) for state-changing endpoints. Session lifetime, max sessions per user, and cookie security flags are all in AuthSettings.

OAuth piggybacks on the session machinery: the Google provider is wired end-to-end (callback creates a session, sets the cookie); the GitHub provider is scaffolded with the same provider/factory shape so adding it is a settings change plus a credential pair. API keys are a third auth path — separate hashed-credential model, separate validation flow — covered in their own section below.

---

API keys with scrypt-hashed credentials

The new api_keys module (in backend/src/modules/api_keys/) is a developer-facing credential system: users create named API keys with permissions and usage limits, the service validates them on requests, and per-call usage is recorded for analytics.

Credential storage went through several iterations driven by CodeQL alerts. The starting point was a plain hashlib.sha256(api_key) — flagged as py/weak-sensitive-data-hashing because CodeQL pattern-matches any SHA-family primitive receiving "sensitive" data and assumes password hashing. Switched to HMAC-SHA256 with SECRET_KEY as the pepper — better defense-in-depth, but CodeQL still flagged it because the underlying primitive is SHA256. Final state: scrypt with a per-row salt, stored as scrypt$N$r$p$salt_b64$derived_b64 with N=2**14, r=8, p=1, dklen=32. scrypt is on CodeQL's strong-KDF allowlist and the alerts close.

Per-row salt makes key_hash non-deterministic, so direct lookup by hash dies. validate_api_key now extracts the prefix from the incoming key, queries APIKey rows by the indexed key_prefix column with raw SQLAlchemy (select(APIKey).where(...).execution_options(populate_existing=True)), and verifies each candidate with _verify_api_key (constant-time compare via hmac.compare_digest). The populate_existing=True is needed because crud_api_keys.update(...) calls earlier in the same session can leave stale ORM objects in the identity map. The key_prefix is 8 chars from secrets.token_urlsafe; prefix collisions are vanishingly rare, so typical lookups return one row, occasionally two.

Why: scrypt verification on every API request that authenticates with a key is genuine overhead (~30-50ms on commodity hardware), but the slower-than-SHA hash function is the entire point of the rule, and the overhead is acceptable for the request rates a starter targets. Higher-RPS deployments should cache validation upstream.

A specific bug surfaced after this rewrite: the prefix-extraction code used api_key.split("_", 2)[1], which is wrong because secrets.token_urlsafe() uses an alphabet that includes _. When the random 8-char prefix happened to draw an underscore (~22% of the time given the alphabet), the split returned the wrong substring and validation failed. Fixed by extracting the prefix by position (api_key[4:12]) and adding a regression test that forces an underscore into the prefix.

---

Swappable infrastructure behind ABCs

Cache, rate limit, and session storage all share the same shape: an abstract base in base.py, concrete backends in backends/{redis,memcached,memory}.py, and a factory in storage.py (or provider.py) that picks one based on settings. The boilerplate ships Redis as the default, Memcached as the alternate, and in-memory for tests. Swapping is an env var — SESSION_BACKEND=memcached, CACHE_BACKEND=memcached.

The cache provider exposes a @cache decorator and a programmatic cache_get/cache_set API. Rate limiting is a middleware that consults per-tier, per-path rules from the database. Sessions are read via FastAPI dependencies (get_current_session_data, get_current_user).

A specific structural choice that came up during the cleanup pass: the abstract base classes live in */base.py, not in the same module as the factory. When a factory imports concrete backends at module top-level, and those backends import the abstract base, putting the base in the factory's module creates a cycle. Splitting cache/base.py, rate_limit/base.py, auth/session/base.py lets concrete backends import the base without participating in the factory's import graph. The session module didn't have this split until the deferred-imports pass forced it (see the lint section below).

---

SQLAdmin replaces CRUDAdmin

interfaces/admin/ is now a SQLAdmin integration: Admin(app, engine, ...) mounted at /admin, custom auth backend backed by the same session storage as the API, two views to start (UserView, TierView) plus a DataclassModelMixin for adding more without per-view boilerplate. Admin is env-toggled (ADMIN_ENABLED) so the /admin mount only happens when explicitly opted in.

The CRUDAdmin → SQLAdmin swap traded a custom-built admin for a community library that handles more of the form-rendering and relationship-traversal grunt work. The mixin is the project-specific veneer on top.

---

Taskiq replaces ARQ

Workers run via taskiq worker infrastructure.taskiq.worker:default_broker. Tasks are registered through a small TaskRegistry for development visibility (bp doesn't expose this yet, but it's a natural future feature). Brokers are env-selectable: Redis (default) or RabbitMQ via taskiq-aio-pika. The infra layer also exposes a DBSession dependency so tasks can use the same async DB session pattern as routes.

Compared to ARQ, Taskiq's middleware/dependency-injection model lines up with FastAPI's, which keeps the two halves of the app feeling like one codebase rather than two.

---

Production security validator

infrastructure/security/production_validator.py is a startup gate that refuses to boot in production with insecure configuration. It checks SECRET_KEY strength against a default-value blocklist, DB credential strength, Redis configuration, CORS policy (no * with credentials), session cookie flags, admin password presence, debug mode, OpenAPI exposure, and CREATE_TABLES_ON_STARTUP (must be false in prod — Alembic should own schema). Critical errors raise ProductionSecurityError and stop the app; non-critical issues log warnings.

The bp env validate command runs this same validator in production-mode against the current settings regardless of the configured ENVIRONMENT, so a dev or staging configuration can be audited before promotion.

---

Workspace split: backend/ and cli/

The repo root is now a uv workspace with two members. backend/ is the deployable FastAPI application — pyproject.toml, Dockerfile, src/, tests/, migrations/. cli/ is the developer/operator CLI — its own pyproject.toml, its own src/cli/ package, its own dependencies (typer, jinja2, fastapi-boilerplate as a workspace dep). Workspace members share a single .venv at the repo root, but the prod Dockerfile only copies backend/src/ so the deploy artifact stays lean — no Typer, no Jinja templates, no CLI surface.

fastapi-boilerplate/
├── pyproject.toml          # workspace root, no shipped artifact
├── uv.lock
├── backend/
│   ├── pyproject.toml      # name = "fastapi-boilerplate"
│   ├── Dockerfile          # multi-stage: dev / migrate / prod
│   └── src/{interfaces,infrastructure,modules}/
└── cli/
    ├── pyproject.toml      # name = "fastapi-boilerplate-cli", scripts.bp = "cli.app:app"
    └── src/cli/

uv sync --all-packages --all-extras from the repo root installs both members. uv run bp ... works from any directory in the repo. To install bp machine-wide outside the repo: uv tool install --editable ./cli.

Why: plugins are the whole point of the restructure, and plugin packages ship as separate Python packages anyway. Splitting the host CLI into its own workspace member matches that packaging shape — the cli is a Python package that depends on fastapi-boilerplate and contributes to its plugin entry points, exactly like third-party plugins will.

---

bp CLI with two plugin extension points

bp is a Typer application with in-tree commands and entry-point-based plugin discovery. The two extension points are deliberately separate:

• bp.commands — a Typer sub-app mounted under the root, e.g. bp aws deploy .... Plugin packages declare entry points whose values resolve to a typer.Typer instance. Discovery happens at app construction. Built-in command names (deploy, env, feature) shadow plugins with the same name and emit a warning rather than letting plugins silently override.
• bp.features — a Feature instance that bp feature list / bp feature info / bp feature apply can introspect and run. Features describe themselves with a FeatureManifest (static metadata) and produce a FeaturePlan (a tuple of FileOps and reserved Codemod/Hook slots) when invoked. The installer applies the plan transactionally with rollback on failure.

A broken plugin must not break the CLI. Discovery wraps each EntryPoint.load() in a broad except and emits a warning, so the working subset of commands and features stays usable when one plugin is misconfigured.

In-tree commands today: bp deploy generate {local,prod,nginx} (renders compose files via the built-in deploy feature; replaces the old setup.py/deploy.py scaffolder), and bp env gen-secret / bp env validate. The deploy feature ships with three Jinja templates — local hot-reload, single-host prod, prod-behind-nginx — and the installer renders them into the project root.

# Feature plugin shape (manifest + plan)
@dataclass(frozen=True)
class FeatureManifest:
    name: str
    version: str
    summary: str
    requires_features: tuple[str, ...] = ()
    adds_dependencies: tuple[str, ...] = ()

@dataclass(frozen=True)
class FeaturePlan:
    files: tuple[FileOp, ...]
    codemods: tuple[Codemod, ...] = ()    # reserved; raises NotImplementedError today
    hooks: tuple[Hook, ...] = ()           # reserved; raises NotImplementedError today

The reserved Codemod / Hook slots are intentional: plugin authors can declare codemods and hooks today and the installer gains support over time without breaking the manifest schema. The v1 installer raises NotImplementedError if a plan asks for them.

---

Documentation: full rewrite + MkDocs → Zensical

Every page under docs/user-guide/ was rewritten to match the post-rebase reality (sessions instead of JWT, SQLAdmin instead of CRUDAdmin, Taskiq instead of ARQ, the new module layout, the new CLI). Three new pages cover the CLI: docs/cli/index.md, docs/cli/commands.md, docs/cli/plugins.md. The README was slimmed but kept self-contained — quickstart still works without leaving the README.

The site generator moved from MkDocs (with mkdocs-material) to Zensical, configured via zensical.toml at the repo root. Zensical has a more modern theme variant out of the box and supports the triple-nested nav the user-guide subsections need without theme overrides. Local docs preview is uvx zensical serve; build is uvx zensical build. mkdocs.yml is gone; nothing else still references MkDocs.

Why: waiting for MkDocs 2.0 wasn't going to happen on this PR's timeline, and Zensical lets the docs site adopt the modern theme today.

---

CI workflows updated for the workspace

The three workflows (tests.yml, linting.yml, type-checking.yml) were rewritten for the workspace layout. All of them sync with uv sync --all-packages --all-extras (the --all-packages flag is required to include extras from workspace members — without it, the backend's dev extra holding pytest/mypy/ruff is dropped). All uv run invocations use --no-sync to prevent the implicit re-sync from undoing what the explicit sync step set up.

• Tests (backend/): cd backend && uv run --no-sync pytest
• Lint (covers backend src, cli src, and backend tests): uv run --no-sync ruff check backend/src cli/src backend/tests
• Type check (backend): cd backend && uv run --no-sync mypy src --config-file pyproject.toml
• Type check (cli): cd cli && uv run --no-sync mypy -p cli

---

Lint enforcement: PLC0415 (no deferred imports)

PLC0415 is now in extend-select for both backend/pyproject.toml and cli/pyproject.toml. Lifting deferred imports to module-top happened in two passes — first across backend/src/ and cli/src/ (17 deferred imports across 8 files), then across backend/tests/ (40 deferred imports across 7 test files). The lint workflow scans backend/tests so test-side regressions are caught.

Lifting the session storage factory's backend imports surfaced a circular import: storage.py → backends/{memory,redis,memcached}.py → storage.py (for AbstractSessionStorage). Resolved by extracting AbstractSessionStorage to auth/session/base.py, mirroring the cache/base.py and rate_limit/base.py pattern already used elsewhere. Concrete backends now import from ..base; storage.py re-exports AbstractSessionStorage from .base so existing callers keep working unchanged.

A few side effects of this rule worth naming: bp env gen-secret now imports the boilerplate's settings module on every invocation (was deferred); taskiq_aio_pika becomes a hard import-time requirement of infrastructure/taskiq/brokers.py; cli.features.registry imports the in-tree DeployFeature eagerly. Startup costs are negligible.

Why: the marginal startup wins from lazy imports aren't worth the cognitive cost of "where does this name come from?" questions during code review or debugging. Optional dependencies that genuinely shouldn't be loaded at all are handled at the package-level (declare them as extras), not by hiding imports inside functions.

---

CI fixes that were not config bugs

Two CI failures during the polish phase looked like configuration issues but turned out to be something else entirely. Both worth naming because the wrong fixes would have papered over real problems.

The cli mypy failure (Cannot find implementation or library stub for module named "cli.lib.project") was not a mypy_path / explicit_package_bases / py.typed problem, even though those got tried. The actual cause: .gitignore had unanchored lib/ and lib64/ patterns (standard Python venv ignores from the cookiecutter templates). Those patterns matched cli/src/cli/lib/ and the entire lib/ source subpackage was never committed to git. Locally everything worked because the files were on disk; CI checked out from git and found 13 files instead of 17. Fix: anchor the patterns to repo root (/lib/, /lib64/) and commit cli/src/cli/lib/*.py.

The pytest failure (cannot import name 'raise_for_deprecated_parameter' from 'testcontainers.core.utils') was a dependency-overlap problem. backend/pyproject.toml declared both testcontainers>=4.10.0 and testcontainers-postgres>=0.0.1rc1. The legacy testcontainers-postgres==0.0.1rc1 pulls in testcontainers-core==0.0.1rc1, and both that and testcontainers==4.14.2 ship testcontainers/core/utils.py and testcontainers/postgres/__init__.py at the same paths. Whichever installs last wins. Locally the order produced a working combination; CI's order paired the postgres module from 4.14.2 with a core/utils.py from rc1 that's missing raise_for_deprecated_parameter. Fix: drop the legacy package entirely and use the [postgres] extra: "testcontainers[postgres]>=4.10.0".

---

Documentation Updates

README.md:

• Trimmed the redundant content that was duplicated in the docs site
• Updated the Features section to match the new auth, admin, and worker stack
• Quickstart still self-contained; depth links out to the docs site

docs/cli/: new section

• index.md — what bp is, install, first commands
• commands.md — full reference for bp deploy generate, bp env gen-secret, bp env validate
• plugins.md — entry-point shapes, manifest/plan, writing a command vs. a feature plugin, packaging

docs/user-guide/authentication/sessions.md: new

• Server-side session model, cookie flags, CSRF, revocation
• Replaces the deleted jwt-tokens.md

docs/user-guide/: every page rewritten end-to-end

• Module names, paths, code samples, commands all updated for the new layout
• project-structure.md rewritten to describe the workspace

zensical.toml: new at repo root, replaces mkdocs.yml

• Modern theme variant
• Triple-nested nav for user-guide subsections via inline-table arrays
• docs_dir = "docs", brand purple palette via existing CSS

backend/Dockerfile: rewritten

• Multi-stage: requirements-stage exports lockfile-pinned deps, base installs them, dev adds dev extras, migrate runs Alembic, prod is the runtime artifact
• Only backend/src/ is copied into the prod stage; the cli is not in the deploy artifact

.github/workflows/{tests,linting,type-checking}.yml: rewritten for the workspace layout (covered in CI section above)

---

Test Plan

Automated (runs in CI)

• cd backend && uv run --no-sync pytest — 287 passing, 1 warning
• uv run --no-sync ruff check backend/src cli/src backend/tests — clean
• cd backend && uv run --no-sync mypy src --config-file pyproject.toml — clean (114 src files)
• cd cli && uv run --no-sync mypy -p cli — clean (21 src files)
• cd backend && uv run --no-sync mypy . — clean including tests (186 files)

Three-layer architecture / vertical-slice modules

• All routes still mount under /api/v1 after the move
• interfaces/main.py builds an app via the factory and includes admin when ADMIN_ENABLED
• User CRUD round-trip, tier CRUD round-trip, api-key CRUD round-trip

Server-side sessions + CSRF

• tests/integration/auth/test_endpoints.py — login → cookie → authenticated request → logout → cookie cleared
• tests/unit/infrastructure/auth/session/test_manager.py — create/get/update/delete/extend/exists across all three backends
• tests/unit/infrastructure/auth/session/test_api.py — CSRF middleware + dependency-injected session lookups
• tests/unit/infrastructure/auth/session/test_rate_limiting.py — login rate limit hit
• OAuth services unit-tested for Google + provider-factory (GitHub scaffolded, not E2E-tested)

API keys (scrypt + prefix lookup)

• test_create_api_key — key returned exactly once, format fai_<prefix>_<rest>, key_prefix stored
• test_validate_api_key_success / _inactive / _expired / _invalid / _no_permission
• test_wildcard_permissions — wildcard resource + wildcard action both validated
• test_api_key_hash_roundtrip — non-deterministic salt, verify-after-hash succeeds
• test_validate_api_key_with_underscore_in_prefix — regression for the split-on-_ bug; explicitly forces _ into the prefix
• test_record_usage / test_get_key_usage / test_get_usage_analytics / test_get_user_summary

Swappable backends

• Cache: tests/unit/infrastructure/cache/backends/test_{redis,memcached}.py plus decorator + provider tests
• Rate limit: backends/test_{redis,memcached}.py plus middleware + fail-open tests
• Sessions: backends/test_{redis,memcached}.py (memory backend exercised via the manager tests)

Production validator

• tests/unit/infrastructure/security/test_production_validator.py — every check has a passing-config case and a failing-config case

CLI (manual, no automated tests yet)

• bp deploy generate local --output-dir /tmp/x — renders the local compose template
• bp deploy generate prod — renders the prod compose template
• bp deploy generate nginx — renders both compose and nginx/default.conf
• bp env gen-secret — outputs a 64-char hex string
• bp env validate — runs the production validator against current settings
• bp feature list / bp feature info deploy
• Plugin discovery: a broken bp.commands entry point emits a warning and the rest of the CLI keeps working

Documentation build

• uvx zensical serve — site builds, no broken-link warnings
• All docs/cli/*.md pages render with code blocks and nav

---

Dependencies

New runtime dependencies on backend/:

• aiomcache — Memcached client for the session/cache/rate-limit backends
• fastsecure — session crypto helpers
• itsdangerous — CSRF token signing
• redis — Redis client (was already present)
• sqladmin — admin panel (replaces CRUDAdmin)
• taskiq, taskiq-redis, taskiq-aio-pika — worker stack (replaces ARQ)
• user-agents — session metadata

New dev dependencies on backend/:

• testcontainers[postgres] — Postgres container for integration tests (replaces the legacy testcontainers-postgres package, see CI section)
• pytest-xdist[psutil] — parallel test runs

cli/ has its own dependency tree: typer, jinja2, fastapi-boilerplate (workspace dep). No CLI deps are pulled into the backend deploy artifact.

---

Breaking Changes

• Project layout. src/app/ is gone. Imports of the form from src.app.X import Y will fail at import time. New form: from infrastructure.X import Y, from interfaces.X import Y, from modules.X import Y. Anyone with a fork on top of this branch needs an import sweep.
• Auth model. JWT access/refresh tokens are gone, the token_blacklist table is gone. Any client sending Authorization: Bearer <jwt> will be rejected. Cookie-based session auth is the API surface now; CSRF tokens are required for state-changing endpoints (CSRF_ENABLED=true by default).
• Admin panel. CRUDAdmin views are gone; SQLAdmin is the replacement. Any custom admin view code from a fork needs to be ported to the SQLAdmin ModelView shape (the DataclassModelMixin reduces the boilerplate).
• Workers. ARQ is gone. arq app.core.worker.WorkerSettings is replaced by taskiq worker infrastructure.taskiq.worker:default_broker. Any custom ARQ jobs need to be re-registered as Taskiq tasks.
• API key hash format. Stored key_hash values are now scrypt$N$r$p$salt$derived, not plain SHA256 hex. Any existing API keys hashed with the old SHA256 (or the brief HMAC-SHA256 interim) will not validate. For boilerplate users this is fine — they're starting fresh. Anyone running this code in prod must invalidate existing keys and have users regenerate.
• Settings layout. src/app/core/config.py is gone. Settings classes are split into focused groups under infrastructure/config/settings.py (AuthSettings, CacheSettings, RateLimiterSettings, etc.) and composed into a single Settings class. Env var names are largely unchanged but a few moved (e.g., session-related vars are under AuthSettings now, not SecuritySettings).
• Sync command. cd backend && uv sync --extra dev no longer pulls all the deps because workspace member extras need --all-packages. The new form is uv sync --all-packages --all-extras from the repo root. Anyone running the old command will silently get a venv missing pytest/mypy/ruff.
• Deployment scaffolder. setup.py / deploy.py (the standalone scripts) and the scripts/{local_with_uvicorn,gunicorn_managing_uvicorn_workers,production_with_nginx}/ artifact directories are gone. bp deploy generate {local,prod,nginx} does this now and writes to the project root.
• Docs site generator. mkdocs.yml is gone. Local serve is uvx zensical serve, not uvx --with mkdocs-material mkdocs serve. mkdocs gh-deploy workflows need to be updated to zensical build + a separate Pages deploy step.
• Removed dead code. src/app/api/v1/posts.py, src/app/models/post.py, src/app/schemas/post.py, src/app/crud/crud_posts.py — the demo Posts feature is gone. src/app/core/db/{token_blacklist,crud_token_blacklist}.py — JWT blacklist gone with JWT itself. Imports referencing any of these will fail at import time (loud, not silent).