Skip to content

feat: Composite startup verbs for the flagsmith entrypoint#240

Open
khvn26 wants to merge 4 commits into
mainfrom
feat/composite-startup-verbs
Open

feat: Composite startup verbs for the flagsmith entrypoint#240
khvn26 wants to merge 4 commits into
mainfrom
feat/composite-startup-verbs

Conversation

@khvn26

@khvn26 khvn26 commented Jun 29, 2026

Copy link
Copy Markdown
Member

Changes

Closes #239.

In this PR, we add the composite startup verbs to the flagsmith CLI, fully compatible with Core API's run-docker.sh.

New flagsmith verbs:

  • serve
  • migrate
  • run-task-processor
  • migrate-and-serve

They mirror run-docker.sh's sequencing exactly, but run in a single process, so the Django app registry is initialised once instead of once per manage.py subprocess. They stay generic — no Core-API specifics — driven by settings the consuming app provides:

  • FLAGSMITH_MIGRATE_DATABASES (default ["default"]) — databases the migrate step applies, in order
  • FLAGSMITH_WAIT_FOR_MIGRATIONS_DATABASES (default ["default"]) — databases run-task-processor waits on
  • FLAGSMITH_STARTUP_COMMANDS (default []) — commands run between migrate and serve in migrate-and-serve

SKIP_WAIT_FOR_DB is honoured in the verb layer. The server hand-off goes through the existing start command via the management CLI path (not call_command, which can't pass through start's gunicorn arguments alongside its subparsers).

Separately, the task processor's worker tuning arguments (--numthreads, --sleepintervalms, --graceperiodms, --queuepopsize) now default from the TASK_PROCESSOR_* environment variables, falling back to the values run-docker.sh used, so that tuning survives the move to the entrypoint. environs is added to the task-processor extra, which now reads it.

The README documents the new verbs and their settings and environment variables.

How did you test this code?

Unit tests cover each verb's exact call sequence, the settings-driven database lists, the bare-vs-targeted migrate split, SKIP_WAIT_FOR_DB, argument pass-through, dispatch routing, and the task processor env defaults, including the TASK_PROCESSOR_SLEEP_INTERVAL fallback and precedence.

khvn26 added 3 commits June 26, 2026 19:44
Add serve, migrate, run-task-processor and migrate-and-serve verbs to the
`flagsmith` CLI, mirroring the role sequencing Core API's run-docker.sh
performs in shell but running in a single process so the Django app registry
is initialised once instead of per step.

The verbs stay generic: the databases to migrate or wait for, and any extra
startup commands, are driven by settings the consuming app provides
(FLAGSMITH_MIGRATE_DATABASES, FLAGSMITH_WAIT_FOR_MIGRATIONS_DATABASES,
FLAGSMITH_STARTUP_COMMANDS).

beep boop
Default --numthreads, --sleepintervalms, --graceperiodms and --queuepopsize
from the TASK_PROCESSOR_* environment variables (falling back to the values
run-docker.sh used), so the tuning survives the move to the `flagsmith`
entrypoint. Adds environs to the task-processor extra, which now reads it.

beep boop
@khvn26 khvn26 requested a review from a team as a code owner June 29, 2026 03:30
@khvn26 khvn26 requested review from emyller and removed request for a team June 29, 2026 03:30

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces composite startup commands to the flagsmith entrypoint to handle startup sequencing in a single process, and adds environment variable configuration for the task processor. The review feedback highlights a critical bug in how the legacy TASK_PROCESSOR_SLEEP_INTERVAL (seconds) is parsed as milliseconds, which can cause validation failures or high CPU usage. Additionally, the feedback suggests splitting startup commands with arguments using shlex.split to prevent execution failures, updating the unit tests accordingly, and clarifying the README documentation.

Comment on lines 33 to 41
parser.add_argument(
"--sleepintervalms",
type=int,
help="Number of millis each worker waits before checking for new tasks",
default=2000,
default=env.int(
"TASK_PROCESSOR_SLEEP_INTERVAL_MS",
env.int("TASK_PROCESSOR_SLEEP_INTERVAL", 500),
),
)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The legacy environment variable TASK_PROCESSOR_SLEEP_INTERVAL is defined in seconds (e.g., 0.5 or 2), whereas --sleepintervalms expects milliseconds.

  1. If TASK_PROCESSOR_SLEEP_INTERVAL is set to a float like 0.5, env.int will raise a ValidationError because "0.5" is not a valid integer.
  2. If it is set to an integer like 2 (representing 2 seconds), env.int will return 2, which means 2 milliseconds, causing the task processor to spin aggressively and consume 100% CPU.

We should parse TASK_PROCESSOR_SLEEP_INTERVAL as a float, multiply by 1000, and convert to an integer.

Suggested change
parser.add_argument(
"--sleepintervalms",
type=int,
help="Number of millis each worker waits before checking for new tasks",
default=2000,
default=env.int(
"TASK_PROCESSOR_SLEEP_INTERVAL_MS",
env.int("TASK_PROCESSOR_SLEEP_INTERVAL", 500),
),
)
legacy_sleep_interval = env.float("TASK_PROCESSOR_SLEEP_INTERVAL", None)
legacy_sleep_interval_ms = (
int(legacy_sleep_interval * 1000)
if legacy_sleep_interval is not None
else 500
)
parser.add_argument(
"--sleepintervalms",
type=int,
help="Number of millis each worker waits before checking for new tasks",
default=env.int(
"TASK_PROCESSOR_SLEEP_INTERVAL_MS",
legacy_sleep_interval_ms,
),
)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TASK_PROCESSOR_SLEEP_INTERVAL originally expected milliseconds.

Comment on lines +72 to +78
def migrate_and_serve(argv: list[str], *, prog: str) -> None:
"""Migrate, run any configured startup commands, then start the API server."""
_migrate()
startup_commands: list[str] = getattr(settings, "FLAGSMITH_STARTUP_COMMANDS", [])
for command in startup_commands:
django_execute_from_command_line(["flagsmith", command])
django_execute_from_command_line(["flagsmith", "start", "api", *argv])

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

If a startup command in FLAGSMITH_STARTUP_COMMANDS contains arguments (e.g., "bootstrap --force"), passing it directly to django_execute_from_command_line as a single string will fail because Django expects the subcommand and its arguments to be separate elements in the argv list. We should split the command string using shlex.split.

Suggested change
def migrate_and_serve(argv: list[str], *, prog: str) -> None:
"""Migrate, run any configured startup commands, then start the API server."""
_migrate()
startup_commands: list[str] = getattr(settings, "FLAGSMITH_STARTUP_COMMANDS", [])
for command in startup_commands:
django_execute_from_command_line(["flagsmith", command])
django_execute_from_command_line(["flagsmith", "start", "api", *argv])
def migrate_and_serve(argv: list[str], *, prog: str) -> None:
"""Migrate, run any configured startup commands, then start the API server."""
import shlex
_migrate()
startup_commands: list[str] = getattr(settings, "FLAGSMITH_STARTUP_COMMANDS", [])
for command in startup_commands:
django_execute_from_command_line(["flagsmith", *shlex.split(command)])
django_execute_from_command_line(["flagsmith", "start", "api", *argv])

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment on lines +56 to +81
def test_add_arguments__legacy_sleep_interval__used_as_fallback(
monkeypatch: pytest.MonkeyPatch,
) -> None:
# Given
monkeypatch.delenv("TASK_PROCESSOR_SLEEP_INTERVAL_MS", raising=False)
monkeypatch.setenv("TASK_PROCESSOR_SLEEP_INTERVAL", "750")

# When
args = _parse_defaults()

# Then
assert args.sleepintervalms == 750


def test_add_arguments__sleep_interval_ms__takes_precedence_over_legacy(
monkeypatch: pytest.MonkeyPatch,
) -> None:
# Given
monkeypatch.setenv("TASK_PROCESSOR_SLEEP_INTERVAL_MS", "300")
monkeypatch.setenv("TASK_PROCESSOR_SLEEP_INTERVAL", "750")

# When
args = _parse_defaults()

# Then
assert args.sleepintervalms == 300

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Since TASK_PROCESSOR_SLEEP_INTERVAL is defined in seconds, we should set it to "0.75" in the tests to assert 750 milliseconds.

def test_add_arguments__legacy_sleep_interval__used_as_fallback(
    monkeypatch: pytest.MonkeyPatch,
) -> None:
    # Given
    monkeypatch.delenv("TASK_PROCESSOR_SLEEP_INTERVAL_MS", raising=False)
    monkeypatch.setenv("TASK_PROCESSOR_SLEEP_INTERVAL", "0.75")

    # When
    args = _parse_defaults()

    # Then
    assert args.sleepintervalms == 750


def test_add_arguments__sleep_interval_ms__takes_precedence_over_legacy(
    monkeypatch: pytest.MonkeyPatch,
) -> None:
    # Given
    monkeypatch.setenv("TASK_PROCESSOR_SLEEP_INTERVAL_MS", "300")
    monkeypatch.setenv("TASK_PROCESSOR_SLEEP_INTERVAL", "0.75")

    # When
    args = _parse_defaults()

    # Then
    assert args.sleepintervalms == 300

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread README.md
| --- | --- | --- |
| `SKIP_WAIT_FOR_DB` | unset | When set, skip waiting for the database. |
| `TASK_PROCESSOR_NUM_THREADS` | `5` | Number of worker threads. |
| `TASK_PROCESSOR_SLEEP_INTERVAL_MS` | `500` | Millis each worker waits before checking for new tasks (falls back to `TASK_PROCESSOR_SLEEP_INTERVAL`). |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Clarify that TASK_PROCESSOR_SLEEP_INTERVAL is configured in seconds.

Suggested change
| `TASK_PROCESSOR_SLEEP_INTERVAL_MS` | `500` | Millis each worker waits before checking for new tasks (falls back to `TASK_PROCESSOR_SLEEP_INTERVAL`). |
| `TASK_PROCESSOR_SLEEP_INTERVAL_MS` | `500` | Millis each worker waits before checking for new tasks (falls back to `TASK_PROCESSOR_SLEEP_INTERVAL` in seconds). |

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@codecov-commenter

codecov-commenter commented Jun 29, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.55%. Comparing base (819921a) to head (02f363b).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #240      +/-   ##
==========================================
+ Coverage   97.39%   97.55%   +0.15%     
==========================================
  Files         104      107       +3     
  Lines        4613     4745     +132     
==========================================
+ Hits         4493     4629     +136     
+ Misses        120      116       -4     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

FLAGSMITH_STARTUP_COMMANDS entries are command strings; passing an entry with
arguments (e.g. "bootstrap --skip-fixtures") as a single argv element makes
Django treat the whole string as an unknown subcommand. Shell-split each entry
with shlex before dispatch.

beep boop
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Replace Core's run-docker.sh

3 participants