Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions .github/workflows/func-tests-live.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
name: Live functional tests

# Hits the real Mergify API against
# mergify-clients-testing/mergify-cli-repo PR #1. Runs on every PR.
# NOT wired into the PR `ci-gate` job — an upstream blip must not
# block PRs. See `func-tests/test_live_smoke.py`.

permissions: read-all

on:
pull_request:

jobs:
live-tests:
timeout-minutes: 10
runs-on: ubuntu-24.04
environment: func-tests-live
steps:
- uses: actions/checkout@v6.0.2
- uses: actions/setup-python@v6.2.0
with:
python-version: 3.14

- name: Install Rust toolchain
run: |
rustup toolchain install stable --profile minimal
rustup default stable
- uses: Swatinem/rust-cache@v2

- uses: astral-sh/setup-uv@v8.1.0
with:
enable-cache: true
version-file: requirements-uv.txt

- name: Live smoke tests
shell: bash
env:
LIVE_TEST_MERGIFY_TOKEN: ${{ secrets.MERGIFY_CLI_LIVE_TEST_MERGIFY_TOKEN }}
run: uv run --locked poe live-test
24 changes: 24 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,30 @@ with `invoke_without_command=True` for consistent "did you mean?"
suggestions. Add an explicit `click.echo(ctx.get_help())` in the group
callback when `ctx.invoked_subcommand is None` for help display.

## Rust Port Workflow

The CLI is being ported from Python to Rust incrementally. The shipped
binary is `mergify` (built from `crates/mergify-cli`); commands not yet
ported fall through to a Python shim implemented by the
`crates/mergify-py-shim` crate, which invokes `python -m mergify_cli`
on the bundled Python source. Native Rust commands are dispatched
directly. Drift between the two implementations is prevented
structurally: when porting a command, the Python implementation MUST
be deleted in the same PR that adds the Rust implementation. There is
no period where both copies coexist.

A single PR therefore contains:

1. The Rust implementation (in the relevant `crates/*` crate) plus tests.
2. Removal of the Python implementation file(s) and their tests.
3. Any wiring updates (click registration, shim allow-list, etc.).

Reviewers should reject PRs that port a command without removing the
Python copy. Removing the Python copy without a Rust replacement is
fine when the command is being deprecated/dropped from the CLI — the
rule is "no two live copies of the same command", not "every Python
copy must be replaced".

## Documentation

When adding or changing a CLI feature, always update the documentation:
Expand Down
136 changes: 0 additions & 136 deletions PORT_STATUS.toml

This file was deleted.

38 changes: 38 additions & 0 deletions func-tests/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Live functional tests

End-to-end smoke tests that drive the real `mergify` binary
against the real Mergify API at
`mergify-clients-testing/mergify-cli-repo` PR #1.

Coverage:

- `mergify config simulate` — `POST /v1/repos/{owner}/{repo}/pulls/{n}/simulator`
- `mergify ci scopes-send` — `POST /v1/repos/{owner}/{repo}/pulls/{n}/scopes`
- `mergify ci junit-process` — OTLP traces upload + quarantine check

Each test fires when the real API's URL, auth, or wire format
diverges from what the CLI expects. Asserts only "endpoint exists,
accepts our payload, returns 2xx" — never response content, since
the test tenant's state is not under test control.

## Running

CI: `.github/workflows/func-tests-live.yaml` runs on every PR. Not
wired into the PR `ci-gate`, so an upstream blip cannot block PRs
— informational only for now.

Locally:

```bash
LIVE_TEST_MERGIFY_TOKEN=<app-key> uv run poe live-test
```

Skipped if `LIVE_TEST_MERGIFY_TOKEN` is unset.

## Adding a test

- Mark with `pytest.mark.live` (the module-level `pytestmark = pytest.mark.live`
in `test_live_smoke.py` already does this).
- Use the `cli` fixture to invoke the binary and `live_token` to
inject the token.
- Assert exit code only — never response content.
148 changes: 148 additions & 0 deletions func-tests/conftest.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
#
# Copyright © 2021-2026 Mergify SAS
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""Shared fixtures for the live functional-test harness.

Tests in this directory drive the real `mergify` binary against
the real Mergify API. The `cli` fixture builds the invocation
with a clean environment (CI/GitHub/Buildkite vars scrubbed) so
runs are deterministic regardless of where they execute.
"""

from __future__ import annotations

import dataclasses
import os
import pathlib
import shutil
import subprocess
import typing


if typing.TYPE_CHECKING:
from collections.abc import Mapping
from collections.abc import Sequence

import pytest


# Environment variables that the CLI auto-detects from the surrounding
# CI runner. Scrub them so a developer running tests inside GitHub
# Actions / Buildkite doesn't get different behavior than a clean
# laptop run.
_CI_ENV_VARS = (
"CI",
"GITHUB_ACTIONS",
"GITHUB_REPOSITORY",
"GITHUB_REF",
"GITHUB_HEAD_REF",
"GITHUB_BASE_REF",
"GITHUB_EVENT_PATH",
"GITHUB_EVENT_NAME",
"GITHUB_OUTPUT",
"GITHUB_STEP_SUMMARY",
"GITHUB_TOKEN",
"BUILDKITE",
"BUILDKITE_PULL_REQUEST",
"BUILDKITE_PULL_REQUEST_BASE_BRANCH",
"BUILDKITE_BRANCH",
"BUILDKITE_COMMIT",
"MERGIFY_API_URL",
"MERGIFY_TOKEN",
"MERGIFY_CONFIG_PATH",
"MERGIFY_TEST_EXIT_CODE",
"ACTIONS_STEP_DEBUG",
)


@dataclasses.dataclass(frozen=True)
class CliResult:
returncode: int
stdout: str
stderr: str


@pytest.fixture
def live_token() -> str:
"""Skip the live test if `LIVE_TEST_MERGIFY_TOKEN` isn't set."""
token = os.environ.get("LIVE_TEST_MERGIFY_TOKEN", "").strip()
if not token:
pytest.skip("LIVE_TEST_MERGIFY_TOKEN unset")
return token


def _resolve_mergify_binary() -> pathlib.Path | None:
"""Locate the `mergify` binary in the active venv (or PATH)."""
venv = os.environ.get("VIRTUAL_ENV")
if venv:
candidate = pathlib.Path(venv) / "bin" / "mergify"
if candidate.exists():
return candidate
candidate = pathlib.Path(venv) / "Scripts" / "mergify.exe"
if candidate.exists():
return candidate
found = shutil.which("mergify")
return pathlib.Path(found) if found else None


@pytest.fixture(scope="session")
def mergify_binary() -> pathlib.Path:
binary = _resolve_mergify_binary()
if binary is None:
pytest.skip(
"`mergify` binary not found; run `uv sync` to install it",
)
return binary


@pytest.fixture
def cli(
tmp_path: pathlib.Path,
mergify_binary: pathlib.Path,
) -> typing.Callable[..., CliResult]:
"""Return a callable that runs `mergify <args>` in a subprocess.

Runs from a fresh temp directory with CI-detection env vars
scrubbed. Stdin is closed so any accidental interactive prompt
fails fast instead of blocking. A 30s timeout caps pathological
hangs.
"""

def _run(
*args: str,
env: Mapping[str, str] | None = None,
cwd: pathlib.Path | None = None,
) -> CliResult:
full_env = {k: v for k, v in os.environ.items() if k not in _CI_ENV_VARS}
if env:
full_env.update(env)

cmd: Sequence[str] = [str(mergify_binary), *args]
proc = subprocess.run(
cmd,
capture_output=True,
text=True,
check=False,
stdin=subprocess.DEVNULL,
env=dict(full_env),
cwd=str(cwd or tmp_path),
timeout=30,
)
return CliResult(
returncode=proc.returncode,
stdout=proc.stdout,
stderr=proc.stderr,
)

return _run
Loading
Loading