Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
94 changes: 82 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,6 @@ Create a new GitHub Actions workflow in your project, e.g. at [.github/workflows
name: Commit Check

on:
push:
pull_request:
branches: 'main'

Expand All @@ -50,7 +49,7 @@ jobs:
contents: read
pull-requests: write
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with:
fetch-depth: 0 # Required for merge-base checks
- uses: commit-check/commit-check-action@v2
Expand All @@ -60,7 +59,7 @@ jobs:
author-name: false
author-email: false
job-summary: true
pr-comments: ${{ github.event_name == 'pull_request' }}
pr-comments: true
```

> [!NOTE]
Expand Down Expand Up @@ -231,25 +230,96 @@ or gate on individual rules.

## GitHub Action Job Summary

By default, commit-check-action results are shown on the job summary page of the workflow.
By default, commit-check-action results are shown on the job summary page of the
workflow. The report below is reproduced as the action renders it, except that
its title is a heading in the real thing — it is bold here so it stays out of
this page's table of contents — and the footer names the version that actually
ran.

### Success Job Summary

![Success job summary](https://github.com/commit-check/.github/blob/main/screenshot/success-job-summary.png)
Passing runs stay to one line, with the detail folded away:

### Failure Job Summary
> <img src="https://raw.githubusercontent.com/commit-check/commit-check-action/main/assets/logo.png" width="20" align="top" alt=""> **Commit Check**
>
> ✅ **All 3 checks passed**
>
> <details>
> <summary>Show all 3 checks</summary>
>
> ```text
> Commit message
> ✔ PR title (feat: add login page)
> ✔ Commit 1/2 (feat: add login page)
> Branch
> ✔ Branch (feature/add-login)
> ```
>
> </details>
>
> _commit-check &lt;version&gt; · [Rules reference](https://commit-check.com/rules/)_

![Failure job summary](https://github.com/commit-check/.github/blob/main/screenshot/failure-job-summary.png)
### Failure Job Summary

## GitHub Pull Request Comments
Failures open with a count, then a table of only the scopes that failed — every
rule ID links to its documentation — with the full tree still one click away:

### Success Pull Request Comment
> <img src="https://raw.githubusercontent.com/commit-check/commit-check-action/main/assets/logo.png" width="20" align="top" alt=""> **Commit Check**
>
> ❌ **2 of 4 checks failed**
>
> | Scope | Checked value | Failed checks |
> |---|---|---|
> | Commit 2/2 | `bad msg` | [CC001 message](https://commit-check.com/rules/#cc001) |
> | Branch | `my-changes` | [CC201 branch](https://commit-check.com/rules/#cc201) |
>
> <details>
> <summary>Show all 4 checks</summary>
>
> ```text
> Commit message
> ✔ PR title (feat: add login page)
> ✔ Commit 1/2 (feat: add login page)
> ✖ Commit 2/2 (1 failure)
> CC001 message
> value: bad msg
> The commit message should follow Conventional Commits.
> Suggest: Use <type>(<scope>): <description>
> Branch
> ✖ Branch (1 failure)
> CC201 branch
> value: my-changes
> The branch should follow Conventional Branch.
> Suggest: Use <type>/<description> with allowed types
> ```
>
> </details>
>
> _commit-check &lt;version&gt; · [Rules reference](https://commit-check.com/rules/)_

![Success pull request comment](https://github.com/commit-check/.github/blob/main/screenshot/success-pr-comments.png)
A scope is one thing that was checked — a commit message, the branch, the author
— not one rule evaluation, so the total matches the ✔/✖ lines you can count and
does not grow with the number of rules in your config.

### Failure Pull Request Comment
## GitHub Pull Request Comments

![Failure pull request comment](https://github.com/commit-check/.github/blob/main/screenshot/failure-pr-comments.png)
With `pr-comments: true` the same report is posted as a pull request comment.
It is the same Markdown: the job summary and the comment are both rendered by
`render_report`, so the two surfaces cannot disagree. See
[Success Job Summary](#success-job-summary) and
[Failure Job Summary](#failure-job-summary) above for what it looks like.

What differs is the lifecycle rather than the content:

- The comment is **edited in place** on later runs rather than added to, so a
pull request carries one Commit Check comment however many times CI runs. It
stays after the checks pass, showing the ✅ report rather than disappearing.
- Comments are identified by a hidden `<!-- commit-check-action -->` marker, so
reformatting the visible text does not orphan the previous one. If several
marked comments somehow exist, the newest is kept and the rest deleted.
- A comment from a version predating the marker is adopted rather than
duplicated — but only when a bot posted it, since the older signal was just a
title prefix that a person could type by hand.

## Fork PR Comments

Expand Down
4 changes: 2 additions & 2 deletions docs/fork-pr-comments.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ jobs:
check:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with:
fetch-depth: 0
- uses: commit-check/commit-check-action@v2
Expand Down Expand Up @@ -148,7 +148,7 @@ jobs:
pull-requests: write
steps:
# SAFE: checkout the merge commit, NOT the PR head
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with:
fetch-depth: 0
- uses: commit-check/commit-check-action@v2
Expand Down
2 changes: 1 addition & 1 deletion examples/commit-check-workflow-a.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ jobs:
check:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
with:
fetch-depth: 0
- uses: commit-check/commit-check-action@v2
Expand Down
16 changes: 12 additions & 4 deletions main.py
Original file line number Diff line number Diff line change
Expand Up @@ -882,17 +882,25 @@ def add_pr_comments(results: list[ScopeResult]) -> int:
return 0 if all(scope.status == "pass" for scope in results) else 1
except GithubException as e:
if e.status == 403:
# GithubException.data is whatever the response decoded to, which
# is None for an empty body and a str for a non-JSON one. Reaching
# for .get unguarded would raise inside this handler and escape the
# function, turning the best-effort path into a step failure.
detail = e.data.get("message") if isinstance(e.data, dict) else None
print(
"::warning::Unable to post PR comment (403 Forbidden). "
"Ensure your workflow grants 'issues: write' permission. "
f"Error: {e.data.get('message', str(e))}",
"Ensure your workflow grants 'pull-requests: write' permission. "
f"Error: {detail or e}",
file=sys.stderr,
)
return 0
Comment thread
coderabbitai[bot] marked this conversation as resolved.
print(f"Error posting PR comment: {e}", file=sys.stderr)
# Annotated, not just printed: posting the comment is best-effort and
# never fails the step, so without an annotation the run is green, the
# comment is absent, and nothing says why.
print(f"::warning::Unable to post PR comment: {e}", file=sys.stderr)
return 0
except Exception as e:
print(f"Error posting PR comment: {e}", file=sys.stderr)
print(f"::warning::Unable to post PR comment: {e}", file=sys.stderr)
return 0


Expand Down
99 changes: 99 additions & 0 deletions main_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -1085,6 +1085,105 @@ def test_skips_when_comment_is_up_to_date(self):
mock_pull_request.create_comment.assert_not_called()


class _StubGithubException(Exception):
"""Stands in for github.GithubException, which is mocked away in these tests.

The real class has to be a genuine exception type or the ``except`` clause
in add_pr_comments raises TypeError before the handler is reached.
"""

def __init__(self, status, data=None):
super().__init__(f"status {status}")
self.status = status
# Stored as given, exactly as PyGithub does. Coercing a falsy payload
# to {} here would hide the very shapes the handler has to survive.
self.data = data


class TestAddPrCommentsFailures(unittest.TestCase):
"""Posting the comment is best-effort, but it must never fail silently.

Every branch here returns 0 so the step stays green — which is the point:
without an annotation the run is green, the comment is absent, and nothing
on the page says why.
"""

def _run(self, side_effect):
mock_pull_request = MagicMock()
mock_pull_request.get_comments.return_value = []
mock_pull_request.create_comment.side_effect = side_effect
mock_repo = MagicMock()
mock_repo.get_issue.return_value = mock_pull_request

github_module = MagicMock()
github_module.GithubException = _StubGithubException
github_module.Github.return_value.get_repo.return_value = mock_repo

with (
patch("main.PR_COMMENTS_ENABLED", True),
patch("main.is_fork_pr_with_readonly_token", return_value=False),
patch.dict(
os.environ,
{
"GITHUB_TOKEN": "token",
"GITHUB_REPOSITORY": "owner/repo",
"GITHUB_REF": "refs/pull/12/merge",
},
),
patch.dict(sys.modules, {"github": github_module}),
patch("builtins.print") as mock_print,
):
rc = main.add_pr_comments([fail_scope()])
printed = [
call[0][0]
for call in mock_print.call_args_list
if call[0] and isinstance(call[0][0], str)
]
return rc, printed

def test_forbidden_names_the_permission_that_actually_grants_this(self):
rc, printed = self._run(
_StubGithubException(403, {"message": "Resource not accessible"})
)
self.assertEqual(rc, 0)
warning = next(w for w in printed if "::warning::" in w)
# pull-requests, not issues: a PR comment is written with the
# pull-requests scope, and this hint is the only guidance a user gets.
self.assertIn("pull-requests: write", warning)
self.assertNotIn("issues: write", warning)
# The API's own wording is what tells the user which resource was
# refused, so the hint has to carry it through.
self.assertIn("Resource not accessible", warning)

def test_forbidden_with_a_non_mapping_payload_still_warns(self):
# data is whatever the body decoded to: None when empty, a str when it
# is not JSON. Reaching for .get on either raises inside the handler
# and escapes the function, which would fail the step.
for payload in (None, "forbidden"):
with self.subTest(payload=payload):
rc, printed = self._run(_StubGithubException(403, payload))
self.assertEqual(rc, 0)
warning = next(w for w in printed if "::warning::" in w)
self.assertIn("pull-requests: write", warning)
self.assertIn("status 403", warning)

def test_other_api_errors_are_annotated(self):
rc, printed = self._run(_StubGithubException(500, {"message": "boom"}))
self.assertEqual(rc, 0)
self.assertTrue(
any("::warning::" in w for w in printed),
f"a failed post must be annotated, got: {printed}",
)

def test_unexpected_errors_are_annotated(self):
rc, printed = self._run(RuntimeError("network went away"))
self.assertEqual(rc, 0)
self.assertTrue(
any("::warning::" in w and "network went away" in w for w in printed),
f"a failed post must be annotated, got: {printed}",
)


class TestIsForkPrWithReadonlyToken(unittest.TestCase):
def test_fork_pr_with_pull_request_event(self):
with (
Expand Down