Skip to content

Generate AI commit summaries for automated API updates - #87

Merged
dblock merged 3 commits into
slack-ruby:masterfrom
dblock:ai-generated-update-summary
Aug 12, 2026
Merged

Generate AI commit summaries for automated API updates#87
dblock merged 3 commits into
slack-ruby:masterfrom
dblock:ai-generated-update-summary

Conversation

@dblock

@dblock dblock commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

Problem

The daily update workflow (.github/workflows/update.yml) commits scraped API changes with a generic, date-only message like Updated from Slack docs, 2026/08/11 (see ff2d6a0). This tells you nothing about what actually changed without opening the diff.

Fix

Add an AI-generated commit summary step to the workflow, using only GitHub Actions (no extra Ruby code):

  • New .github/prompts/commit-message.prompt.yml: a structured prompt for actions/ai-inference@v1 that asks a model to turn a git diff / git diff --stat into a short commit title (<=72 chars, imperative mood) and a summary of 1-5 bullet points. Uses responseFormat: json_schema so the response is guaranteed valid, parseable JSON.
  • update.yml changes:
    • Adds permissions: contents: write / models: read (required for GitHub Models inference; no extra secrets or Copilot subscription needed, it uses the default GITHUB_TOKEN).
    • Checks whether rake api:update produced any changes; if not, skips the AI call entirely.
    • Captures git diff --stat and a truncated git diff (20,000 chars) to files and passes them into the prompt via file_input.
    • Runs actions/ai-inference@v1 with the prompt file, parses title/summary out of the JSON response with jq.
    • Falls back to the original generic, date-based message if there are no changes, the AI step fails, or the response is malformed/empty, so the workflow never blocks on this step.
    • Passes the resulting multi-line message to the existing EndBug/add-and-commit@v9 step, unchanged otherwise.

Example resulting commit message

Add chat.appendStream and chat.stopStream methods

- Added chat.appendStream and chat.stopStream methods
- Added metadata argument to chat.postEphemeral
- Marked send_on_behalf_not_allowed error on chat.postMessage

Verified .github/workflows/update.yml and .github/prompts/commit-message.prompt.yml parse as valid YAML, and that the embedded jsonSchema is valid JSON.

dblock and others added 3 commits August 11, 2026 20:30
The daily "update" workflow committed changes with a generic,
date-only message ("Updated from Slack docs, YYYY/MM/DD"), which
doesn't say anything about what actually changed.

Add a GitHub Models-backed step (actions/ai-inference@v1, using the
default GITHUB_TOKEN with models: read permission, no extra
secrets/subscription needed) that summarizes git diff / git diff
--stat of the scraped changes into a short commit title (<=72
chars) and a few bullet points, via a structured
.github/prompts/commit-message.prompt.yml with a JSON schema
response so the output is easy to parse reliably.

Falls back to the original generic, date-based message if there are
no changes, the AI call fails, or the response is malformed, so the
workflow never blocks on this step.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@dblock
dblock merged commit bc55a96 into slack-ruby:master Aug 12, 2026
2 checks passed
@dblock
dblock deleted the ai-generated-update-summary branch August 12, 2026 00:40
dblock added a commit to dblock/slack-ruby-client that referenced this pull request Aug 12, 2026
The Update API workflow committed a single generic CHANGELOG.md line
for every automated update PR: "Update API from
slack-api-ref@<rev>", with no indication of what actually changed.

Following the same approach as slack-ruby/slack-api-ref#87, add a
GitHub Models-backed step (actions/ai-inference@v1, default
GITHUB_TOKEN with models: read permission, no extra secrets needed)
that summarizes the diff produced by rake slack:api:update into a
short list of grouped, descriptive entries (new/removed methods,
argument/validation changes, error code changes, etc.) via a
structured .github/prompts/changelog-entries.prompt.yml with a JSON
schema response.

Each generated entry becomes its own CHANGELOG.md bullet, all tagged
with the same automated PR number, replacing the old single generic
line. The same list is also included under a "## Changes" heading in
the PR body for reviewers. Falls back to the original generic,
rev-based message if there are no changes, the AI step fails, or the
response is empty/malformed, so the workflow never blocks on this
step. Uses Ruby (already set up in this workflow) instead of Python
for the inline CHANGELOG-rewriting script, for consistency with the
rest of this Ruby project.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
dblock added a commit to dblock/slack-ruby-client that referenced this pull request Aug 12, 2026
The Update API workflow committed a single generic CHANGELOG.md line
for every automated update PR: "Update API from
slack-api-ref@<rev>", with no indication of what actually changed.

Following the same approach as slack-ruby/slack-api-ref#87, add a
GitHub Models-backed step (actions/ai-inference@v1, default
GITHUB_TOKEN with models: read permission, no extra secrets needed)
that summarizes the diff produced by rake slack:api:update into a
short list of grouped, descriptive entries (new/removed methods,
argument/validation changes, error code changes, etc.) via a
structured .github/prompts/changelog-entries.prompt.yml with a JSON
schema response.

Each generated entry becomes its own CHANGELOG.md bullet, all tagged
with the same automated PR number, replacing the old single generic
line. The same list is also included under a "## Changes" heading in
the PR body for reviewers. Falls back to the original generic,
rev-based message if there are no changes, the AI step fails, or the
response is empty/malformed, so the workflow never blocks on this
step. Uses Ruby (already set up in this workflow) instead of Python
for the inline CHANGELOG-rewriting script, for consistency with the
rest of this Ruby project.

Also locks simplecov below 1.1.0 in the Gemfile: 1.1.0 changed
Result#to_hash to write a float timestamp to
coverage/.resultset.json, which coverallsapp/github-action's parser
(typed Int64?) fails to deserialize, breaking the Coveralls step in
CI (coverallsapp/github-action#269, fix tracked upstream in
coverallsapp/coverage-reporter#191).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant