Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,3 +45,4 @@ jobs:
echo "All openclaw version fields are up to date ($LATEST)."
- run: npx tsc --noEmit
- run: npx vitest run
- run: npm run build
160 changes: 160 additions & 0 deletions .github/workflows/openclaw_version_tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,160 @@
name: OpenClaw Version Test

on:
pull_request:
branches: [main]
schedule:
# Mondays 07:00 UTC — 8am Prague in winter (CET), 9am Prague in summer (CEST).
- cron: "0 7 * * 1"
workflow_dispatch:
Comment thread
protoss70 marked this conversation as resolved.

concurrency:
group: openclaw-version-test-${{ github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}

jobs:
discover:
runs-on: ubuntu-latest
outputs:
versions: ${{ steps.filter.outputs.versions }}
steps:
- uses: actions/setup-node@v4
with:
node-version: 22

- id: filter
name: Pick latest 3 OpenClaw versions
shell: bash
run: |
set -euo pipefail

# Drop pre-releases (anything with '-'), sort by semver, take the latest 3.
versions=$(
npm view openclaw versions --json \
| jq -r '.[] | select(test("-") | not)' \
| sort -V \
| tail -n 3
)

if [ -z "$versions" ]; then
echo "No stable OpenClaw versions found on npm." >&2
exit 1
fi

json=$(echo "$versions" | jq -R . | jq -sc .)
echo "versions=$json" >> "$GITHUB_OUTPUT"
echo "Selected versions: $json"

test:
needs: discover
runs-on: ubuntu-latest
name: OpenClaw ${{ matrix.version }}
strategy:
fail-fast: false
matrix:
version: ${{ fromJson(needs.discover.outputs.versions) }}
steps:
- uses: actions/checkout@v4

- uses: actions/setup-node@v4
with:
node-version: 22

- name: Install OpenClaw ${{ matrix.version }} and smoke-test plugin
shell: bash
run: |
set -euo pipefail

# On PRs, pack the PR's source and install that tarball. On schedule /
# manual dispatch, install the currently-published @latest from npm.
if [[ "$GITHUB_EVENT_NAME" == "pull_request" ]]; then
echo "::group::npm install && npm pack (PR branch)"
cd "$GITHUB_WORKSPACE"
npm install
tarball=$(npm pack | tail -n 1)
PLUGIN_SPEC="$GITHUB_WORKSPACE/$tarball"
echo "Packed: $PLUGIN_SPEC"
echo "::endgroup::"
else
PLUGIN_SPEC="@apify/apify-openclaw-plugin@latest"
echo "Using published spec: $PLUGIN_SPEC"
fi

WORKDIR="$RUNNER_TEMP/openclaw-test"
mkdir -p "$WORKDIR"
cd "$WORKDIR"

echo "::group::npm install openclaw@${{ matrix.version }}"
npm init -y >/dev/null
npm install "openclaw@${{ matrix.version }}"
echo "::endgroup::"

echo "::group::openclaw plugins install $PLUGIN_SPEC"
npx --no-install openclaw plugins install "$PLUGIN_SPEC"
echo "::endgroup::"

echo "::group::openclaw plugins list"
list_output=$(npx --no-install openclaw plugins list)
echo "$list_output"
if ! echo "$list_output" | grep -q "apify-openclaw-plugin"; then
echo "FAIL: apify-openclaw-plugin not present in 'plugins list' output" >&2
exit 1
fi
echo "::endgroup::"

echo "::group::openclaw plugins inspect apify-openclaw-plugin --runtime --json"
inspect_json=$(npx --no-install openclaw plugins inspect apify-openclaw-plugin --runtime --json)
echo "$inspect_json" | jq .
if ! echo "$inspect_json" | jq -e '.. | strings | select(. == "apify")' >/dev/null; then
echo "FAIL: 'apify' tool not found in plugin runtime inspect output" >&2
exit 1
fi
echo "::endgroup::"

echo "OK: plugin loaded and apify tool is registered on openclaw ${{ matrix.version }}"

# Stable aggregator check — mark THIS one as required in branch protection.
# The matrix job names ("OpenClaw 2026.5.X") rotate as discovery picks up new
# versions, but this job's name never changes.
required:
needs: [discover, test]
if: always()
runs-on: ubuntu-latest
steps:
- name: All OpenClaw version tests must pass
shell: bash
run: |
if [[ "${{ needs.discover.result }}" != "success" || "${{ needs.test.result }}" != "success" ]]; then
echo "discover=${{ needs.discover.result }} test=${{ needs.test.result }}"
echo "One or more OpenClaw version tests failed."
exit 1
fi
echo "All OpenClaw version tests passed."

# Slack notification on scheduled failure — disabled for now.
# To re-enable: uncomment this job and set the `SLACK_WEBHOOK_URL` repo secret.
#
# notify:
# needs: test
# if: always() && needs.test.result == 'failure' && github.event_name == 'schedule'
# runs-on: ubuntu-latest
# steps:
# - name: Notify Slack on scheduled failure
# env:
# SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
# RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
# shell: bash
# run: |
# if [ -z "${SLACK_WEBHOOK_URL:-}" ]; then
# echo "SLACK_WEBHOOK_URL secret not configured — skipping notification."
# exit 0
# fi
#
# payload=$(jq -nc \
# --arg text "🚨 *OpenClaw Version Test failed* (scheduled run)
# One or more OpenClaw versions failed the plugin install smoke test.
# <${RUN_URL}|View run details>" \
# '{text: $text}')
#
# curl -sS -X POST -H 'Content-Type: application/json' \
# --data "$payload" "$SLACK_WEBHOOK_URL"
3 changes: 3 additions & 0 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,9 @@ jobs:
- name: Run tests
run: npx vitest run

- name: Build
run: npm run build

- name: Commit version bump
uses: apify/actions/signed-commit@v1.1.2
with:
Expand Down
40 changes: 39 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ Single tool with 3 actions:
The tool description includes instructions for the agent:
- **Sub-agent delegation:** Tool should be used by a sub-agent that returns only relevant extracted data, not raw dumps.
- **Batching:** Batch multiple URLs into a single run (e.g. `startUrls: [{url: "..."}, ...]`).
- **Known actors:** Compact comma-separated list of 57 actors across Instagram, Facebook, TikTok, YouTube, Google Maps, and more.
- **Known actors:** Compact comma-separated list of 20k+ actors across Instagram, Facebook, TikTok, YouTube, Google Maps, and more.
- **Support:** Directs users to integrations@apify.com for issues.

## Key Architecture Decisions
Expand Down Expand Up @@ -79,8 +79,46 @@ The wizard merges safely: preserves existing config, adds to `tools.alsoAllow` w
- **Type-check:** `npx tsc --noEmit`
- **Test:** `npx vitest run`
- **Pack (dry run):** `npm pack --dry-run`
- **Bump OpenClaw to latest:** `npm run bump:openclaw` *(install-tests against the latest stable openclaw, then updates `devDependencies.openclaw` + both `compat` fields if it passes; user reviews and commits)*
- **Current state:** 1 test file, 10 tests passing.

## Updating the OpenClaw Version

The `OpenClaw Version Test` workflow blocks any PR whose pinned openclaw version lags behind the latest stable on npm. To keep up:

1. Run `npm run bump:openclaw`. The script packs the plugin, installs it against `openclaw@latest` in a temp directory, runs `plugins list` + `plugins inspect` (same smoke as CI), then runs local `typecheck` + `vitest`. If any step fails, no files are touched.
2. On success, `package.json` (`devDependencies.openclaw`, `openclaw.compat.builtWithOpenClawVersion`, `openclaw.compat.pluginSdkVersion`) and `package-lock.json` are updated. Review `git diff`, then commit as `chore: bump openclaw to <version>`.
3. `peerDependencies.openclaw` uses `">="` and is intentionally not touched.

Claude should default to this script when asked to bump OpenClaw or when the version test is failing — do not run the underlying `npm install --save-dev openclaw@X` + `npm pkg set ...` commands manually.

## CI Workflows

### `ci.yml` — fast PR gate
- Runs `npx tsc --noEmit` + `npx vitest run` on Node 22.
- Triggers on push/PR to `main`.

### `openclaw_version_tests.yml` — OpenClaw runtime compatibility matrix
Validates the plugin actually installs and loads inside a real OpenClaw runtime.

- **Triggers:**
- `pull_request` to `main` — packs the PR branch (`npm install && npm pack`) and installs that tarball, so each PR is validated against its own diff.
- `schedule` — Mondays at `07:00` UTC (`0 7 * * 1`), which is 8am Prague (CET) in winter / 9am Prague (CEST) in summer.
- `workflow_dispatch` — manual re-run.
- **Schedule / dispatch mode:** installs `@apify/apify-openclaw-plugin@latest` from npm (validates the currently-shipping release).
- **Versions tested:** the `discover` job calls `npm view openclaw versions --json`, drops pre-releases (anything containing `-`), `sort -V | tail -n 3` to pick the latest 3 stable versions. Auto-updates — no manual list maintenance.
- **Smoke test per version:** all four phases run in **one consolidated bash step** that `cd`s to `$RUNNER_TEMP/openclaw-test` (splitting across steps with `working-directory:` caused a path mismatch — don't do that):
1. `npm install openclaw@<matrix-version>` in a fresh `$RUNNER_TEMP/openclaw-test` dir.
2. `npx openclaw plugins install <spec>` (tarball path on PR; `@latest` on schedule).
3. `npx openclaw plugins list` — must contain `apify-openclaw-plugin`.
4. `npx openclaw plugins inspect apify-openclaw-plugin --runtime --json` — must surface the `apify` tool name (matched loosely via `jq '.. | strings | select(. == "apify")'` since the JSON shape may evolve across OpenClaw versions).
- **Aggregator job (`required`)** — runs after the matrix with `if: always()`, fails if `discover` or `test` didn't succeed. This is the **stable required status check** in branch protection — the per-version matrix legs (`OpenClaw 2026.x.y`) rotate as discovery picks up new releases, so don't pin those.
- **Slack notification** — a `notify` job is scaffolded but commented out. To re-enable, uncomment it and add a `SLACK_WEBHOOK_URL` repo secret; it only fires on `schedule` failures.

### Branch protection on `main`
- `required` (the aggregator job above) is enforced as a required status check via the GitHub branch protection API.
- Set up via `gh api -X PUT repos/apify/apify-openclaw-plugin/branches/main/protection --input -` with a JSON body — the flag-form `-F nested.key=value` doesn't work for the nested object schema.

## Coding Style

- TypeScript (ESM). Prefer strict typing; avoid `any`.
Expand Down
13 changes: 10 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -180,15 +180,22 @@ The tool description instructs agents to delegate `apify` calls to a sub-agent.
npm install

# Type check
npx tsc --noEmit
npm run typecheck

# Run tests
npx vitest run
npm test

# Pack (dry run)
# Build compiled output to dist/ (required for publish)
npm run build

# Pack (dry run) — npm runs `prepublishOnly` (build) automatically before packing
npm pack --dry-run
```

`dist/` is generated by `npm run build` and is not checked in. The published npm
tarball ships `dist/` so newer OpenClaw versions (which no longer JIT-load
TypeScript) can install the plugin.

## Support

For issues with this integration, contact [integrations@apify.com](mailto:integrations@apify.com).
Expand Down
5 changes: 4 additions & 1 deletion openclaw.plugin.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
{
"id": "apify-openclaw-plugin",
"name": "Apify",
"description": "Universal web scraping and data extraction via Apify — scrape any platform using 57+ Actors across social media, maps, search, e-commerce, and more.",
"description": "Universal web scraping and data extraction via Apify — scrape any platform using 20k+ Actors across social media, maps, search, e-commerce, and more.",
"contracts": {
"tools": ["apify"]
},
"configSchema": {
"type": "object",
"additionalProperties": false,
Expand Down
Loading
Loading