From 72cd93385f8af6e404e50a80f2ddba569e30df4a Mon Sep 17 00:00:00 2001 From: Lino Giger <68745352+LinoGiger@users.noreply.github.com> Date: Mon, 13 Apr 2026 12:00:15 +0200 Subject: [PATCH 1/3] adjusted ai agents skills docs --- docs/ai_agents.md | 66 +++++++++++++++++++++++ docs/claude_code.md | 91 -------------------------------- docs/migration_guide.md | 112 ---------------------------------------- docs/quickstart.md | 2 +- mkdocs.yml | 10 ++-- 5 files changed, 70 insertions(+), 211 deletions(-) create mode 100644 docs/ai_agents.md delete mode 100644 docs/claude_code.md delete mode 100644 docs/migration_guide.md diff --git a/docs/ai_agents.md b/docs/ai_agents.md new file mode 100644 index 00000000..93d1e886 --- /dev/null +++ b/docs/ai_agents.md @@ -0,0 +1,66 @@ +# Rapidata in your AI agent + +Let your coding agent write the Rapidata integration for you. The official Rapidata skill teaches agents how to use the SDK — create labeling jobs, configure audiences, run benchmarks, and more — so you can just describe what you want in plain English. + +## Install + +Pick your agent. One command. Done. + +| Agent | Install | +|-------|---------| +| **Claude Code** | `claude plugin marketplace add RapidataAI/skills && claude plugin install rapidata-sdk-plugin@rapidata-sdk-marketplace` | +| **Cursor** | `npx skills add RapidataAI/skills -a cursor` | +| **Windsurf** | `npx skills add RapidataAI/skills -a windsurf` | +| **Copilot** | `npx skills add RapidataAI/skills -a github-copilot` | +| **Cline** | `npx skills add RapidataAI/skills -a cline` | +| **Codex** | `npx skills add RapidataAI/skills -a codex` | +| **Gemini CLI** | `npx skills add RapidataAI/skills -a gemini-cli` | +| **Any other** | `npx skills add RapidataAI/skills` | + +Install once. Works in every session after that. That's it. + +## Usage + +### Automatic + +The agent loads the skill when it sees Rapidata-related work. Just ask naturally: + +``` +Create a comparison job that evaluates image quality between two models +``` + +``` +Set up a custom audience with 3 qualification examples for prompt adherence +``` + +### Manual + +On Claude Code, invoke the skill directly: + +``` +/rapidata +``` + +``` +/rapidata How do I set up early stopping with a confidence threshold? +``` + +Other agents follow their own conventions — Cursor rules, Copilot instructions, etc. The skill activates whenever the file is loaded into context. + +??? note "No install — just the raw SKILL.md" + + If your framework doesn't match any of the above, drop the raw file into your agent's context: + + [**SKILL.md on GitHub**](https://github.com/RapidataAI/skills/blob/main/plugins/rapidata-sdk-plugin/skills/rapidata/SKILL.md) + + Raw URL for fetching: + + ``` + https://raw.githubusercontent.com/RapidataAI/skills/main/plugins/rapidata-sdk-plugin/skills/rapidata/SKILL.md + ``` + + Common integration patterns: + + - **Custom agent / SDK**: Load the file into your system prompt when the agent detects Rapidata-related work. + - **RAG / knowledge base**: Index it alongside your other docs and retrieve on Rapidata-related queries. + - **Anthropic Agent SDK**: Register it as a skill — the file already has the required frontmatter. diff --git a/docs/claude_code.md b/docs/claude_code.md deleted file mode 100644 index 59fc3683..00000000 --- a/docs/claude_code.md +++ /dev/null @@ -1,91 +0,0 @@ -# Using Rapidata with Claude Code - -The Rapidata SDK has an official [Claude Code](https://claude.com/product/claude-code) plugin that teaches Claude how to use the SDK. Once installed, Claude can help you write Rapidata integration code, create labeling jobs, set up audiences, and more - either automatically when it detects you're working with Rapidata, or on demand via the `/rapidata` command. - -## Installation - -### 1. Add the marketplace - -In Claude Code, run: - -``` -/plugin marketplace add rapidataAI/skills -``` - -### 2. Install the plugin - -``` -/plugin install rapidata-sdk-plugin@rapidata-sdk-marketplace -``` - -That's it. The `/rapidata` skill is now available. - -## Usage - -### Automatic - -Claude will automatically use its knowledge of the Rapidata SDK when it detects you're working with Rapidata code. Just ask naturally: - -``` -Create a comparison job that evaluates image quality between two models -``` - -``` -Set up a custom audience with 3 qualification examples for prompt adherence -``` - -### Manual - -Invoke the skill directly for SDK guidance: - -``` -/rapidata -``` - -Or with a specific question: - -``` -/rapidata How do I set up early stopping with a confidence threshold? -``` - -## What the plugin provides - -The plugin gives Claude detailed knowledge of: - -- **Classification, comparison, and ranking jobs** — creating job definitions, configuring parameters, and interpreting results -- **Audiences** — finding curated audiences, creating custom audiences with qualification examples, applying demographic filters -- **Flows** — continuous ranking without full job setup -- **MRI / Benchmarks** — comparing and ranking AI models on leaderboards -- **Settings** — `NoShuffleSetting`, `AllowNeitherBothSetting`, `MarkdownSetting`, and other display options -- **Error handling** — working with `FailedUploadException` for partial upload failures -- **Legacy order API** — backwards-compatible order creation - -## Keeping the plugin up to date - -To pull the latest version of the plugin: - -``` -/plugin marketplace update -``` - -## Adding the plugin to your project - -To make the plugin available to all contributors of a project automatically, add this to your project's `.claude/settings.json`: - -```json -{ - "extraKnownMarketplaces": { - "rapidata-sdk-marketplace": { - "source": { - "source": "github", - "repo": "rapidataAI/skills" - } - } - }, - "enabledPlugins": { - "rapidata-sdk-plugin@rapidata-sdk-marketplace": true - } -} -``` - -Contributors will be prompted to install the marketplace and plugin when they open the project. diff --git a/docs/migration_guide.md b/docs/migration_guide.md deleted file mode 100644 index 47f3a2f5..00000000 --- a/docs/migration_guide.md +++ /dev/null @@ -1,112 +0,0 @@ -# Migration Guide: Orders to Audiences - -This guide helps you migrate from the legacy Order API to the recommended Audience & Job API. - -!!! note - The Order and Validation APIs (`client.order`, `client.validation`) remain available for backwards compatibility in the near term. - -We observe higher quality responses when first curating an audience that is vetted on your specific task. While it was possible to do this with the previous Order API, the new Audience & Job API puts the recommended workflow front and center. The new paradigm revolves around two steps: - -1. **Curate an Audience**: Out of the millions of people available through Rapidata, select an audience that performs especially well on your task. This audience is curated by giving people validation examples and only accepting high performers. - -2. **Define the Job**: Define the annotation task in a Job Definition, which can then be assigned to an audience for processing. - -## Side-by-Side Comparison - -### Legacy: Orders - -```python -from rapidata import RapidataClient - -client = RapidataClient() - -# Step 1: Create validation set with quality control examples -validation_set = client.validation.create_compare_set( - name="Image Comparison Validation", - instruction="Which image follows the prompt better?", - datapoints=[["good_example.jpg", "bad_example.jpg"]], - truths=["good_example.jpg"], - contexts=["A sign that says 'Diffusion'."] -) - -# Step 2: Create order with validation set reference -order = client.order.create_compare_order( - name="Prompt Alignment Comparison", - instruction="Which image follows the prompt better?", - datapoints=[ - ["flux_image.jpg", "midjourney_image.jpg"], - ["flux_image2.jpg", "midjourney_image2.jpg"] - ], - contexts=["A cat on a chair", "A sunset over mountains"], - validation_set_id=validation_set.id, - responses_per_datapoint=10 -) - -# Step 3: Run and get results -order.run() -results = order.get_results() -``` - -### New: Audiences & Jobs - -```python -from rapidata import RapidataClient - -client = RapidataClient() - -# Step 1: Create audience with qualification examples built-in -audience = client.audience.create_audience(name="Prompt Alignment Judges") -audience.add_compare_example( - instruction="Which image follows the prompt better?", - datapoint=["good_example.jpg", "bad_example.jpg"], - truth="good_example.jpg", - context="A sign that says 'Diffusion'." -) - -# Step 2: Create job definition -job_definition = client.job.create_compare_job_definition( - name="Prompt Alignment Comparison", - instruction="Which image follows the prompt better?", - datapoints=[ - ["flux_image.jpg", "midjourney_image.jpg"], - ["flux_image2.jpg", "midjourney_image2.jpg"] - ], - contexts=["A cat on a chair", "A sunset over mountains"], - responses_per_datapoint=10 -) - -# Step 3: Assign and get results -job = audience.assign_job(job_definition) -results = job.get_results() -``` - -## What Changed - -| Legacy | Recommended | Notes | -|--------|-------------|-------| -| `client.order.create_*_order()` | `client.job.create_*_job_definition()` | Same parameters | -| `client.validation.create_*_set()` | `audience.add_*_example()` | Simpler API | -| `validation_set_id` parameter | Not needed | Built into audience | -| `filters` parameter | `filters` on audience creation | Applied at audience level | -| `order.run()` | `audience.assign_job(job_def)` | Automatic | - -## Key Benefits - -- **Simpler**: No separate validation set management -- **Reusable**: One audience can run multiple jobs or a job on multiple audiences -- **Previewable & Editable**: `job_definition.preview()` before assigning -- **Result Alignment**: The labelers in an audience are filtered according to your examples, improving the quality of the results - -## Quick Reference - -**Classification:** - -- `create_classification_order()` → `create_classification_job_definition()` -- `create_classification_set()` → `audience.add_classification_example()` - -**Compare:** - -- `create_compare_order()` → `create_compare_job_definition()` -- `create_compare_set()` → `audience.add_compare_example()` - -Results work the same way: `results.to_pandas()`, `results.to_json()` diff --git a/docs/quickstart.md b/docs/quickstart.md index 16cef354..16a6f506 100644 --- a/docs/quickstart.md +++ b/docs/quickstart.md @@ -175,4 +175,4 @@ print(results) - Learn about [Classification Jobs](examples/classify_job.md) for categorizing data - Understand the [Results Format](understanding_the_results.md) - Configure [Early Stopping](confidence_stopping.md) based on confidence thresholds -- Migrating from Orders? See the [Migration Guide](migration_guide.md) +- Let your [AI agent](ai_agents.md) write the integration code for you — one-line install for Claude Code, Cursor, Copilot, and 40+ more diff --git a/mkdocs.yml b/mkdocs.yml index 9b7f2a59..521a746f 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -72,10 +72,8 @@ plugins: - mri_advanced.md Flows: - flows.md - Claude Code: - - claude_code.md - Migration: - - migration_guide.md + AI Agents: + - ai_agents.md enable_markdown_urls: true enable_llms_txt: true enable_llms_full: true @@ -129,9 +127,7 @@ nav: - Model Ranking: - Getting Started: mri.md - Advanced: mri_advanced.md - - Resources: - - Claude Code: claude_code.md - - Migration Guide: migration_guide.md + - AI Agents: ai_agents.md theme: name: material From 8619dfd0e17944ffca04c83036c21f9d857a62c3 Mon Sep 17 00:00:00 2001 From: Lino Giger <68745352+LinoGiger@users.noreply.github.com> Date: Mon, 13 Apr 2026 12:08:31 +0200 Subject: [PATCH 2/3] added update section back --- docs/ai_agents.md | 39 ++++++++++++++++++++++++++++----------- docs/quickstart.md | 2 +- 2 files changed, 29 insertions(+), 12 deletions(-) diff --git a/docs/ai_agents.md b/docs/ai_agents.md index 93d1e886..a3284d12 100644 --- a/docs/ai_agents.md +++ b/docs/ai_agents.md @@ -19,6 +19,19 @@ Pick your agent. One command. Done. Install once. Works in every session after that. That's it. +??? note "No install — just the raw SKILL.md" + + If your framework doesn't match any of the above, drop the raw file into your agent's context: + + [**SKILL.md on GitHub**](https://github.com/RapidataAI/skills/blob/main/plugins/rapidata-sdk-plugin/skills/rapidata/SKILL.md) + + Raw URL for fetching: + + ``` + https://raw.githubusercontent.com/RapidataAI/skills/main/plugins/rapidata-sdk-plugin/skills/rapidata/SKILL.md + ``` + + ## Usage ### Automatic @@ -47,20 +60,24 @@ On Claude Code, invoke the skill directly: Other agents follow their own conventions — Cursor rules, Copilot instructions, etc. The skill activates whenever the file is loaded into context. -??? note "No install — just the raw SKILL.md" +## Keeping the skill up to date - If your framework doesn't match any of the above, drop the raw file into your agent's context: +The Rapidata SDK evolves constantly — new task types, new audience features, better defaults. Pull the latest version so your agent stays in sync. - [**SKILL.md on GitHub**](https://github.com/RapidataAI/skills/blob/main/plugins/rapidata-sdk-plugin/skills/rapidata/SKILL.md) +**Claude Code:** - Raw URL for fetching: +``` +/plugin marketplace update +``` - ``` - https://raw.githubusercontent.com/RapidataAI/skills/main/plugins/rapidata-sdk-plugin/skills/rapidata/SKILL.md - ``` +**Everything else:** - Common integration patterns: +```bash +npx skills update rapidata +``` + +Or update every skill you've installed at once: - - **Custom agent / SDK**: Load the file into your system prompt when the agent detects Rapidata-related work. - - **RAG / knowledge base**: Index it alongside your other docs and retrieve on Rapidata-related queries. - - **Anthropic Agent SDK**: Register it as a skill — the file already has the required frontmatter. +```bash +npx skills update +``` diff --git a/docs/quickstart.md b/docs/quickstart.md index 16a6f506..bad5ce05 100644 --- a/docs/quickstart.md +++ b/docs/quickstart.md @@ -175,4 +175,4 @@ print(results) - Learn about [Classification Jobs](examples/classify_job.md) for categorizing data - Understand the [Results Format](understanding_the_results.md) - Configure [Early Stopping](confidence_stopping.md) based on confidence thresholds -- Let your [AI agent](ai_agents.md) write the integration code for you — one-line install for Claude Code, Cursor, Copilot, and 40+ more +- Let your [AI agent](ai_agents.md) write the integration code for you — one-line install for Claude Code, Cursor, Copilot, and many more From e1b4ddd7c6738bc140015e5e8d2d9438d6ddb126 Mon Sep 17 00:00:00 2001 From: Lino Giger <68745352+LinoGiger@users.noreply.github.com> Date: Mon, 13 Apr 2026 12:15:48 +0200 Subject: [PATCH 3/3] update to terminal command --- docs/ai_agents.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/ai_agents.md b/docs/ai_agents.md index a3284d12..5e90f082 100644 --- a/docs/ai_agents.md +++ b/docs/ai_agents.md @@ -64,13 +64,13 @@ Other agents follow their own conventions — Cursor rules, Copilot instructions The Rapidata SDK evolves constantly — new task types, new audience features, better defaults. Pull the latest version so your agent stays in sync. -**Claude Code:** +**Claude Code**: -``` -/plugin marketplace update +```bash +claude plugin marketplace update ``` -**Everything else:** +**Everything else**: ```bash npx skills update rapidata