Rewrite synthetic data gen docs for Kiln Pro batch planning#13
Open
sfierro wants to merge 2 commits into
Open
Conversation
Rewrite the Set Up A Data Guide section for the v2 flow: Manual vs Kiln Pro chooser, the Kiln Pro copilot path (connect, add example inputs, generation settings, draft), and the shared Review and Refine cycle. Remove references to output examples, the output model picker, and applying the guide at output generation, which no longer exist. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Audit the Kiln app's synthetic data generation behavior and rewrite the docs to match, restructuring around the two build flows. Accuracy fixes (verified against the Kiln repo): - Fine-tuning split tag is fine_tune_data, not fine_tuning_data - Eval split tag names come from the eval's own tag filters; only the 80/20 ratio is fixed - Correct the eval template section: there are four eval-derived templates (Requirements Eval, Desired Behaviour Eval, Appropriate Tool Use, Issue Eval), auto-selected from the eval's type rather than picked from a menu of two - Document the six built-in templates - Refresh out-of-date model recommendations - Note the Data Guide applies at the topic and input stages only Structure: - Frame both flows as the same four stages, with only the first stage differing (Kiln Pro plans the batch; manual builds a topic tree), so inputs, outputs, and saving are documented once instead of twice - Scope templates and custom guidance to the manual flow, since the template dropdown and per-stage guidance are manual-only; Kiln Pro uses a single prefilled plan-level Guidance box - Move data gen model choice ahead of the build flows - Clarify that Manual refers to how coverage is decided, not to hand-authoring data Also repoint links in evaluations.md and fine-tuning-guide.md, including two that pointed at a #human-guidance anchor that did not exist. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Updates the Synthetic Data Generation docs for the new Kiln Pro batch planning flow, and corrects drift in the existing (manual) content. Behavior was audited against the Kiln app source rather than inferred.
Accuracy fixes
Verified against the Kiln repo:
fine_tune_data, notfine_tuning_dataStructure
The previous revision inserted the Pro flow alongside the untouched manual content, which left the two flows reading as unrelated and duplicated their shared steps.
Link fixes
evaluations.mdandfine-tuning-guide.mdrepointed to the current section#human-guidanceanchor that did not exist and were already broken before this changeNotes for review
<!-- TODO IMAGE -->/<!-- TODO VIDEO -->placeholders mark screenshots still needed for the Pro flow🤖 Generated with Claude Code