Add optional Stream QA pre-processor#17
Open
Trahloc wants to merge 1 commit into
Open
Conversation
Introduces rules.json and qa_validator.js for slop/loop detection, sampler scramble on supported APIs, and continue-via-generation-stopped trampoline. Default qaProfile is disabled so stock Sorcery behavior is unchanged. Co-authored-by: Cursor <cursoragent@cursor.com>
There was a problem hiding this comment.
Pull request overview
Adds an opt-in “Stream QA” pre-processor that can clean streamed text via regex rules and optionally scramble sampling parameters when slop/loop patterns are detected, while keeping Sorcery’s existing marker/script pipeline intact.
Changes:
- Add
qaProfilesetting (defaultdisabled) and a new settings UI dropdown. - Introduce
rules.json(profiles + regex rules) andqa_validator.js(rule loading, regex cleaning, sampler scrambling, stop/continue trampoline). - Hook Stream QA into the streaming pipeline ahead of marker processing.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| settings.js | Adds qaProfile to default extension settings. |
| settings.html | Adds Stream QA Profile dropdown to the extension settings UI. |
| rules.json | Defines QA profiles and regex patterns for slop/loop detection. |
| qa_validator.js | Implements rule loading/caching, regex compilation, cleaning logic, and sampler scrambling/restore. |
| main.js | Installs Stream QA into onProgressStreaming and wires the new dropdown to settings. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
125
to
127
| function installStreamHook() { | ||
| if (hookToBeInstalled) { | ||
| if (hookToBeInstalled || settings.qaProfile !== "disabled") { | ||
| const markerRegex = new RegExp(settings.markerRegex, "g"); |
| for (const [level, categories] of Object.entries(regexRules)) { | ||
| compiled[level] = {}; | ||
| for (const [category, patterns] of Object.entries(categories)) { | ||
| compiled[level][category] = patterns.map((pattern) => new RegExp(pattern, "gi")); |
Comment on lines
+54
to
+69
| function scrambleSamplers(profile) { | ||
| if (profile.temp_min == null || profile.top_k_min == null) return; | ||
| const target = getSamplerTarget(); | ||
| if (!target) return; | ||
| const { settings, tempKey, topKKey } = target; | ||
| if (!originalParams) { | ||
| originalParams = { | ||
| target, | ||
| temp: settings[tempKey], | ||
| top_k: settings[topKKey], | ||
| }; | ||
| } | ||
| const randomTemp = Math.random() * (profile.temp_max - profile.temp_min) + profile.temp_min; | ||
| const randomTopK = Math.floor(Math.random() * (profile.top_k_max - profile.top_k_min + 1)) + profile.top_k_min; | ||
| settings[tempKey] = parseFloat(randomTemp.toFixed(2)); | ||
| settings[topKKey] = randomTopK; |
| async function loadRules() { | ||
| if (rulesCache) return rulesCache; | ||
| try { | ||
| const response = await fetch("/scripts/extensions/third-party/sorcery/rules.json"); |
Comment on lines
+13
to
+21
| "looping": ["(.{15,})\\1+"] | ||
| }, | ||
| "medium": { | ||
| "slop": ["(?:testament to|tapestry of|delve into).+?", "\\b(delve|tapestry|testament)\\b"], | ||
| "looping": ["(.{30,})\\1+"] | ||
| }, | ||
| "low": { | ||
| "slop": ["(?:testament to|tapestry of).+?"], | ||
| "looping": ["(.{50,})\\1+"] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds an opt-in stream quality pre-processor inspired by DavidAU-style auto-correction, without changing Sorcery's marker/script model.
rules.json— profiles (temp/top_k bounds, regex strictness) and slop/loop patterns (community-tunable, no JS edits)qa_validator.js— standalone engine: pre-compiled regex, strip on hit, sampler scramble for textgen/openai/kobold APIs, stop/continue viaGENERATION_STOPPEDtrampoline +queueMicrotaskmain.js/settings.js/settings.html— ~25 lines: import, one hook line before marker logic, profile dropdown (defaultdisabled)Design constraints
qaProfile: "disabled"→ stock SorcerymatchAll/runScript; marker pipeline untouchedTest plan
qaProfile: disabled— identical to stock Sorcery (with/without scripts)textgenerationwebui_settingsMade with Cursor