Fix RTL rendering for Hebrew and Arabic scripts - #76
Open
Jonathan-Asher wants to merge 40 commits into
Open
Conversation
… add state-change debounce to prevent DoS (CWE-400)
fix: add session token auth and connection limit to DirectorServer
fix: enforce connection limit, offload broadcast to background queue,…
The page sidebar was only visible when multiple pages existed, but the "Add Page" button lives inside the sidebar — making it impossible to add pages from a single-page state. Also, pressing play always reset to page 0 (the welcome text) instead of reading whichever page the user was currently editing. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds customizable color and brightness controls for teleprompter annotation cues (e.g. [pause], [smile], [breath]). Previously these were hardcoded to white at fixed opacities. - Cue Color: 6 color presets (matches highlight color options) - Cue Brightness: 4 levels (Dim, Low, Medium, Bright) - Settings preview includes [pause] sample annotation - Remote viewer (BrowserServer) updated to use cue color - DirectorServer state includes cue color for consistency Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
fix: always show page sidebar and read current page on play
feat: add cue color and brightness settings for annotation text
In Classic and Voice-Activated modes, the overlay would immediately switch to a "Done" screen and auto-dismiss 1 second after the auto-scroll reached the last word. This made timed mode unusable because speakers typically finish talking 10-20 seconds after the scroll ends. Now in timer-based modes (Classic/Voice-Activated), when the scroll reaches the end on the last page: - The prompter text stays visible instead of switching to "Done" - The overlay does not auto-dismiss - The speaker can close manually via the X button or Esc key Word Tracking mode behavior is unchanged (auto-dismiss is appropriate there since it knows when the speaker actually finishes). Fixes f#29
Address code review findings: - ExternalDisplayView: gate doneView and speechRecognizer.stop() on wordTracking mode, matching NotchOverlayView/FloatingOverlayView - BrowserServer: suppress isDone in classic/silencePaused modes on last page so browser clients keep showing prompter text - Revert accidental DEVELOPMENT_TEAM change in project.pbxproj
The timerWordProgress was incrementing unboundedly after the scroll reached the end, unlike the SwiftUI views which guard with !isDone. Add a scrollDone check before incrementing to stop wasting CPU on the 100ms broadcast timer.
SFSpeechRecognizer silently returns no results when receiving multi-channel audio buffers. USB audio interfaces like the RODECaster Pro II send 2-channel 48kHz audio, and the previous `format: nil` tap delivered these buffers unchanged to the recognition request. Create a mono AVAudioFormat at the hardware sample rate when the device has more than one channel and pass it to installTap, letting AVAudioEngine handle the downmix automatically. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Introduce an overlay transparency mode that uses a blurred background and adjustable tint opacity. - Add NotchBlurView (NSVisualEffectView wrapper) to provide behind-window blur. - Update NotchOverlayView and preview to render either a solid black island or a blurred, clipped island with a dark tint driven by opacity. - Add NotchSettings properties overlayTransparency and overlayTransparencyOpacity, persisted via UserDefaults (keys: "overlayTransparency", "overlayTransparencyOpacity") and initialized with sensible defaults. - Expose a Toggle and an opacity Slider in SettingsView to enable transparency and control amount; update reset defaults to include the new settings. This enables a see-through notch overlay option where desktop content shows through while keeping text readable via a configurable dark tint.
Add overlay transparency with blur and slider
…cognition fix: downmix multi-channel audio to mono for SFSpeechRecognizer
…pm-scroll Fix: Keep text visible after WPM auto-scroll reaches the end
Addresses two user-reported bugs: (1) highlight not tracking at the right speed, jumping erratically or lagging behind speech, and (2) mic appearing to stall out and stop picking up audio after ~60 seconds. Root causes identified and fixed: **Seamless recognition restart (P0)** - Split cleanupRecognition() so AVAudioEngine stays alive across SFSpeechRecognitionTask restarts, eliminating audio gaps - Add pre-emptive 55-second restart timer to beat Apple's ~60s timeout - Update matchStartOffset to recognizedCharCount before each restart so new sessions match from the correct position - Thread-safe request swapping via NSLock for audio I/O thread safety - Add contextualStrings from remaining source text for better STT accuracy **Fix fuzzy matching false positives (P1)** - Remove overly permissive `contains` check from isFuzzyMatch that caused "and" to match "demand", "the" to match "other", etc. - Tighten prefix matching to require minimum 3-char words - Require exact match for 2-char words (no edit distance tolerance) - Fix charLevelMatch skip-both fallback: no longer advances lastGoodOrigIndex on genuine mismatches (gibberish no longer matches) - Fix wordLevelMatch +1 space overcount on last matched word - Fix unicode scalar vs Character count mismatch in charLevelMatch **Confidence gating (P2)** - Replace blind max(charResult, wordResult) with agreement-based selection - Add sliding window requiring 2-of-3 recent results to agree before committing large forward jumps (small steps always pass through) **Retry resilience (P3)** - Distinguish timeout errors (code 1110/216) from real errors - No retry limit for expected timeouts; immediate soft restart - Backoff with retry limit only for genuine errors **Architecture cleanup (P4)** - Merge two polling timers in observeDismiss() into one - Fix retain cycle in dismiss() asyncAfter closure - Add isDismissing guard to prevent double-dismiss - Fix cancelled-task error callback race in restartTask() Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ned mode Two bugs fixed: 1. Clicking a page in the sidebar then pressing Play always started from page 1 instead of the selected page. Root cause: the `sidebarSelection` setter wrapped the `currentPageIndex` update in `DispatchQueue.main.async`, deferring it asynchronously. Since SwiftUI binding setters are already called on the main thread, this wrapper was unnecessary and caused `run()` to read the stale value (0) before the update applied. Fix: remove the async dispatch so `currentPageIndex` is updated synchronously on selection. 2. `showPinned()` was the only display mode not calling `installKeyMonitor()`, so the ESC key did not work to dismiss the overlay in pinned mode. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fix: start playback from selected page and install key monitor in pin…
fix: voice tracking highlight and mic stall bugs
jumpTo tore down the recognition task and started a fresh one, dropping the audio spoken during the restart window (~1s to warm) — exactly the words the user re-speaks right after tapping a word to jump, so the highlight appeared to stop following. Jumps now keep the task running and record a transcript anchor: matching ignores everything transcribed before the jump and continues from the new offset with zero dropped audio. The anchor resets whenever a new task starts a fresh transcript (soft restart, new session). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
beginRecognition and restartTask both gave up permanently when SFSpeechRecognizer reported unavailable — a state that is usually transient (the recognition service churns briefly after a task cancellation or an audio device config change). beginRecognition's guard didn't even set isListening = false, so the app went silently deaf: engine stopped by the preceding cleanup, no retry, no error surfaced, until a full app relaunch. Observed live: a USB mic (Elgato Wave) sample-rate renegotiation ~100ms after engine start triggered restartTask, the cancelled task's error callback scheduled a fresh beginRecognition, and that hit the unavailable guard and returned. Both guards now retry with the same backoff the invalid-format guard already uses. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Findings from an adversarial self-review of the first two commits: - restartTask/beginRecognition now clear lastSpokenText along with the anchor: a jump taken after a task restart but before its first result anchored on the old transcript and froze matching until the next ~55s restart. - The anchor is now the transcript prefix string, trimmed by surviving common prefix (with bounded slack) instead of a raw char count, so the recognizer revising earlier text no longer swallows post-jump speech or leaks the pre-jump transcript. - Results delivered within 300ms of a jump are ignored — they were computed against the pre-jump position. - jumpTo no longer resets retryCount (tapping words could keep a failing availability-retry loop alive forever) and falls back to a full restart for far jumps (>500 chars, refreshing contextualStrings for the new section) or when the engine died silently. - A nil SFSpeechRecognizer (unsupported locale) fails fast instead of silently retrying a permanent condition; exhausted retries in restartTask stop the audio engine instead of leaving the mic hot. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
fix: recognition resilience — keep task alive across jumps, retry transient recognizer unavailability
… mode - Skip bracketed notes and highlight the following spoken words - Commit multi-word phrases when both match strategies progress - Catch up during fast reading by preferring the further match - Widen resync windows for dropped/inserted words - Voice-Activated mode runs on microphone energy without speech recognition
…cripts (f#63) - MarqueeTextView: Detect RTL content via isRTLUnicodeScalar(), flip VStack alignment to .trailing for RTL, and set .environment(\.layoutDirection) so word flow lines render right-to-left. - HighlightingTextEditor: Add isRTLUnicodeScalar() and containsRTLText() helpers, plus updateWritingDirection() that sets textView.baseWritingDirection to .rightToLeft when RTL content is detected, ensuring proper caret and text alignment in the editor. Fixes garbled/truncated display when dictating or editing Arabic, Hebrew, Persian, Urdu, and other RTL text. Co-authored-by: Hermes Agent <hermes@nousresearch.com> Co-authored-by: Fatih Kadir Akın <f@users.noreply.github.com>
The teleprompter had partial RTL support that broke down in four ways.
Word spacing collapsed entirely. `WordFlowLayout` spaced words by appending
a trailing space to each `Text` ("word "). Text layout trims trailing
whitespace when measuring, so in an RTL line every gap measured zero and the
whole line ran together as one unbroken string of characters. Words are now
spaced by the enclosing `HStack` instead, which is direction-agnostic.
Embedded LTR runs were reversed. Flipping `layoutDirection` reverses every
word box on the line, including consecutive Latin words that should stay in
reading order among themselves — "Claude Code" inside a Hebrew sentence
rendered as "Code Claude". Each line is now reordered by the Unicode
Bidirectional Algorithm's L2 rule at word granularity, so only the runs that
need reversing are reversed, and each word is wrapped in a directional
isolate so its own punctuation and bracket mirroring resolve against the run
it belongs to.
The editor lost its direction on every keystroke. `applyHighlighting` resets
all attributes over the full range, wiping the paragraph style that carried
the writing direction, so Hebrew jumped back to left-aligned as soon as you
typed. The paragraph style is now part of the default attribute set and of
the typing attributes.
The browser teleprompter treated every Hebrew word as a stage cue. Its
`isAnnotation` and `letterCount` helpers tested against a hardcoded list of
character ranges that omitted Hebrew, Arabic, Greek, Thai and more, so those
words were classified as emoji-only: rendered dim and italic, skipped by the
highlight scan, and never scrolled to. Both helpers now use `\p{L}`/`\p{N}`,
matching the native side's `isLetter || isNumber`, and the text container
gets a `dir` attribute so the browser's own bidi engine orders the line.
Base direction is now taken from the dominant script rather than the first
strong character, so a Hebrew script that opens with a Latin product name or
speaker label still lays out right-to-left. The director page's script panes
get `dir="auto"`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reading a Hebrew script in the teleprompter is currently not usable. There is partial RTL support already (
TextDirection.swift, thelayoutDirectionflip inWordFlowLayout), but it breaks down in four independent ways.1. Word spacing collapses entirely
WordFlowLayoutspaces words by appending a trailing space to eachText(Text(item.word + " ")). Text layout trims trailing whitespace when measuring, so in an RTL line every gap measures zero and the whole line renders as one unbroken string:Words are now spaced by the enclosing
HStackinstead, which is direction-agnostic. LTR output is unchanged.2. Embedded LTR runs render reversed
Flipping
layoutDirectionreverses every word box on the line, including consecutive Latin words that should stay in reading order among themselves:For the input
...והיא עובדת מצוין עם Claude Code על מק, the embedded Latin pair renders as Code Claude before this change and Claude Code after it.Each line is now reordered with the Unicode Bidirectional Algorithm's L2 reordering rule applied at word granularity (
bidiVisualOrder), so only the runs that need reversing are reversed. Each word is additionally wrapped in a directional isolate (RLI/LRI…PDI) so its own trailing punctuation and bracket mirroring resolve against the run it belongs to rather than against the ambient layout direction — otherwiseשלום,puts the comma on the wrong side.Numbers are treated as LTR, matching UBA rule I1 (European numbers always take an even embedding level), so
1.6.3inside Hebrew text keeps its digits in order while still sitting at the correct point in the RTL flow.3. The editor loses its direction on every keystroke
HighlightingTextEditor.updateWritingDirectionsets the base writing direction, butapplyHighlightingthen callstextStorage.setAttributesover the full range with only.fontand.foregroundColor— wiping the paragraph style that carries the direction. IntextDidChangethe two run in that exact order, so Hebrew snaps back to left-aligned as soon as you type. The paragraph style is now part of the default attribute set and oftypingAttributes/defaultParagraphStyle, so it survives the reset.4. The browser teleprompter treats every Hebrew word as a stage cue
isAnnotationandletterCountin the served HTML test against a hardcoded range list:/[a-zA-Z0-9À-ɏЀ-ӿ -鿿가-]/Hebrew, Arabic, Greek, Thai, Devanagari and others are absent, so every word in those scripts is classified as emoji-only: rendered dim and italic in the cue colour, skipped by the next-word scan, and never scrolled to. The browser view is completely non-functional for these languages. Both helpers now use
\p{L}/\p{N}, which matches the native side'sisLetter || isNumber, and the text container gets adirattribute so the browser's own bidi engine orders each line.Base direction heuristic
textBaseDirectionuses first-strong detection, which misreads a script opening with a Latin product name or speaker label — a single leadingTextreamlays an entire Hebrew page out left-to-right. AddeddominantBaseDirection, which picks whichever script has more letters; scripts are read whole, so the dominant script is the better signal at page level.textBaseDirectionis left in place.Also added
dir="auto"to the director page's script panes.Verification
Rendered
WordFlowLayoutdirectly throughImageRendereracross four fixtures — pure Hebrew, Hebrew with embedded English and numbers, Hebrew punctuation, and an English control. All three Hebrew cases are correct after the change; the English control is pixel-identical before and after. The browser-side classification and direction helpers were checked against Hebrew, Arabic, CJK, Greek, Latin, digit-only, emoji and bracketed-cue inputs.Built and running locally on macOS 15 (Apple Silicon).
🤖 Generated with Claude Code