|
| 1 | +# Screenshot to AI — Capture Page or Screen and Chat with AI |
| 2 | + |
| 3 | +**Date**: 2026-03-31 |
| 4 | + |
| 5 | +- New `📷` camera button injected into the AI chat input bar for screen/page capture |
| 6 | +- Dropdown menu with three capture modes: Capture Page, Capture Screen, Upload Image |
| 7 | +- Capture Page uses `html2canvas` to snapshot the current document; AI panel is hidden during capture for a clean shot |
| 8 | +- Capture Screen uses `getDisplayMedia` (browser screen-share picker) with live frame extraction from a hidden `<video>` element |
| 9 | +- Fixed black-screen capture bug: video element must be in DOM for GPU decoder to render real frames |
| 10 | +- Fixed black-screen capture bug: wait for `timeupdate` event instead of single `requestAnimationFrame` before grabbing frame |
| 11 | +- Fixed black-screen capture bug: canvas dimensions read from both `track.getSettings()` and `video.videoWidth/videoHeight` with explicit `> 0` guard |
| 12 | +- Captured image injected into `pendingAttachments` via `_ai.addFilesToPending()` and AI panel opened automatically |
| 13 | +- Auto-send fires after 900ms (covers panel open animation + async attachment processing) with a nested 150ms inner gap |
| 14 | +- Auto-send only pre-fills input if it is currently empty (preserves existing user text) |
| 15 | +- Self-healing button injection: `injectButtonIfMissing()` runs at module load, on AI-panel toggle clicks, and via 2-second fallback poll |
| 16 | +- Dropdown toggle uses inline `style.display` (not CSS classes) so it works even with stale cached `ai-panel.css` |
| 17 | +- All dropdown menu items use inline styles so the UI is fully CSS-cache independent |
| 18 | +- Flash overlay shown during capture via `showFlash()` / `hideFlash()` with `ai-screenshot-flash` element |
| 19 | +- Vision model warning toast if selected model does not support image input (non-blocking) |
| 20 | +- Module registered in `src/main.js` inside a `try/catch` wrapper to prevent initialization failures |
| 21 | +- Styles for wrapper, dropdown, and flash overlay added to `css/ai-panel.css` |
| 22 | +- Camera button HTML added to AI panel template in `js/modal-templates.js` |
| 23 | + |
| 24 | +--- |
| 25 | + |
| 26 | +## Summary |
| 27 | + |
| 28 | +Adds a **Screenshot to AI** feature: users click the 📷 camera button in the AI chat input bar, choose to capture the current page, share a specific screen/window, or upload an image, and the captured frame is automatically attached to the AI chat and sent for analysis. |
| 29 | + |
| 30 | +--- |
| 31 | + |
| 32 | +## 1. Self-Healing Camera Button Injection |
| 33 | +**Files:** `js/ai-screenshot.js`, `js/modal-templates.js` |
| 34 | +**What:** The camera button is included in the AI panel HTML template. In addition, `injectButtonIfMissing()` dynamically injects the button at runtime whenever `#ai-file-input` is found but `#ai-screenshot-btn` is missing — handling stale browser caches. It runs at module load, on AI-toggle click events (capture phase listener), and via a 2-second `setTimeout` fallback. |
| 35 | +**Impact:** The button appears reliably regardless of caching behaviour. |
| 36 | + |
| 37 | +## 2. CSS-Independent Dropdown |
| 38 | +**Files:** `js/ai-screenshot.js` |
| 39 | +**What:** The dropdown menu and all its items use inline `style` attributes instead of relying on `.active` CSS class toggling. The `show`/`hide` is managed by directly setting `menu.style.display`. |
| 40 | +**Impact:** The dropdown works correctly even when `ai-panel.css` is served from a stale browser cache that predates this feature. |
| 41 | + |
| 42 | +## 3. Capture Page (html2canvas) |
| 43 | +**Files:** `js/ai-screenshot.js` |
| 44 | +**What:** `capturePageScreenshot()` temporarily removes the AI panel from the DOM, waits 300ms for the close animation, runs `html2canvas` on `document.body` with `useCORS: true` and `scale: min(devicePixelRatio, 2)`, then restores the panel. Excluded elements: `#ai-panel`, `#ai-panel-overlay`, `.toast-container`, `#ai-screenshot-flash`. |
| 45 | +**Impact:** Produces a clean, full-page screenshot without the AI panel visible in the image. |
| 46 | + |
| 47 | +## 4. Capture Screen — Black Screen Fix |
| 48 | +**Files:** `js/ai-screenshot.js` |
| 49 | +**What:** `captureScreenScreenshot()` calls `getDisplayMedia`, appends the `<video>` element to DOM (hidden, `1×1px`, fixed off-screen), waits for `onloadedmetadata` + `video.play()`, then waits for the `timeupdate` event (fires only when real pixel data flows), then waits 2 additional `requestAnimationFrame` ticks, captures the frame to a canvas, stops tracks, and removes the video element. |
| 50 | +**Impact:** Eliminates completely black screenshots caused by capturing before the GPU video decoder had rendered any frames. `timeupdate` is the browser-guaranteed signal that real frame data is available. |
| 51 | + |
| 52 | +## 5. Auto-Send Timing Fix |
| 53 | +**Files:** `js/ai-screenshot.js` |
| 54 | +**What:** Changed auto-send `setTimeout` from 400ms to 900ms, with an inner 150ms gap before clicking the send button. Added a guard that skips pre-filling the input if the user has already typed something. |
| 55 | +**Impact:** Ensures the attachment is fully processed by `addFilesToPending` before the send button fires; prevents clobbering existing chat input. |
| 56 | + |
| 57 | +## 6. Module Registration |
| 58 | +**Files:** `src/main.js` |
| 59 | +**What:** `await import('../js/ai-screenshot.js')` wrapped in `try/catch` in the Phase 3d module loading block. |
| 60 | +**Impact:** A failure in `ai-screenshot.js` (e.g., missing vendor dependency) does not crash the rest of the application startup. |
| 61 | + |
| 62 | +## 7. Styles |
| 63 | +**Files:** `css/ai-panel.css` |
| 64 | +**What:** Added styles for `.ai-screenshot-wrapper`, `.ai-screenshot-btn`, `.ai-screenshot-menu`, `.ai-screenshot-item`, and `.ai-screenshot-flash` (full-screen flash overlay with `bi-camera` icon and message text). |
| 65 | +**Impact:** Provides the visual design for the camera button, dropdown, and capture feedback overlay. |
| 66 | + |
| 67 | +--- |
| 68 | + |
| 69 | +## Files Changed (4 total) |
| 70 | + |
| 71 | +| File | Lines Changed | Type | |
| 72 | +|------|:---:|------| |
| 73 | +| `js/ai-screenshot.js` | +300 | New module — full screenshot capture pipeline | |
| 74 | +| `css/ai-panel.css` | +148 | New styles for button, dropdown, and flash overlay | |
| 75 | +| `js/modal-templates.js` | +24 | Camera button HTML in AI panel input template | |
| 76 | +| `src/main.js` | +6 | Module registration with try/catch | |
0 commit comments