feat(grok): adopt media generation and voice transcription plumbing - #233
Merged
Conversation
Two more adoptions from the grok-cli deep-dive, implemented as pluggable hawk-side frameworks with a provider-neutral contract (the provider backends for image/video generation and speech-to-text live in eyrie and can be wired via SetMediaEngine / SetTranscriber). Media generation (internal/tool/GenerateMedia): - MediaEngine interface (GenerateImage/GenerateVideo) with provider-neutral options; SetMediaEngine installs a backend, nil by default with a clear error until wired. - The tool persists generated assets locally under the user state generated-media dir (the hawk analog of .grok/generated-media) and returns local paths + metadata, so media remains usable after provider URLs expire. - Supports text-to-image/video, source-based editing/image-to-video, MIME extension mapping, and a 50 MB download cap for provider-hosted URLs. - Registered in cmd/chat_tools.go with safety capabilities + permission aliases. Telegram voice transcription (internal/stt + gateway wiring): - stt package: Transcriber interface (SetTranscriber, Enabled), safe attachment download to a confined temp dir with a path-traversal guard and 50 MB cap, media-type extension inference, and a clear no-backend error. - Telegram gateway now parses voice/audio attachments, resolves file_id via getFile, downloads, transcribes when a backend is installed, and forwards the transcript prefixed with [Voice transcript]. Falls back to text when no transcriber is installed (existing behavior preserved). Out of scope: provider Batch API, X search, and desktop computer-use still require external provider endpoints or a native macOS accessibility backend that do not exist in eyrie today. Verification: go build ./... clean; new tests green (stt 7, GenerateMedia 5, telegram voice 2); tool, daemon, safety, cmd, engine and testaudit suites pass; golangci-lint 0 issues; gofmt clean.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adopts two more capabilities from the grok-cli deep-dive, implemented as pluggable hawk-side frameworks with a provider-neutral contract. The provider backends (image/video generation, speech-to-text) live in eyrie and can be wired via
SetMediaEngine/SetTranscriber.Changes
Media generation —
internal/tool/GenerateMediaMediaEngineinterface (GenerateImage/GenerateVideo) with provider-neutral options; nil by default with a clear error until wired.generated-mediadir (hawk analog of.grok/generated-media) and returns local paths + metadata, so media stays usable after provider URLs expire.cmd/chat_tools.gowith safety capabilities + permission aliases.Telegram voice transcription —
internal/stt+ gateway wiringsttpackage:Transcriberinterface (SetTranscriber,Enabled), safe attachment download into a confined temp dir with a path-traversal guard and 50 MB cap, media-type extension inference, clear no-backend error.file_idviagetFile, downloads, transcribes when a backend is installed, forwards transcript as[Voice transcript] <text>. Falls back to existing text behavior when no transcriber is installed.Out of scope
Provider Batch API, X search, and desktop computer-use still need external provider endpoints (in eyrie) or a native macOS accessibility backend that don't exist today.
Testing
internal/stt(7 tests),GenerateMediatool (5 tests), telegram voice (2 tests)go testsuite (incl. tool, daemon, safety, cmd, engine, testaudit),go vetgolangci-lint0 issues,gofmtclean