Add video-file playback to HTMLVideoElement (src) for VideoTexture

[julapy]

What

Adds video-file playback to HTMLVideoElement, so Babylon.js content can play a video file/URL and use its frames as a VideoTexture — the same web API, unmodified Babylon.js:

const video = new HTMLVideoElement();
video.src = "app:///assets/clip.mp4"; // or an absolute path / http(s) URL
video.loop = true;
video.muted = true;
const texture = new BABYLON.VideoTexture("vid", video, scene, false, true, undefined, { autoPlay: true, loop: true });
material.emissiveTexture = texture;
// ...standard control API: video.play(), pause(), currentTime = 3.0, video.duration, events, etc.

Previously the HTMLVideoElement polyfill only supported srcObject (a camera MediaStream). This adds the src path alongside it; the two sources are mutually exclusive, matching the web. Any number of videos can play simultaneously (each element owns its own decoder + texture).

How

Commit 1 — VideoPlayer backend:

• VideoPlayer.h: a small platform abstraction (play/pause/loop/muted/volume/seek, duration/currentTime/dimensions, per-frame UpdateTexture, and load/ended/seeked events).
• Apple/VideoPlayer.mm: AVPlayer + AVPlayerItemVideoOutput requesting BGRA frames (no YUV pass), viewed zero-copy via CVMetalTextureCache and blitted into a persistent MTLTexture, wired to Babylon's InternalTexture with the same bgfx::overrideInternal-on-render-thread retry pattern as CameraDevice::UpdateCameraTexture. Loops via seek-to-zero.

Commit 2 — HTMLVideoElement.src:

• Extends NativeVideo with src, currentTime (get/seek), duration, loop, muted, paused, ended, isNative, and events (loadedmetadata/loadeddata/canplay/playing/pause/seeked/ended/resize). Player callbacks (which may fire on any thread) are marshalled onto the JS thread via JsRuntime::Dispatch; the element self-references while it owns a source so it isn't collected mid-playback. UpdateTexture routes to the camera stream or the video player depending on the active source.

Only the Apple backend is implemented here; other platforms return nullptr from VideoPlayer::Create (video src is a no-op, srcObject/camera unaffected).

Testing

Verified on device (iPhone 15 Pro Max, iOS 26.5, Xcode 26, Babylon.js 9.9.1 UMD) via the iOS Playground: a 2000×2000 h264 clip on two independent HTMLVideoElements, both looping, one seeked out of phase mid-run to exercise the control API — sustained 60 FPS (~1 ms GPU). Also runs concurrently with an active immersive-ar session and WebXR raw-camera-access plane in the same scene.

🤖 Generated with Claude Code

[bkaradzic-microsoft]

Latest bgfx can do hardware video decoding on D3D11, D3D12, Vulkan, and Metal. You might want to check that feature so that feature it's not macOS only.

[julapy]

hi @bkaradzic-microsoft - "latest bgfx" - is that already in BN and is it just a matter of hooking up video decoding?

[ryantrem]

Yes, bgfx is the rendering abstraction (over D3D, OpenGL, Vulkan, Metal) that Babylon Native uses.

[bkaradzic-microsoft]

@julapy I need sync with latest and then HW video decoding will be available in BN.