Skip to content

Fix support for external trackers via VMC#1827

Open
Seneral wants to merge 14 commits into
SlimeVR:mainfrom
Seneral:contrib-VMC
Open

Fix support for external trackers via VMC#1827
Seneral wants to merge 14 commits into
SlimeVR:mainfrom
Seneral:contrib-VMC

Conversation

@Seneral
Copy link
Copy Markdown
Contributor

@Seneral Seneral commented Apr 29, 2026

This PR intends to fix the existing support for external tracking via VMC.

Known Issues:

  • OSCRouter spams "Packet is too large for the buffer in use"
    • Buffer is just randomly small, data is NOT unexpectedly large
    • No apparent reason why buffer is downsized, might be data race
    • WONT FIX: JavaOSC is slated to be replaced with a simpler reimplementation, no point fixing every last issue with it rn
  • OSC sometimes fails to listen when restarted due to refreshSettings
    • Causes OSC to stop working randomly, even when not in settings
    • WONT FIX: Hopefully addressed by osc reimplementation

Requires SlimeVR/SolarXR-Protocol#213

@github-actions github-actions Bot added Area: Server Related to the server Area: VMC Related to the VMC driver labels Apr 29, 2026
@github-actions github-actions Bot added Area: GUI Related to the GUI Area: VRCOSC Related to the VRChat OSC driver labels May 6, 2026
@github-actions github-actions Bot added the Area: Application Protocol Related to communication with apps like the GUI, overlay, games label May 8, 2026
@Seneral Seneral force-pushed the contrib-VMC branch 2 times, most recently from 4fb7572 to abcb6e3 Compare May 9, 2026 13:12
@github-actions github-actions Bot added Area: Skeletal Model Deals with the model of the skeleton and its pose Area: Translation Improvements or additions to translations labels May 9, 2026
@Seneral
Copy link
Copy Markdown
Contributor Author

Seneral commented May 9, 2026

The first few commits are in PR #1840
After that, this is ready for review, too

@Seneral Seneral force-pushed the contrib-VMC branch 2 times, most recently from 71f1cac to a5310e9 Compare May 10, 2026 23:16
@Seneral
Copy link
Copy Markdown
Contributor Author

Seneral commented May 10, 2026

Alright this should be just about everything needed for basic external tracker support via VMC.

There are a few things I'd change about the UI still:
I added a rescale tracking toggle to both OSC Router and VMC settings.
OSC Router really needs it, but it requires a VRM to be loaded in VMC settings. This is not obvious to the user.
The toggle in VMC is perhaps redundant, turning it off matches current behaviour without a VRM supplied at all.
I also don't like that it's not very obvious by just looking at the settings page why and when the VRM is needed.
I'd like to change it or associate it closely with that toggle somehow. Input appreciated.

Since IK Solver is now properly integrated (#142), tracking arms from controllers should now be deprecated IMO.
It's inferior in every way except if you absolutely need the hand position to match up, even if the IK solver determines the arm reach is insufficient.
This would also mean adding a warning to anybody that has "Force Arms From HMD" set to off - it should be on, the skeletal hierarchy should be one tree with head as the root, and IK solver resolving all positional constraints within (including hands).

The IK Solver should be able to handle trackers on any body part, though not optimally yet:

  • Two serial IK chains (e.g. head-hip-feet) would not optimally converge, can likely be easily addressed once I have those trackers to test with
  • Overconstrained scenarios (e.g. two slime arm trackers + positional hand tracker) may be unstable due to fixed lerp factor

Ideally, there'd be configuration in the GUI whether a given tracker needs positional calibration in IKConstraints.kt or not. Currently, an HMD on the head and hand-trackers are assumed to be aligned properly, all other trackers have a rudimentary offset calibration on reset (untested by me).

Finally, I'd consider not squashing some of these commits as they're quite varied and tracing back the reasoning for each of them would be painful without the context of the messages.
I'll open this for review after I address at least the remaining GUI issues.

Seneral added 14 commits May 11, 2026 16:32
Explicitly control skeleton height via head pos or estimation
Apply horizontal skeleton position to grid only to keep skeleton centered
This gives full positional feedback without modifying the existing visualisation behaviour
This misrepresented true yaw rotation used internally in the server
Very relevant if position is displayed, resulting in inconsistent visualisation
Intention likely was to prevent visual jumps on a reset
- only relevant for mocap mode without external head rotation
- if desired in the future, should be done server-side, just for mocap-mode
For accurate vertical positioning of a target VRM, its model is required
But when it is not provided, it should still work with the default scale
This patch doesn't scale the skeleton when the VRM height isn't provided
Whether the VMC label is human-readable may depend on the source
For OSCRouter and VRCOSC this is undesired, final application should handle timetags
Since these share the OSCPortIn with VMCHandler, this needs to dispatch immediately as well
Considering most timetags for tracking are in the past, this is fine
Any timetags for tracking data in the future are likely to be errors anyway
Fleshed out rescaling of VMC output based on VRM
Added option to rescale external tracking to avatar scale
- rescales all tracking data ("/Pos") routed through OSC router
Shoulders now always get updated with best guess
Also use shoulder tracker in absence of any other arm tracker (unlikely)
It is odd that upper-chest is seemingly mishandled
But if it is just assigned normally, VNyan will apply double yaw rotation
This should be better documented
Fix playspace setting being only visible when headtracker is used
- The whole point of it is to enable mocap mode
Reordered panels to make more sense
Resized some panels to fit in one view
@Seneral Seneral marked this pull request as ready for review May 11, 2026 14:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area: Application Protocol Related to communication with apps like the GUI, overlay, games Area: GUI Related to the GUI Area: Server Related to the server Area: Skeletal Model Deals with the model of the skeleton and its pose Area: Translation Improvements or additions to translations Area: VMC Related to the VMC driver Area: VRCOSC Related to the VRChat OSC driver

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant