Impact of Multi-View Fusion and Biomechanical Modeling on Markerless Motion Tracking

This repository contains the code and utilities for the paper "Impact of Multi-View Fusion and Biomechanical Modeling on Markerless Motion Tracking".

📄 Paper

Title: Impact of Multi-View Fusion and Biomechanical Modeling on Markerless Motion Tracking
Link: IEEE Xplore

🏗️ Repository Structure

Code_github/
├── CV_methods/          # Links to external computer vision methods
├── preprocessing/       # Data preprocessing and method execution scripts
│   ├── scripts/        # Various configuration and execution scripts

🔬 Computer Vision Methods

This project evaluates and compares multiple state-of-the-art computer vision methods for human motion tracking. The following methods are used in this study:

Single-View Methods

• TCMR - Temporal Context Matters: Enhancing Single Image Prediction with Prior Motion
• VIBE - Video Inference for Human Body Pose and Shape Estimation
• GLoT - Global-to-Local modeling for video-based 3D human pose and shape estimation
• MPSNET - Multi-Person Scene Parsing
• NIKI - Neural Inverse Kinematics
• CLIFF - Carrying Location Information in Full Frames
• PARE - Part Attention Regressor for 3D Human Body Estimation
• HybrIK - A Hybrid Analytical-Neural Inverse Kinematics Solution
• HybrIKx - Extended HybrIK implementation
• HMR2.0 (4D-Humans) - Reconstructing and Tracking Humans with Transformers
• ReFit - Recurrent Fitting Network for 3D Human Recovery
• WHAM - Reconstructing World-grounded Humans with Accurate 3D Motion
• BEDLAM - A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motion

Multi-View Methods

• OpenCap - Open-source markerless motion capture

📁 Main Components

Preprocessing Scripts (preprocessing/scripts/)

Contains various configuration and execution scripts for:

• Data Configuration: Subject-specific and general data configuration files
• Bounding Box Refinement: Scripts for improving detection accuracy
• Method Execution: Scripts to run different CV methods (YOLO.)

🚀 Getting Started

Prerequisites

• Python 3.7+
• Required packages listed in each method's repository
• CUDA-capable GPU (recommended for deep learning methods)

Installation

1. Clone this repository
2. Install the required computer vision methods from their respective repositories (see CV_methods links above)
3. Install common dependencies:

pip install numpy pandas matplotlib opencv-python joblib

Usage

1. Configure your data paths in the appropriate data_config_*.py files
2. Run preprocessing scripts for bounding box detection and refinement
3. Execute the desired CV methods using the corresponding run_*.py scripts
4. Use utility functions for analysis and visualization

📝 Citation

If you use this code in your research, please cite:

@ARTICLE{11204549,
  author    = {Li, Zhixiong and Shin, Soyong and Phan, Vu and Meinders, Evy and Halilaj, Eni},
  title     = {Impact of Multi-View Fusion and Biomechanical Modeling on Markerless Motion Tracking},
  journal   = {IEEE Transactions on Biomedical Engineering},
  year      = {2025},
  volume    = {},
  number    = {},
  pages     = {1--10},
  keywords  = {Cameras; Biomechanics; Tracking; Computer vision; Biological system modeling; Accuracy; Legged locomotion; Computational modeling;                      Videos; Calibration; Biomechanics; Computer vision; Kinematics; Motion Measurement},
  doi       = {10.1109/TBME.2025.3622032}
}

🙏 Acknowledgments

We thank the authors of all the computer vision methods used in this study for making their code publicly available.