Theia3D Basics

Theia3D Basics

Theia3D is a markerless motion capture solution that utilizes synchronized video data to produce accurate and reliable 3D pose estimates of humans visible within the video data. It leverages deep learning algorithms trained to identify humans and accurately predict the 2D positions of 124 keypoints on the human body, in every video frame of every camera. By fitting a scaled subject-specific inverse kinematic model to the keypoint predictions, the human’s pose is reconstructed in 3D and tracked throughout their movement. This data-driven approach results in a robust solution that is generalizable across environments and movements, allowing the accessible collection of high quality 3D motion capture data where it was previously impossible.

Here, we describe the basic framework for collecting markerless motion capture data using Theia3D; for more detailed instructions, please refer to the appropriate section of this documentation and any accompanying videos.

Please referr to this blog post about the General Framework for Data Collections.

Calibration

Calibration is a crucial step for any 3D motion capture solution, and is equally important for Theia3D markerless motion capture data. The recommended calibration method depends on your camera system, but the concept and result is the same across all: determine the intrinsic and extrinsic parameters for all cameras in your system. These parameters allow lens effects and the 3D position and orientation of each camera to be determined, which is the key to producing robust 3D reconstructions. See Data Collection for details on recording intrinsic and extrinsic calibrations, and Calibration Menu for details on processing calibration trials.

The calibration methods supported by Theia3D include:

• Chessboard calibration, using a large printed chessboard pattern to automatically obtain intrinsic and extrinsic camera parameters.

• Third-party calibrations, such as wand calibrations implemented by third-party motion capture hardware suppliers.

• Object calibration, using an object with known dimensions or with precisely measured key point positions to manually obtain extrinsic camera parameters.

Movement Data Collection

Once you have obtained sufficient calibration data, the next step is collecting your movement trials. Theia3D produces robust 3D pose estimates across varying environments, for humans wearing typical body-fitting clothing, and is task-agnostic. Therefore, if the human(s) in your video are clearly visible and captured with appropriate resolution, frame rate, and exposure, the Theia3D algorithms can generally track their motion without issue. Record your movement data with relative freedom and ease, following the recommendations outlined in the Data Collection Principles section.

‍Data Processing

Having collected calibration and movement trials, you have everything required to process your markerless motion capture data. Theia3D includes several tools to help organize your data as required, process and check your camera system calibration, and analyze your movement trials. If you have collected numerous movement trials, you can use the accompanying Theia3D Batch application to batch analyze your trials without active supervision on your part; however, we always recommend manually checking your calibrations and a few trials in Theia3D first. Detailed information on the Theia3D Batch companion application can be found in Batch Processing .