Looking for a quick walk-through of a tool or feature? Check out our Tutorial Videos playlist for a complete overview of available videos.

Or, navigate to the relevant section of this documentation and you'll find any specific videos embedded there.

If you encounter any issues or topics not covered by these resources, please feel free to contact our support team at [email protected].

2025.1.7

• Released 2025-10-22

  • Minor bug fixes and improvements

2025.1.6

• Released 2025-08-05

  • Minor bug fixes and improvements

New Theia Self-Service Portal

We’re excited to announce that Theia’s new customer portal is now live and can be found at . This self-service portal gives you greater control over your software licenses and downloads. Additional portal informaion can be found in the section * Device management features (such as activating or deactivating machines) will not be available until the new licensing system goes live on August 11.

2025.1.2

• Improved visual alignment of the toe and foot segments in the skeleton visualization.

  • This update enhances display accuracy only, no model changes

2025.1.1

• Added support for NVIDIA GeForce RTX 50 series graphics cards

To learn more about performance improvements with this new generation of graphics cards, check out our recent .

2025.1.0

Theia3D

• Theia theme and rebranding

• Improved error messages in dialogs, including links to troubleshooting pages

• Added links to web pages with additional information for tools and interfaces throughout

• Organize Videos Advanced tab:

Theia3D Batch

• Theia theme and rebranding

• Save Videos option in Batch processing

  • Select display options and specify camera views to save

• Display Errors table added to simplify viewing warnings and errors

To read about the update to Theia Axiom, check out our .

Check Calibration

The extrinsic camera calibration can be checked after the trial has been analyzed fully. For each view, the 2D representation of the skeleton in that view is compared to the 3D skeleton computed from all views. If the alignment can be improved in at least 30% of the frames by translating the entire skeleton within the 2D image for that view, this information is reported in the dialog. You can then choose to deactivate the cameras that are out of calibration and clear the entire analysis.

Overview

Theia3D is an advanced deep learning algorithm-based software which runs and processes data on your local PC. This means that there are specific requirements in order for it to operate properly and perform as intended, which are laid out below.

Extrinsic object calibration trials are used to determine the position and orientation of every camera in your system relative to the desired global coordinate system. Extrinsic object calibration uses the static position of a calibration object with known dimensions or known positions of specific key points on the object. These 3D dimensions or positions must be measured with high precision. The calibration object should be sufficiently large or the calibration key points should be spaced far apart within the capture volume and most points should be visible in every camera view. The key points can be coplanar or can vary in all three global dimensions.

A calibration object file must be created for your calibration object, containing 3D coordinates of the object key points. Each line of the file must contain three comma-separated values representing the global x-, y-, and z-coordinates of the key point, in millimeters. Each key point should be on its own line.

Object calibrations must be performed at the resolution that will be used to record your movement data. If you will be using more than one resolution during your data collection, you must record object calibration trials at each resolution.

After installing Theia software, it is necessary to activate the license prior to using the software. To activate your Theia3D license, run Theia3D as an administrator by right-clicking on the icon and choosing Run as administrator.

When running Theia3D as an administrator, a prompt to enter the license key will appear, as below. Enter your license key in this window and click Activate. You can access your license key from the using the login information provided at the time of purchase.

If the PC is connected to the internet, the license will automatically activate and Theia3D will open.

If the PC is not connected to the internet, a dialog window similar to that shown below will appear, indicating that it is necessary to contact Theia support via our or by emailing [email protected].

After contacting Theia Support, enter the Activation Code as provided by our team and click

Graph Joint Angles

• Subject: The person to plot Euler angles for.

The startup window appears if Select GPUs on launch is enabled in the Theia3D .

GPU Selection‍

Select which graphics card(s) to use when running Theia3D.

Camera Setup

Camera system setups differ from location to location, and may be subject to challenging data collection environment constraints. General recommendations for setting up your camera system include:

Most of the Display options shown here can also be toggled ON and OFF using the Display toggle buttons in the sidebar.

Show/Hide 3D View

Toggle showing the 3D View in the application. If toggled ON but the 3D View is not visible, drag the 3D View open from the right border of the application window.

Cancel Analysis

Cancel the currently running analysis. This option is available (and replaces all other options) only when analysis is running.

Theia3D Basics

Theia3D is a markerless motion capture solution that utilizes synchronized video data to produce accurate and reliable 3D pose estimates of humans visible within the video data. It leverages deep learning algorithms trained to identify humans and accurately predict the 2D positions of 124 keypoints on the human body, in every video frame of every camera. By fitting a scaled subject-specific inverse kinematic model to the keypoint predictions, the human’s pose is reconstructed in 3D and tracked throughout their movement. This data-driven approach results in a robust solution that is generalizable across environments and movements, allowing the accessible collection of high quality 3D motion capture data where it was previously impossible.

Here, we describe the basic framework for collecting markerless motion capture data using Theia3D; for more detailed instructions, please refer to the appropriate section of this documentation and any accompanying videos.

Please referr to this blog post about the General Framework for Data Collections.

Calibration

Calibration is a crucial step for any 3D motion capture solution, and is equally important for Theia3D markerless motion capture data. The recommended calibration method depends on your camera system, but the concept and result is the same across all: determine the intrinsic and extrinsic parameters for all cameras in your system. These parameters allow lens effects and the 3D position and orientation of each camera to be determined, which is the key to producing robust 3D reconstructions. See for details on recording intrinsic and extrinsic calibrations, and for details on processing calibration trials.

The calibration methods supported by Theia3D include:

• Chessboard calibration, using a large printed chessboard pattern to automatically obtain intrinsic and extrinsic camera parameters.

• Third-party calibrations, such as wand calibrations implemented by third-party motion capture hardware suppliers.

• Object calibration, using an object with known dimensions or with precisely measured key point positions to manually obtain extrinsic camera parameters.

Movement Data Collection

Once you have obtained sufficient calibration data, the next step is collecting your movement trials. Theia3D produces robust 3D pose estimates across varying environments, for humans wearing typical body-fitting clothing, and is task-agnostic. Therefore, if the human(s) in your video are clearly visible and captured with appropriate resolution, frame rate, and exposure, the Theia3D algorithms can generally track their motion without issue. Record your movement data with relative freedom and ease, following the recommendations outlined in the section.

‍Data Processing

Having collected calibration and movement trials, you have everything required to process your markerless motion capture data. Theia3D includes several tools to help organize your data as required, process and check your camera system calibration, and analyze your movement trials. If you have collected numerous movement trials, you can use the accompanying Theia3D Batch application to batch analyze your trials without active supervision on your part; however, we always recommend manually checking your calibrations and a few trials in Theia3D first. Detailed information on the Theia3D Batch companion application can be found in .

What Will You Do?

Theia3D is the premier solution for accurate, reliable, and generalizable markerless motion capture. With our software at your fingertips, you'll unlock the power to analyze human movement like never before. Whether you're a researcher exploring human movement, a sports biomechanist tasked with unleashing athletes' full potential, or a product developer working on the next generation of athletic equipment, Theia3D empowers you to:

• Obtain motion data without the need for suits, sensors, or markers.

• Transform raw video footage into precise, 3D models of human movement.

• Integrate effortlessly with leading motion capture software.

• Dive into the future of motion capture and elevate your project to new heights.

This online resource acts as a one-stop shop for almost any question related to using and getting the most from Theia3D. Visiting for the first time? Check out the and sections. Looking for details on a specific setting or tool? Look under the section header corresponding to the relevant dropdown menu from the software.

Still looking for more information or find something that's missing? We'd love to hear from you directly! Please email or .

Using this Documentation

New users should start at the guide. This guide contains tutorials that introduce you to the software interface and walk you through the most common workflows - such as calibration and analysis - so that you can harness the power of Theia3D for yourself.

The reference guides contain detailed information about the interface, settings, and tools available in Theia3D. They are excellent resources for advanced users looking to fine-tune workflows and optimize Theia3D for their specific use cases.

Contact Us

For any questions and issues, please reach out to .

Intrinsic lens calibration trials are used to determine parameters associated with the camera lenses and are used to correct for distortion and other visual effects. Lens calibration trials are required for all camera systems that make use of a chessboard calibration method, except for Sony RX0-ii cameras. Users with wand calibration-capable systems (e.g. Qualisys Miqus, Vicon FLIR Blackfly S systems) are not required to complete lens calibration unless they are planning to use the chessboard calibration method.

Recommendations:

• Record lens calibration trials at a relatively low frame rate (20-30 Hz) to reduce file size and processing time.

• Keep the chessboard as flat as possible during the calibration trial.

• Use a computer monitor facing the person performing the calibration to provide visual feedback during the calibration.

• Move slowly and deliberately to prevent chessboard blur.

Recording Lens Calibrations:

1. Place all cameras side by side on a desk or table, facing the same direction and capturing as similar views as possible. It is generally easier to calibrate the lenses with the cameras in 'landscape' orientation.

2. Stand at a distance where the chessboard occupies approximately 1/4 of the camera views.

3. Begin the recording.

4. Slowly move the chessboard in a systematic grid pattern, covering the entire field of view for every camera. Ensure the chessboard goes slightly beyond every edge of every camera field of view, and every corner.

1. Take a step back, and repeat Step 4., covering the camera fields of view again.

2. While covering the field of view, angle the chessboard slightly in multiple directions, varying its orientation throughout.

3. When you are confident that the entire field of view has been covered for all cameras, end the recording.

To process intrinsic lens calibrations, see .

Camera Setup & Position

When setting up cameras, use full circle or oval configurations with multiple sagittal and frontal views. Position cameras at a height of 4 to 8 feet (1 to 2.5 meters).

Aiming for the participant to appear approximately 500 pixels tall in the frame.

• ~½ frame height (landscape @ 1080p)

• ~¼ frame height (portrait @ 1080p)

Avoid unusual angles and strive to keep all cameras flat and level for consistent data capture.

For more information please refer to our

Lighting

​Proper lighting is crucial for high-quality video data in markerless motion capture. Outdoors, be mindful of the sun's position to avoid harsh backlighting and adjust camera settings to prevent overexposed or dark images. Indoors, ensure adequate lighting, as ambient levels are typically lower; aim for at least 500 lux, with 1,000 lux preferred. Be cautious of natural light sources like windows, which can cause silhouette effects.

Use light sources with a color temperature between 3,500K and 6,500K and a Color Rendering Index (CRI) of 70 or higher for accurate color representation. Diffuse, non-directional lighting minimizes shadows and glare, enhancing video quality. Avoid light sources that flicker near your recording frame rate to prevent inconsistent brightness

For more information please refer to our .

Frame Rate

​Selecting the appropriate frame rate is crucial for accurately capturing motion in data collection. A frame rate too low may miss subtle movements, while an excessively high frame rate can degrade video quality and result in unnecessarily large data files and long processing times. To determine the optimal frame rate, start with a literature review to understand standard practices for similar studies. Next, assess your camera hardware's capabilities, ensuring it can support the desired frame rate without compromising resolution. Some cameras offer 'windowing' features to increase frame rates by capturing only a portion of the sensor; however, this should be used cautiously to avoid missing parts of the subject. Conduct pilot studies to test different frame rates, reviewing the clarity and quality of the captured movements. Be mindful that higher frame rates may require more lighting and can lead to issues like frame drops due to increased data throughput. Balancing these factors through thorough testing and consideration will help you select a frame rate that meets your experiment's needs.

For more information please refer to our

Extrinsic chessboard calibration trials are used to determine the position and orientation of every camera in your camera system relative to your desired global coordinate system. As with lens calibration trials, users with wand calibration-capable systems (e.g. Qualisys Miqus, Vicon FLIR Blackfly S systems) are not required to complete chessboard calibration.

A minimum of one chessboard calibration trial should be collected every time you set up your camera system, but collecting multiple is recommended. A new calibration is required any time a camera is moved, so having multiple calibration trials collected throughout a long data collection session can help prevent data loss due to accidental or unnoticed camera movements.

Recommendations:

• Record chessboard calibration trials at a relatively low frame rate (20-30 Hz) to reduce file size and processing time.

• Keep the chessboard as flat as possible during the calibration trial.

• Focus on achieving groupings of 3+ cameras that can see the chessboard at all times.

• Align the chessboard with marks on the ground so you can confirm that the global coordinate system is positioned correctly when processing movement trials.

Avoid:

• Obstructions and occlusions; be aware of any objects within your capture volume which may partially or completely block the chessboard pattern from the view of the cameras. Even partial obstructions of the chessboard pattern may prevent it from being detected.

  • For treadmill capture volumes, be aware of treadmill safety rails, harnesses, etc.

  • For athletic environments, be aware of safety netting or equipment. Where possible, move or remove netting during calibration to ensure a direct view of the calibration chessboard.

Recording Chessboard Calibrations:

1. Set up your camera system as desired for your data collection, following the recommendations in .

2. Choose or place visible marks on the ground which will be aligned with the global coordinate system, for confirmation when processing data.

3. Check that the chessboard is fully visible in at least 3 camera views when it is placed at the desired global coordinate system origin position.

4. Begin the recording.

To process extrinsic chessboard calibrations, see .

Installation Steps

The Modify People IDs tool allows you to swap the unique person ID numbers assigned to each person when multiple people are tracked in a trial. Click and drag an ID and drop it onto another ID to swap those two ID numbers. This change to the IDs is immediately visible in the 2D and 3D views. You can also right-click on an ID to display a context menu with options for removing that tracked person or making it person 0.

Theia3D Batch is a companion application that allows you to process multiple movement trials automatically. Once your calibration and movement data are collected the data needs to be organized (each trial in its own subfolder), and calibrations need to be processed and/or assigned. Once that is completed, you can select the parent folder in Theia3D Batch, configure your settings preferences, and begin processing. This tool is ideal for analyzing large datasets without manual intervention. However, it is recommended to first verify your calibrations and review a few individual trials in Theia3D to ensure quality before using batch processing.

‍

The Setup preferences pane contains options and parameters for the software setup and startup.

Visual3D Path

Path to Visual3D.exe location. Must be set in order to load data in Visual3D from Theia3D. Use the Visual3D Path button to browse to and select your Visual3D.exe file.

Select GPUs on launch.

Show or hide the startup/GPU selection dialog when Theia3D launches.

‍

Explanation

This error message arises when loading video data. It indicates that the videos files from the selected trial were written using an unsupported video file codec.

Possible Solutions

Theia3D requires video files to be encoded using certain supported video codecs. If the videos you are trying to load were encoded with an unsupported codec, they cannot be loaded by Theia3D. When this error is encountered, the best solution is to re-export the videos in a supported codec, or to convert the existing videos to a supported codec.

For OptiTrack Prime Color cameras, verify that you have selected the correct video export settings, particularly Video Format: MJPEG. If the incorrect video format was selected, the video files may not be written using a video codec supported by Theia3D and will need to exported from Motive again, in the proper format.

‍The Toggle Views tool can be used to turn on and off individual camera views for the currently loaded trial. When turned off, camera views are greyed out in the 2D viewer area, and are not used when analyzing the trial.

Toggle Views adjustments are included when saving preferences files.

Settings Menu

Shortcut: Ctrl + ,

‍

The settings available at the top of the preferences pane enable the user to load, save, and save default preferences.

The Video Enhancement tool allows you to improve video quality before processing data, which can be especially useful for calibration under challenging lighting conditions. Changes made using this tool are previewed for the loaded 2D views, and any enhancements are applied before running the 2D inference, lens calibration, or chessboard calibration processes. Therefore, what you see in the 2D views is how the video data will be processed.

All Views

Apply the selected enhancements to all camera views. Alternatively, select an individual camera from the dropdown menu to apply individual camera enhancements.

One of the benefits offered by Theia3D markerless motion capture is automated tracking, which allows it to analyze large datasets without human intervention or supervision. This is achieved using the Theia3D Batch companion application to Theia3D, which allows a list of trials to be curated and batch analyzed sequentially. While batch processing is efficient and does not require supervision, we always recommend that you manually examine and check the quality of your markerless data and calibrations using Theia3D, before setting up a batch analysis. This can prevent poor calibrations or other issues with the data from going unnoticed until after the batch analysis has been completed.

There are a variety of tools available within Theia3D Batch to organize and analyze multiple trials. The data to be processed must be in a single directory that can contain as many levels of subdirectories as desired to organize the data. However, each branch of the directory must end with a folder containing the data for a single trial. This must be a folder of video data as described in that also contains the calibration file for the trial. It is critical that the IDs of the cameras in the calibration file are the same as the IDs of the video file subfolders. When using this tool, the first step is to make sure your video data are organized and the calibration has been added to the files. Once these steps are complete, your data is prepared for batch processing. Proceed to and for details on setting up and executing a batch analysis.

The preferences file associated with each trial determines the prefences used when that trial is analyzed. This allows you to customize the preferences used across different trials within a single batch analysis, providing greater control over the analysis process. If there are not any preferences PXT files assigned to the trials within your selected batch analysis root folder, the Preferences column will indicate that the default preferences will be used for your trials.

Right-clicking on the preferences column displays a context menu for the preferences, with the following options:

Edit

Opens the Settings dialog to change the preferences for the current trial.

The Display Video Metadata tool can be used to easily review relevant metadata for the videos currently loaded, including:

Camera Id

Camera ID dropdown, allowing separate cameras to be selected and reviewed.

This section describes the basic camera system requirements for recording data to be used with Theia3D. See for additional recommendations and principles to follow when recording video data for Theia3D.

Number of Cameras

Theia3D requires a minimum of six cameras for tracking; however, we recommend a minimum of eight cameras for most capture volumes. The number of cameras required will increase with movement complexity, capture volume size, capture volume complexity, and the number of people to be tracked. The field of view and joint visibility requirements outlined below are the best guides when determining the number of cameras required for a specific capture volume.

Movement data for tracked people from a processed trial can also be exported to .json file format for more open-ended use in scripting environments and as an easily readable file output format. Use the Save Skeleton Poses button to save JSON output files, or navigate to Save Skeleton Poses under the .

Output JSON files contain:

• Theia3D version number, engines, and kinematic model information

• Trial frame rate

• Preferences used during process, such as start/end frame, max people, smoothing frequency, etc.

‍

Explanation

This error message arises when loading video data. It indicates that the folder selected when browsing to load video data contains data that is not formatted as required by Theia3D (See ). The video data may all be located within a single folder (i.e. all video files together in one folder), instead of each video file being nested within its own folder with a matching camera ID folder name.

The Adjust Calibration tool can be used to modify the position and orientation of the global coordinate system (GCS) after completing a chessboard or object calibration, or after loading an existing calibration file. The GCS projection on each 2D camera view and within the 3D View is updated live as the Position and Angle sliders are used to modify these parameters of the GCS localization.

This tool can be used to reposition the GCS to a more desirable position or orientation if the desired origin frame could not be used from the chessboard calibration trial, for any other methodological reason that requires the GCS in a specific position.

Position (X, Y, Z)

X, Y, Z sliders can be used to change the position of the GCS relative to its original position, measured in mm.

After running a batch analysis, the output files that were selected in the batch interface will be available. All output data formats are saved in separate folders alongside the batch directory folder containing the raw input data. The possible output data folders are as follows, but may not all be present if they were not selected for output:

• [data folder name]_c3d

• [data folder name]_fbx

• [data folder name]_json

‍

Explanation

This error message arises when loading a calibration file. It indicates that the selected calibration file does not contain the required calibration parameters to calibrate the camera system of the loaded videos. The calibration file may be missing intrinsic parameters, extrinsic parameters, or a combination of both, which prevent it from calibrating the camera system for the video data.

Explanation

This error message arises when attempting to run Theia3D or Theia3D Batch software prior to activating the license on the PC, and includes both of the dialog windows shown above. This usually occurs immediately after installing Theia software on a new PC.

The rendering preferences pane contains advanced options and parameters for how the data should be played back and rendered.

Playback

Playback rate during continuous playback. A value of 1 will show every frame, 2 will show every other frame, etc. (Default: 1)

‍

Explanation

This error message arises when loading video data. It indicates that the folder selected when browsing for video data to load contains fewer than six camera views.

Check Synchronization

Synchronization of the video images can be checked after the trial has been analyzed fully. For each view, the 2D representation of the skeleton in that view is compared to the 3D skeleton computed from all views. If the alignment for a given view can be improved by shifting its video sequence forward or backward in time, this frame offset is reported in the dialog. You can then choose to correct the synchronization by applying these offsets to the out-of-sync views and clearing the entire analysis.

Explanation

This error message arises when processing a lens calibration trial, and Theia fails to detect sufficient chessboard instances to properly calibrate the lenses.

Pose files can also be saved as FBX file formats for use in animation and other software tools that utilize this file format. Use the Save Skeleton Poses button to save skeleton pose .fbx files, or navigate to Save Skeleton Poses under the .

FBX file outputs can be saved with a variety of different segment coordinate system conventions for your convenience.

‍

Explanation

This error message arises when the chessboard calibration trial does not contain sufficient frames of overlapping visibility of the chessboard in 3 or cameras throughout the trial. This prevents the camera system from being properly calibrated, as there is insufficient information to calculate the position and orientation of all cameras in the system in 3D space.

‍

Explanation

This error message arises when loading video data. It indicates that the videos from the selected trial are of unequal lengths.

Explanation

This error message arises when abnormally high tracking errors are detected for one or more camera views, as reported in the error dialog. As indicated by the dialog, this usually indicates a problem with the camera calibration and may be possible to resolve using the tool.

Explanation

This error message arises when attempting to load a Theia workspace which was saved using Theia3D v2023 or earlier, which are not directly supported. These workspaces cannot be loaded due to incompatibilities in the models from early versions of Theia3D relative to newer versions.

Explanation

This error message arises when loading video data. It indicates that the videos files from the selected trial were written using an unsupported video chroma format for codec h264. Supported formats are 8, 10, or 12 bit YUV 4:2:0.

‍

Explanation

This error message arises when the chessboard calibration algorithm is unable to calculate the position and orientation of the cameras relative to the chessboard. More than one chessboard may have been detected in the calibration trial videos, which can be caused by extra chessboards present in the capture volume or mirrors positioned around the capture volume.

1. 2D Views

The videos from each camera view are shown here and the 3D scene is projected onto each view. When all views cannot fit in the available area, arrows are shown and can be used to move through the available views. Double-click on a view to maximize it. When maximized, the mouse is used to zoom (scroll) and pan (right-click and drag) the view, and clicking on the arrows moves between views. Double-click the maximized view to restore it and show all views. Use Alt + right-click to inspect a view through a magnifying class - the level of magnification is controlled by the mouse wheel and reset with the ‘R’ key.

2. 3D View

The 3D scene is rendered here. The mouse is used to rotate (left-click and drag), pan (right-click and drag), zoom (scroll), and inspect (Alt + right-click) the scene. The 3D scene contains the global coordinate system, cameras, and skeletons of each identified and tracked person. The origin of the global coordinate system is a white sphere, the x-axis is a red arrow, the y-axis is a green arrow, and the z-axis is a blue arrow. The scene enviroment, ground, and lighting can be modified from the menu accessed using the button at the top left of the 3D view.

3. Quick Tools

The quick tools are shortcuts to the most common Theia3D commands. The entire processing pipeline can be executed using these shortcuts.

Most options can be toggled on and off using the quick tool display buttons.

4. Menu

The menu is divided into eight sections.

• - Commands to load and save data.

• - Commands to process data.

• - Options to control what is displayed in the video overlays and 3D scene.

• - Options to display joint angle results or monitor the progress of a batch analysis.

5. Playback Controller

The video data can be played and paused using the play button on the left, or you can scroll through the trial using the timeline slider. The number to the right of the timeline indicates the current frame number, and the playback dropdown allows you to adjust the steps between frames that are displayed. When set to one, every frame is displayed, when set to two, every other frame is displayed, and so on.

‍6. Status Bar

Below the playback timeline controller is a status bar which indicates the current status and any ongoing processes.

Clear Workspace

Shortcut: Ctrl + F4

Clear all loaded data and analysis results.

Load Video Data

Shortcut: Ctrl + O

Load video files (MP4 or AVI) for analysis. To load the videos, browse to and select the folder containing the videos. The structure of this folder must conform to the format described in .

Load Calibration File

Shortcut: Ctrl + Shift + O

Load the camera calibrations for analysis. To load the calibrations, browse to and select the file containing the calibrations for all of the cameras. The calibration file must conform to the format described in .

Load Workspace

Open a previously saved workspace. To load the workspace, browse to and select the directory containing the workspace files (AVI, T3D, P3D).

Save Workspace

Shortcut: Ctrl + S

Save the videos, camera calibration, and analysis results in the current workspace to a directory in a format that can be opened by Theia3D. Select the frames to include in the workspace and press the Save button. Browse to the desired save location and enter the name of the directory that will be created to store the workspace files. To overwrite the current workspace, use the dialog to select a .t3d file in the existing workspace folder. Note that this only overwites the results (T3D and P3D files) and leaves the videos untouched. In this case, all frames in the workspace must be saved. It is not possible to save to an existing workspace folder other than the currently open one.

‍

Save Calibration

Save the current calibration. To save the calibration browse to the desired save location and enter the desired filename.

Save Model

Save the scaled 3D model. To save the scaled model browse to the desired save location and enter the desired filename. Note that if multiple people are tracked a separate model will be saved for each of them. The files are automatically named as filename_personID.tmb, where personid is automatically assigned.

Save Skeleton Poses

Shortcut: Ctrl + Shift + S, 1

Save the 3D pose of the tracked people. Select the person (or all people) and the frame range (or select the option to use the analysis frames) to save in the file. Files can be saved in C3D, FBX, or JSON format. The data included in the files is described in.

For C3D files there is an option to Save multi-subject C3D. If this option is selected, two C3D files will be created - one with the unfiltered pose and one with the filtered pose. Both files will contain all tracked subjects. If the multi-subject option is not selected, separate files will be created for each tracked subject.

‍

For FBX files there is an option to select the exported coordinate system. Preset options are available for common software programs or custom coordinate system can be defined. Saving an FBX file requires the to be used when solving the pose.

‍

For JSON files the user can select which tracked individuals and which frames of the trial to include in the output file. The JSON output files are easily readable in a text editor or used as input for a custom analysis script.

‍

With the desired options selected, press the Save button, browse to the save location and enter the filename. The person ID and filtered tag are appended to the filename as needed.

Save CMZ (Open in Visual3D)

Opens the pose data for all tracked people in Visual3D.

Save Video Overlay

Shortcut: Ctrl + Shift + S, 2

Save each of the 2D view video overlays as an AVI file. The videos saved are identical to the overlays shown in Theia3D; therefore, you should modify the options that affect the 2D visualizations in the and before saving video overlays. Select the desired frame rate and the frame range (or select the option to use the analysis frames) of the videos and click the Save button. Browse to the desired save location and enter the desired filename. One video file will be saved for each overlay and the files are automatically named as filename_cameraID.avi.

Save 3D View

Save the 3D scene as an AVI video file. The video saved is identical to the 3D view shown in the 3D Viewer; therefore you should modify the options that affect the 3D visualizations in the and before saving before saving 3D View videos. Select the desired frame rate and the frame range of the videos and press the Save button. Browse to the desired save location and enter the desired filename.

Assign Calibration Files

The assign calibration file tool is used to copy and nest an extrinsic calibration file within individual trial folders, one level above the video files. This is best practice for storing calibration files with their associated movement trials, and is a requirement for running batch analyses.

Path to Calibration File

Use the Browse button to navigate to and select your extrinsic calibration .txt file.

Plus‍

Use the plus button to add trials to the Trials list by selecting a folder containing movement trials. All trials within the selected folder will be added to the Trials list, regardless of their depth within the selected folder.

‍

Minus‍

Use the minus button to remove currently selected trials from the Trials list. Use shift+click or ctrl+click to select groups or individual trials to remove from the Trials list.

Just Add to Selection

Assign the calibration file to only the currently selected files within the Trials list.

Cancel

Close the Assign Calibration Files tool without assigning any calibration files.

Assign

Assign the calibration file to the trials in the Trials list or only the currently selected trials if Just Add to Selection is selected.

Help

Opens the online documentation for the software.

Submit Support Ticket

Opens a browser window directly to our .

About

The About dialog shows the following information:

• Software version number

• Release date

• License key

• License counts

‍

There are currently three available models in Theia3D:

1. Default Model

   • Consists of two kinematic chains: lower body (pelvis and legs) and upper body (torso, arms, and head).

   • No abdomen or neck segments.

   • Shoulder joints and head segment are allowed 6 degrees of freedom.

2. 1. Consists of one, whole-body kinematic chain.

   2. Abdomen and neck segments included.

   3. Shoulder joints are allowed 6 degrees of freedom.

3. • Consists of five kinematic chains: lower body (pelvis and legs), torso, left arm, right arm, and head.

   • No abdomen or neck segments.

   • Shoulder joints and head segment are allowed 6 degrees of freedom.

Pose

Model pose can be exported to .c3d, .fbx, and .json files.

The .c3d files contain the 4x4 pose matrices for each model segment and the local coordinates of the anatomical landmarks of the distal segments of the model (feet, hands, head). These files can be processed using Visual3D.

The .fbx files contain the hierarchical skeleton, pose, and bone meshes of the animation model. The first frame of the file contains the model in a “T-Pose”. The skeleton must be solved using the Full Body Model in order to save pose files in FBX format.

The .json files contain information about how the trial was processed (Theia3D version, model, preferences, etc.) and the 4x4 pose matrices for each body segment for every frame of the trial.

Inertial Properties

With each model, inertial properties are estimated using established regression equations similar to Dempster’s, which provide values for segment masses, centers of mass, and moments of inertia based on subject-specific anthropometric estimates and validated population data. Theia3D combines deep learning with inverse kinematics to estimate 3D body movement, using calculated segment positions, orientations to support further biomechanical analysis.

Coordinate System

The 4x4 pose matrices for each body segment convey the position and orientation of the segment’s local coordinate system, which are visualized with the x-axis as red, y-axis as green, and z-axis as blue. Local coordinate system origins are visualized as white spheres

Help

Opens the online documentation for the software.

Submit Support Ticket

Opens a browser window directly to our .

About

The About dialog shows the following information:

• Software version number

• Release date

• Contact information

Pose (C3D) files can be saved and used to perform post-processing analysis steps in Visual3D software. Use the Save Skeleton Poses button to save skeleton pose C3D files, or navigate to Save Skeleton Poses under the File Menu. Saved pose C3D files can be opened in Visual3D, and a subject-specific model will automatically be applied to the Theia3D data in Visual3D without requiring the model to be defined.

The movement of each tracked individual is conveyed within the C3D file using ROTATION signals, which are 4x4 pose (position and orientation) matrices for each segment’s local segment coordinate system, for every frame of data. A description of the ROTATION data type is available from Visual3D documentation. Raw tracked landmarks are not included as signals in any output files from Theia3D.

A workspace folder contains the video and data files of a saved workspace. The video files are named according to their unique camera ID. The .t3d and .p3d files contain the analysis results. It is important that the contents of the workspace folder are not modified, including moving files in or out of the folder.

Theia3D workspaces can be used to save data at various stages of analysis, but are most useful for saving analyzed data that will be reviewed later. When a fully analyzed movement trial is saved as a Theia3D workspace, the saved data includes the 2D videos, calibration file, and all data associated with the analysis. Therefore, when a saved workspace is loaded, you can immediately review the analyzed data in the 2D and 3D viewers.

To open a Theia3D workspace while Theia3D is open, use File > Load Workspace and select the folder containing the workspace files.

Top open a Theia3D workspace while Theia3D is closed, you can double-click on the results P3D file.

1. Menu

1. Menu

The File Menu is divided into 5 Sections

The Help Section is divided into 3 Sections

2. Data Path

Click the file folder to search for the folder with data requried to be processed in the Theia3D Batch application.

3. Settings

Settings can be configured to select the specific outputs from the batch process for each trial. More about the settings can be sound in the

4. Trials

The Trials list is populated with all valid trials within the selected batch analysis root folder, once a folder has been selected. Within this section, trials can be searched and details of each indvidual trial can be populated.

5. Status Bar

Status bar indicates the progress of the batch process in terms of which folder is currently processed out of the total number to be processed along with with a progress bar of the progress of the current trial.

What landmarks are used?

Theia3D tracks 124 unique keypoints across the entire body, providing sufficient points on each segment for all segments to be fully observable in 3D space.

We are unable to provide the specific keypoints that are tracked, however many are common anatomical landmarks that are typically used to define rigid body models of body segments.

Can I build custom segments?

Theia3D uses 17 rigid segments to track the movement of the body, as described in the page. The 6-DOF pose of all segments are exported as 4x4 pose matrices within the selected (e.g. C3D, FBX, JSON), along with a select few landmarks for relevant segments (e.g. left and right heels). These segments are defined to have biomechanical and anatomical relevance, and will provide sufficient data for most users' applications.

When using post-processing software such as Visual3D, it is possible to build custom segments using approaches such as defining virtual landmarks relative to the existing segment pose matrices. However, since the raw tracked landmarks are not included as output signals, these custom segments are typically limited to having a different static orientation but tracking the identical movement as the existing output segments using the 4x4 pose matrices.

What is the difference between pelvis_4X4 and pelvis_shifted_4X4 signals?

For all versions of Theia3D, these pose matrix signals express the position and orientation of the pelvis segment. However, there are some very slight differences that may be present, depending on the version of Theia3D used:

For Theia3D v2023 and earlier, these two sets of signals are nearly identical, with the only difference being that the pelvis_shifted_4x4 signal has its origin located at the height of the pelvis plane, whereas the pelvis_4x4 signal has its origin located at the midpoint of the hip joint centers. Otherwise, these signals are identical, and if you were to calculate a vector between their origins you would find it to have a constant length and constant direction within the pelvis reference frame. These separate signals were used to define the proximal (pelvis_shifted_4x4) and distal (pelvis_4x4) ends of the pelvis segment in order to construct the pelvis segment in Visual3D and other similar post-processing software.

For Theia3D Apollo v2024 and later, these signals are identical and are a holdover from previous versions. The pelvis_4x4 origin is now located at the pelvis plane (i.e. where the pelvis_shifted_4x4 signal was previously), and is used to define the proximal end of the pelvis, while the distal end is defined as the midpoint between the hip joint centers which are coincident with the thigh segments origins. Assuming an up-to-date version of Visual3D is used, the pelvis segment will be constructed accordingly.

What is the abdomen segment?

The abdomen segment in the full-body model does not have its own tracked points, it’s scaled and positioned based on the pelvis and thorax through inverse kinematics. It mainly exists to bridge those two segments and provide a continuous chain for FBX/animation export.

Functionally:

• It allows some independent x-y tilt (flexion/extension, ab/adduction) relative to the pelvis.

• It does not capture z-axis rotation (internal/external rotation).

• Its pose should not be interpreted as a true lower-back segment; it’s more of a connector than a biomechanically distinct region.

Video Data

‍

The video data for a single trial must be contained in its own folder, and each video file must be in its own subfolder. The name of each subfolder must be the ID of the corresponding camera. There are no requirements on the structure of the names of the video files, but the names must be unique and videos must be .avi or .mp4 format. For example, the figure above shows video data for a walking trial collected using four cameras with IDs 21375, 21379, 21380, and 21381.

Organizing your videos in this format can be achieved using the tool.

It is also best practice to keep the associated extrinsic calibration file nested within the trial folder and beside the camera ID folders. This can be achieved using the tool.

‍

Explanation

This error message arises when loading a calibration file. It indicates that the selected calibration contains camera IDs that do not match those for the loaded videos.

Possible Solutions

1. Check that the correct calibration file was selected.

The calibration file may be incorrect for the loaded videos. Confirm that you selected the correct calibration file, which should have been generated using a calibration trial from the same data collection session as the loaded movement trial videos.

1. Check the camera IDs within the calibration file and the loaded video file names.

If is performed more than once to organize the video data and different delimiters or different parts of the file names were used to assign ‘Cam ID’, the camera IDs may not match between the calibration file and the loaded videos. If this is the case, the camera ID folder names for the calibration trial and movement trials should be changed to be consistent, and the camera IDs within the calibration file should be modified to match the camera ID folder names.

1. Check that the number of cameras in the calibration file matches the number of videos loaded.

The number of camera views loaded in Theia3D and the number of cameras contained within the calibration file may not match. If the number of camera views loaded in Theia3D does not match the number of cameras listed in the calibration file, double check that you have selected the correct calibration file, that you are not missing any camera views from your loaded movement trial, and that the calibration file has not been modified.

Sony RX0 II

Each Sony RX0 II camera includes the following:

• Camera (1)

• Rechargeable Battery Pack (1)

• Micro USB Cable (1)

• AC Adaptor (1)

• Wrist Strap (1)

• Memory Card Protector (1)

• Startup Guide (1)

• Reference Guide (1)

microSD Card

• A Video Speed Class V60 microSD card that exceeds the Class 10 requirement of the cameras and provides increased download speeds.

Sony Camera Control Box

Each Sony Camera Control Box includes the following:

• Camera Control Box (1)

• Multi Terminal Connecting Cable - Short (1)

• Multi Terminal Connecting Cable - Long (1)

• Micro USB Cable (1)

PoE Network Switch

• 16-Port Gigabit Switch (with power over ethernet)

• The switch provides 1 Gbps connections for each of the cameras and a 10 Gbps connection to the collection computer. When connected to a 10 Gbps NIC card in the collection computer, this provides increased download speeds from the cameras.

• Power Cord

Cables

• 6ft Cat6 Ethernet Cord (1)

• 50ft Cat6 Ethernet Cord (1 per camera)

• Gigabit PoE Splitter (1 per camera)

Additional Hardware

• Male-to-Male 1/4” Thumb Screw (1 per camera)

• Rubber Washer (1 per camera)

‍

Explanation

This error arises when Theia3D is not able to adequately track the people identified within the 2D views throughout 3D space. There are two primary causes of this error:

1. An incorrect calibration file was loaded.

If an incorrect calibration file was loaded, meaning that the camera system is not properly calibrated, then it will not be possible for the people identified within the 2D videos to be tracked throughout 3D space.

1. The Analysis Bounding Box was used with dimensions (length, width, height) of 0.

If the Restrict skeletons to bounding box option was selected but no dimensions were provided for the bounding box, the people visible in the 2D camera views cannot be tracked in 3D space and the Track people not complete error message will appear.

Possible Solutions

1. Check the Restrict skeletons to bounding box option.

‍If you intended to use the Restrict skeletons to bounding box option, either provide the dimensions of your desired bounding box or use the Use Camera Locations option to define the bounding box. If you did not intend to use the Restrict skeletons to bounding box option, deselect it. After making either modification, you will need to Run Analysis (without 2D) to update the person tracking and modelling results.

1. Check that the correct calibration was loaded.

If an incorrect calibration file was loaded, the camera system may not be calibrated properly leading to improper 3D reconstructions from the 2D data. Review the position and orientation of the global coordinate system in each 2D camera view - they should all show the global coordinate system at the same position and orientation. If the global coordinate system is out of place, it is likely that an incorrect calibration was loaded. Locate and load the correct calibration file, and Run Analysis again.

1. Check that there are people sufficiently visible to be tracked.

If there are no people or insufficient views of any people captured in the videos, nobody will be tracked. Be sure that your cameras are set up to sufficiently capture any participants with 3 or more cameras at all times, and record new movement trials.

Explanation

This error message arises when processing a lens calibration trial, and the detected chessboard patterns fail to coverage a sufficient proportion of the camera view. Theia requires the chessboard pattern detections cover at least 60% of the camera view in order to produce lens calibration parameters for that view. Review the Lens Calibration Metrics section for more detail.

Possible Solutions

Check that the correct lens calibration trial videos were loaded. If an trial is loaded and processed using the Lens Calibration tool, this error can occur.

Check that the chessboard pattern is sufficiently visible throughout the recording. If the videos are too dark or too bright, try using the to improve the visibility of the chessboard pattern. If the chessboard is too close to the camera during the trial such that pattern is too large and is never fully visible, a new lens calibration trial will need to be recorded.

Explanation

This error message arises when attempting to use the Organize Videos Tool to organize video files as required by Theia3D. Theia3D requires video data to be MP4 or AVI file formats, so if the directory selected for organizing contains video files of a different format (e.g. MKV, MOV, etc.) Theia3D will be unable to organize or load these videos.

Possible Solutions

To ensure your video data can be organized, loaded, and processed by Theia3D, they should be in .mp4 or .avi file formats. If possible, re-export the original videos to the correct file format, or convert the existing video files

Settings

The Settings section provides widgets to set up the current batch analysis. The data to be processed must be organized in this single root folder that can contain as many levels of subdirectories as desired to organize the data. However, each branch of the directory must end with a folder containing the data for a single trial. This must be a folder of video data as described in that also contains the calibration files for the cameras. It is critical that the IDs of the cameras in the calibration file are the same as the IDs of the video file subfolders. The Settings section enables the user to input the batch analysis root directory and the output format that will be generated by the batch analysis:

The two prerequisites for batch processing of video data is that the videos are organized in the required nested folder structure as described in and that every trial has an assigned calibration file. Therefore, the and tools are included for use in Theia3D Batch.

‍

The Open into options allow you to easily open the currently selected batch processing directory, in either Windows Explorer or the command line.

The following video provides step-by-step instructions for setting up and using your cameras to collect data for use with Theia3D. This includes changing camera settings, synchronizing the cameras, recording data, and downloading videos from the cameras through the web interface. This tutorial demonstrates the basic functionality and recommended settings of the Sony camera control interface. Refer to the Sony documentation for information about other settings and more advanced functionality.

Video overlay (AVI) files can be saved after analysis of a movement trial is completed, and can be saved from within Theia3D or Theia3D Batch. These videos show any of the tracking produced by Theia overlaid on the original video files, which can be useful for Quality Assurance or demonstration purposes.

To save video overlays from Theia3D, use the Save Video Overlay button in the sidebar or under the File dropdown menu.

To save video overlays from Theia3D Batch, select Save Videos and any desired display options and camera views to be saved, prior to running the analysis.

Object calibration is performed in order to determine the position and orientation of all cameras in the system in 3D space using a recorded object calibration trials. See Recording Extrinsic Object Calibrations for detailed instructions for recording object calibration trials.

/Load Videos

Load video files (MP4 or AVI) containing the extrinsic calibration object. To load the videos, browse to and select the folder containing the videos. The structure of this folder must conform to the format described in.

Camera Type

Detects and indicates the camera manufacturer based on the videos loaded. This allows the automatic application of default Sony RX0 II camera intrinsic parameters when the calibration videos were recorded using this camera system (see below).

Load Default RX0 II Intrinsics

Load default lens calibration parameters for Sony RX0 II cameras and apply them to all cameras. This can be used to avoid performing a lens calibration when using the RX0 II cameras. Loading default intrinsics overwrites any existing lens calibration parameters. Note: These parameters are only valid for Sony RX0 II cameras with standard lenses. For any other cameras or lenses these default parameters are invalid.

Load Object

Load a TXT file defining the calibration object. Each line of the file contains comma-separated x-y-z coordinates of one of the key points on the object in mm. To load the object, browse to and select the object file. Once loaded, the object points will be displayed in the table.

Save Object

Save the current object definition. To save the object, browse to the desired save location and enter the desired filename.

Add

Add a calibration object point definition (x, y, z coordinate values).

Remove

Add a calibration object point definition (x, y, z coordinate values).

Auto-Step

If checked, the selected object point will advance each time a point is identified in one of the 2D views.

Cam ID

The ID of the camera corresponding to the current 2D view. Use the drop-down to select a different 2D view. Maximizing a view will automatically select it.

Reset Point

Remove the currently selected point from the current view.

Reset Camera

Remove all points from the current view.

Reset All Cameras

Remove all points from all views.

Calibrate Cameras

Perform the extrinsic calibration for all of the cameras. After performing the calibration, the position of any camera points that were not identified before the calibration are calculated and drawn in the 2D views as an ‘*’.

As with the method, the results dialog window provides the options to Save, Save & Assign, or to acknowledge and close the results dialog window using Ok.

Adjust Origin

When using the checkerboard extrinsic calibration, sometimes the origin may appear at an undefined location because the board was not recognized in the origin frame. This feature allows you to adjust the reference frame by clicking on an object (typically four squares on the checkerboard). Since the frame is just being adjusted (not computed), a calibration needs to be loaded to use this feature, and the relative pose of the cameras will be unchanged.

Organize Videos

The Organize Videos tool converts a folder of videos into a structure that can be used by Theia3D. This functionality is useful when collecting large data sets.

Basic

The Basic tab is the best option when organizing videos that have user-defined file names, such as those collected using OptiTrack, Qualisys, or Vicon camera systems. To use this tool, it is recommended to include relevant details about the trial as part of the file name, consistently separated by one delimiter from the following options:

• Space (' ')

• Hyphen ('-')

• Underscore ('_')

• Period ('.')

With video files named in this way, the dropdown options below can be assigned to each delimited portion of the filename, resulting in an intuitive file structure that maintains this information for each trial:

• Cam ID: (Required) The ID of the camera corresponding to the video file. This must be the same ID as in the calibration file.

• Subject: (Recommended) Unique subject identifier (e.g. sub001).

• Action: (Recommended) The action the subject is performing (e.g. walk).

• Trial: (Recommended) The trial number (e.g. 001 or walk01).

To format video data using this tool, follow the steps below:

1. Select the delimiter used in the file name.

2. Use the Browse button to select the folder containing the videos. Once the directory has been selected, the dialog will parse the name of a sample file. Note that the delimiter can be changed after the folder is selected and the sample file name will be re-parsed.

3. Use the drop down menus to select the reserved term that each part of the filename corresponds to.

4. Click Execute to organize the data into the required structure. The hierarchy of the created directory is Subject > Action > Trial > Cam ID. Cam ID is the only reserved term that

For example, consider a folder with the following video names:

• S0001_Walk_001_8_21375.avi

• S0001_Walk_001_8_21376.avi

• S0002_Walk_001_8_21375.avi

To format these videos, the reserved terms selected are: Subject, Action, Trial, N/A, and Cam ID. The data structure after formatting is:

Top Level Only

When enabled, this option sorts only the video files located directly within the current folder. Any subfolders and the files they contain are ignored during the sorting process. This ensures that only the top-level contents are affected, and can be useful when adding new, unorganized data into a folder with existing, organized data.

Advanced

The Advanced tab is the best option when organizing videos that do not have user-defined file names, such as those collected using Sony RX0 II camera systems. To use this feature, the names of the video files must end with camid_trialid, where camid is the ID of the camera that recorded the video and trialid is a unique identifier for the trial.

‍When exporting videos using the Sony RX0 II camera system browser interface, make sure that the Name of Imported File options are selected as follows:

• Add the Camera Label (prefix)

• Do not add the Shooting Date/Time

To format video data using this tool, follow the steps below:

1. Use the Browse button to select the folder containing the videos. The trials table is then created with one row for each trial. The File column of each rorow is filled with the trialid for that trial.

2. Use the Load Subject and Action Assignments button to load the subject and action IDs for each of the trials from a csv file. Each row of the csv file must contain the trialid, subjectid, and actionid of a single trial, in that order, separated by commas. For example, the csv file used to assign the subject and action IDs in the above figure was:

1. Press the Execute button to format the videos in the required folder structure.

Alternatively, Subject and Action details can be manually assigned to the trials table without using a CSV file:

1. Double click any cell to type in the Subject or Action ID.

2. Click Save Subject and Action Assignments to save the filled table as a CSV.

3. Press Execute to format the videos in the required folder structure.

The following video provides step-by-step instructions for setting up your Sony RX0 II camera system for the first time. This includes configuring the cameras and control boxes, setting up the multi-camera system, and controlling the system through a web browser.

Camera

1. Remove the camera, battery, memory card protector, and microSD card from the packaging.

Format Sony Multicam

The Format Sony Multicam tool is alternative approach to organizing video data recorded using a Sony RX0 II camera system, rather than using the tool. This tool can be used to quickly organize a dataset based on the video timecodes rather than the C000# trial IDs appended upon downloading, which is particularly useful for cases where the C000# trial IDs are misaligned between cameras for the same trial recording.

To use this tool, the raw video data must be recorded using the Sony RX0 II camera system, and should be downloaded using the options: Add the Camera Label (prefix) and Do not add the Shooting Date/Time. The raw video files must be accompanied by a data specific CSV file, which can have any name, however it must be the only .csv file within the directory alongside the video files. The data spec .csv file must have a header row of: Trial,SubjectId,Type,Action,CalibrationId. Each subsequent row should contain the relevant information under each header column that corresponds to the trial.

Chessboard calibration is performed in order to determine the position and orientation of all cameras in the system in 3D space using a recorded chessboard calibration trials. See Recording Extrinsic Chessboard Calibrations for detailed instructions for recording chessboard calibration trials. More information about extrinsic calibration can be found here.

Load Videos

Load video files (.mp4 or .avi) containing the chessboard calibration trial. To load the videos, browse to and select the folder containing the videos. The structure of this folder must conform to the format described in .

Frame Grab Step

The step between frames searched for chessboards. Smaller values take longer to process, but may provide slightly improved calibration results compared to larger values.

Camera Type

Detects and indicates the camera manufacturer based on the videos loaded. This allows the automatic application of default Sony RX0 II camera intrinsic parameters when the calibration videos were recorded using this camera system (see below).

Use Default RX0 II Intrinsics

Automatically selected after loading videos recorded using a Sony RX0 II camera system. This option should be selected to utilize the built-in intrinsic parameters for the Sony RX0 II cameras. Note: These parameters are only valid for Sony RX0 II cameras with standard lenses. For any other cameras or lenses these default parameters are invalid.

‍Load Custom Intrinsics

Loads intrinsic lens parameters for the camera views loaded.

Origin Frame

Frame to use to set the global coordinate system of the capture volume. If the chessboard is not detected in at least three views in this frame, the closest frame in which it can be detected will be used.

Min 2 Cams for Origin Triangulation

Uses two (instead of the default of three) cameras to locate the origin frame. Use this option if the board is difficult to see in the triangulation frame from three cameras.

Normal Axis

Set the coordinate system axis that is defined by the normal axis of the chessboard.

Long Axis

Set the coordinate system axis that is defined by the long axis of the chessboard.

Use Custom Chessboard

Use a custom size chessboard for the calibration. Only select this option if not using the chessboard provided by Theia Markerless.

• Square Size: The width and height of each square on the chessboard. Measured in mm.

• Number of Squares High: The number of inner squares in the vertical direction of the chessboard.

• Number of Squares Wide: The number of inner squares in the horizontal direction of the chessboard.

Calibrate Cameras

Perform the extrinsic calibration for all of the cameras.

Chessboard Calibration Metrics

After the camera calibration is complete, a dialog will appear with the results of the chessboard calibration trial. The result metrics can be interpreted as follows:

Frames

The number of video frames used to calibrate each camera.

RMSE Reprojection

RMSE error of the reprojected 3D chessboard points relative to the detected chessboard points in 2D, for each camera view. Measured in mm. Since this is an error relative to 0 mm for all chessboards regardless of size, a 1 mm RMSE reprojection has the same meaning for all sizes of chessboard.

RMSE Diagonal

RMSE error of the length of the diagonals of the 4 outer corners of the chessboard relative to their known lengths, for all triangulated chessboard detections. Measured in mm. Provides an absolute error measure, not normalized to the size of the chessboard. Therefore, a 1 mm RMSE diagonal for a large board is a smaller percentage error than a 1 mm RMSE diagonal for a small board. Not affected by the number or size of the chessboard squares. Recommend using <1 mm as ‘Excellent’ and <2 mm as ‘Acceptable’; calibrations with RMSE Diagonal above 2 mm are not recommended for use.

RMSE Angle

RMSE error of the angle of the 4 outer corners of the chessboard relative to their known angle of 90 degrees, for all triangulated chessboard detections. Measured in degrees. Since this is an error relative to 90 degrees for all chessboards regardless of size, a 1 degree RMSE angle has the same meaning for all sizes of chessboard. Not affected by the number or size of the chessboard squares.

RMSE Flat

RMSE error of the normal distance from each detected chessboard point in 3D to the flat plane formed by the outer four 3D chessboard points. Measured in mm. Provides an absolute error measure, not normalized to the size of the chessboard. Therefore, a 1 mm RMSE flat for a large board indicates a lesser degree of bend in the chessboard than a 1 mm RMSE flat for a small board.

Origin Triangulation Frame

The video frame used to initialize the position and orientation of the global coordinate system.

After reviewing the calibrations results, the folling options are available:

Save allows the calibration to be saved.

Save & Assign initiates two steps:

1. Opens the Save Calibration window, allowing the calibration to be saved as a .txt file.

2. Opens the Assign Calibration tool, allowing the previously saved calibration .txt file to be immediately assigned to movement trials.

Ok acknowledges and closes the results dialog window.

Chessboard Calibration Review

After the camera calibration is complete, the calibration trial videos can be reviewed for feedback. The following visual cues projected onto the camera views can be useful in determining which portions of the trial contributed to the calibration, and those that did not. This can be useful for optimizing your calibration trial technique. During the chessboard calibration process, areas where the board was waved with good visibility and successfully tracked are reflected in the 3D view with th epoint visualization. This visualization can be used to visually imspect volume coverage across the calibrated capture space.

The Analysis preferences pane contains advanced options and parameters for adjusting how the movement is analyzed, including subject identification and person tracking, model parameters, and 3D reconstruction parameters.

Analysis Frame Range

First and last frame to analyze (inclusive). Modified frame range values are included when preferences are saved. (Default: full trial length)

Image Matte Percentage

Image exclusion border size. Image matte area is displayed as a grey border around the camera views when active. The greyed-out portions of the videos will not be used when running the analysis, allowing poorly calibrated image borders to be ignored. Image matte area is not applied during lens calibration.

Track People Parameters

Max People

Maximum number of people to track. (Default: No max)

Person Tracking Mode

Select the method for determining person tracking priority. Run Analysis (without 2D) must be performed in order to update person tracking.

• Most Visible: Person identification and tracking performed by prioritizing people who are visible in >75% of the total frames from all cameras throughout the entire trial, and ordering them by their distance to the global coordinate system origin.

• Closest to Origin: Person identification and tracking performed by ordering all people by their minimum distance to the global coordinate system origin throughout the entire trial. For example, if a person walks directly over the origin, they will likely be identified as person 0.

• Order of Appearance: Person identification and tracking performed by numbering people in order of appearance within the capture volume.

Remove stale IDs

Remove any person IDs that are missing for 100 consecutive frames or more. This allows a person who is tracked repeatedly within a single trial (e.g. during several passes into and out of the volume) to be tracked and therefore exported separately, for instance as multiple separate c3d files.

Track Rotating People (beta)

Improve tracking of rotating and non-upright people at the expense of speed. (Default: Off)

Solve Skeleton Parameters

Use Saved Model

If not selected, the generic model will be scaled during Solve Skeleton. If selected, a previously saved model can be used by the inverse kinematics algorithm. Use the Use Saved Model button on the next line to select the model to be used. Note that this option is only available when Max People is 1. (Default: Off)

Large Lens Distortion

Uses a larger grid for lens distortion correction. Only use if there is a lot of distortion and loading is slow. (Default: Off)

Default Model (neither Full Body nor Separate Arm and Head model selected)

Uses separated IK chains for the lower body (pelvis and legs) and upper body (torso, arms, and head). No abdomen or neck segments. Shoulder joints and head segment are allowed 6 degrees of freedom. See for a detailed description of this model.

‍Use Full Body Model

Uses a single, whole-body IK chain including pelvis, abdomen, and thorax segments. See for a detailed description of this model. Must be used if saving skeleton poses using FBX format.

‍Use Separate Arm and Head Models

Uses separated IK chains for the lower body (pelvis and legs), torso, arms, and head segments. No abdomen or neck segments. Shoulder joints and head segment are allowed 6 degrees of freedom. See for a detailed description of this model.

Enable 3 DOF Knee

If selected, the model used by the inverse kinematics algorithm will have three degrees of freedom at the knee (flexion/extension, ab/adduction, and internal/external rotation). If not selected, the model will have two degrees of freedom at the knee (flexion/extension and ab/adduction). (Default: Off)

Smoothing Freq. (Hz)

The smoothing frequency used is based on the which provides results equivalent to applying a double-pass Butterworth filter. Theia3D automatically fills gaps and smooths the 3D pose data for all individuals tracked in processed movement trials.

The cutoff frequency of the lowpass filter used to smooth the pose from the inverse kinematics. The default GCVSPL cutoff frequency is 20 Hz, cutoff frequency provides some smoothing and noise reduction while generally being sufficiently high to avoid ‘over-smoothing’ most movements.

General Recommendations:

• For slow movements (e.g. sway, slow walking), a lower cutoff frequency around 6–12 Hz can be used.

• For faster movements (e.g. running or jumping), a higher cutoff frequency around 10–20 Hz may be required.

• Movements with very high segment velocities or accelerations such as baseball pitching or very dynamic changes of direction, cutoff frequencies above 20 Hz may be required.

Show Warning

When applying the filter using the selected smoothing frequency, Theia3D checks whether the applied smoothing frequency is appropriate for the movement and warns the user if they should consider adjusting the frequency. This test can occasionally produce false positive warnings, so these warnings can be disabled if desired.

Analysis Bounding Box

Display/Use

Enables the 3D Analysis Bounding Box for use, which restricts person tracking to within a specified 3D volume. This allows people who are visible but not of interested to be ignored, such as the experimenter or other observers. Run Analysis (without 2D) must be performed in order to update person tracking and skeleton visibility.

Use Camera Locations

The 3D camera locations will be used to establish the edges of the analysis bounding box, rather than the x, y, and z origin positions and length, width, and height dimension definitions.

• Origin X: Position of the center of the bounding box along the global coordinate system X-axis.

• Origin Y: Position of the center of the bounding box along the global coordinate system Y-axis.

• Origin Z: Position of the center of the bounding box along the global coordinate system Z-axis. Minimum value is half the bounding box height.

Camera Calibration File

‍

The camera calibration file contains the calibrations for all of the cameras using a structure similar to XML. An example calibration file is provided with the sample data. Key elements and attributes of the calibration file are outlined here. Note that Qualisys calibrations exported from QTM as TXT files and Vicon calibrations (XCP) can also be used.

‍<calibration>

Top level element with no attributes.

Grouping element that holds <camera> child elements. It has no attributes.

<results>

Element that holds the calibration results metrics, as described in .

<camera>

Element that holds the calibration information for a single camera. Required attributes are: active, serial, and viewrotation. Required child elements are: <transform> and <intrinsic>.

• active: 1 if the camera is used, 0 if unused.

• serial: The camera ID. Must be unique to the camera and match the camera ID used to name the video subfolders.

• viewrotation: Rotation of the camera. 0 if upright, 180 if upside down, 90 or 270 if sideways. This is calculated from the rotation matrix of the calibration.

<transform>

Translation and rotation components of the transformation from global to camera coordinates (p→cam=Rp→global+t→). Required attributes are x, y, z and r11 through r33.

• x, y, z: Elements of the translation component (t→) of the transformation. Expressed in mm.

• rij: Elements of row i and column j of the rotation component (R) of the transformation.

<intrinsic>

Intrinsic camera parameters. Required attributes are: focallength, sensorMinU, sensorMaxU, sensorMinV, sensorMaxV, focalLengthU, focalLengthV, centerPointU, centerPointV, skew, radialDistortion1-3, and tengentialDistortion1-2.

• focallength: Focal length of the lens in mm.

• sensorMinU: Minimum u coordinate of the sensor in pixels.

• sensorMaxU: Maximum u coordinate of the sensor in pixels.

• sensorMinV: Minimum v coordinate of the sensor in pixels.

Intrinsic vs. Extrinsic Calibration

Accurate 3D reconstruction in markerless motion capture requires both intrinsic and extrinsic calibration of all cameras in the system. These two types of calibration serve different but complementary purposes. The same calibration chessboard is used for both the Intrinsic and Extrinsic calibration.

Intrinsic Calibration (Lens Calibration)

Intrinsic calibration characterizes the internal properties of each camera, including:

• Focal length

• Sensor center point

• Skew

• Lens distortion

This calibration corrects image distortion and ensures that pixel locations correspond accurately to 3D rays extending from the camera into space. Intrinsic calibration is performed for all cameras, and the resulting parameters are unique to each camera. These trials can be performed and processed individually or recorded and processed together as in this . The known dimensions of the chessboard allow these parameters to be calculated and any lens effects to be corrected. Details on performing the lens calibration can be found in the

Extrinsic Calibration (System Calibration)

Extrinsic calibration defines the position and orientation of each camera within the global 3D coordinate system. This is essential for aligning multiple cameras into a unified space so they can work together to triangulate 3D points.

Extrinsic calibration requires:

• Chessboard

Single-camera views are insufficient to determine the board’s 3D position and orientation, therefore multi-camera visibility is critical and the best results are obtained when three or more cameras can see the chessboard pattern throughout the calibration trial.

Details on performing the chessboard calibration can be found in the

Why Both Are Required

• Intrinsic calibration ensures each camera produces geometrically correct images by accounting for internal optical characteristics and enables projection from 2D pixels coordinates to 3D rays.

• Extrinsic calibration aligns the position and orientation of each camera relative to a global coordinate system. It requires multiple cameras to view the chessboard simultaneously to resolve the calibration board's 3D pose.

Intrinsic lens calibration is performed using a calibration chessboard and is used to determine parameters associated with the camera lenses and to correct for distortion and other visual effects. Lens calibration is required for OptiTrack Prime Color camera users, and may be required for Qualisys and Vicon camera users who are not using those third parties’ wand calibration procedures. See Recording Intrinsic Lens Calibrations for detailed instructions for recording lens calibration trials.

If your lens calibration trial videos require enhancement to improve the brightness, contrast, or white balance, you can use the Enhance Videos tool. Use the lens calibration Load Videos button, then open and use the Enhance Videos tool to modify your videos. Details on why a lens calibration is needed can be found here.

Load Videos

Load video files (MP4 or AVI) containing the intrinsic lens calibration trial. To load the videos, browse to and select the folder containing the videos. The structure of this folder must conform to the format described in. Note that videos loaded using this dialog are rendered in grayscale.

Frame Grab Step

The step between frames searched for chessboards. Smaller values require longer to process, but may provide improved calibration results compared to larger values.

Use Custom Chessboard

Use a custom size chessboard for the calibration. Only select this option if not using the chessboard provided by Theia Markerless.

• Square Size: The width and height of each square on the chessboard. Measured in mm.

• Number of Squares High: The number of inner squares in the vertical direction of the chessboard.

• Number of Squares Wide: The number of inner squares in the horizontal direction of the chessboard.

Calibrate Lenses

Perform the intrinsic calibration for all of the cameras.

Save Intrinsics

Save the intrinsic calibration parameters in a format that can be loaded during future calibrations. This is useful when the intrinsic parameters do not change between calibrations. To save the intrinsic calibration, browse to the desired save location and enter the desired filename.

Load Intrinsics

Load the intrinsic calibration parameters from a previous calibration. This is useful when the intrinsic parameters do not change between calibrations. To load the intrinsic calibration, browse to and select the previously saved intrinsic lens calibration file.

Merge Intrinsics

Merge the intrinsic calibration parameters from the current calibration trial with those from a previous calibration. This is useful when new cameras have been acquired that need to be added to an existing intrinsic calibration file, or if a new intrinsic calibration trial has been recorded for a subset of cameras that are already in an existing intrinsic calibration file and the old camera parameters should be replaced. To add or replace intrinsic parameters for a set of cameras within an existing intrinsic calibration file, load the intrinsic calibration trial for those cameras, select the Frame Grab Step, and click Calibrate Lenses.

Lens Calibration Metrics

After the lenses have been calibrated, click Merge Intrinsics, then navigate to and open the existing intrinsic calibration file to which the new parameters should be merged. A new, merged intrinsic calibration file will be saved next to the existing file.

After the lens calibration is complete, a dialog will appear with the results of the intrinsic lens calibration trial.

‍The result metrics can be interpreted as follows:

Chessboards

The number of video frames used to calibrate each camera.

Coverage

The proportion (maximum value 1) of the camera view covered during the intrinsic lens calibration trial. Recommend >0.9 as a quality threshold.

Angle

The maximum angle of the chessboard relative to the camera image plane during the intrinsic lens calibration trial. Recommend 30-60 degress as a quality threshold.

Lens Calibration Review

After acknowledging the lens calibration results dialog window, the camera views will be updated to visualize the results of the lens calibration process. The green shading is a heatmap of the detected chessboard across the camera view, where the green area indicates the portion of the view that was covered during the calibration and the intensity of the green area indicates the number of frames in which the chessboard was detected while covering that part of the view.

The Default kinematic model consists of two kinematic chains for the lower body (pelvis and legs) and upper body (thorax, arms, and head).

Upper Body Kinematic Chain

Kinematic Model Tables

Torso Kinematic Chain