Installation Steps

License Activation

After installing Theia software, it is necessary to activate the license prior to using the software. To activate your Theia3D license, run Theia3D as an administrator by right-clicking on the icon and choosing 'Run as administrator'.

If the PC is connected to the internet, the license will automatically activate and Theia3D will open.

After contacting Theia Support, enter the Activation Code as provided by our team and click Activate. The license activation will then be completed, and Theia3D will open.

What Will You Do?

Theia3D is the premier solution for accurate, reliable, and generalizable markerless motion capture. With our software at your fingertips, you'll unlock the power to analyze human movement like never before. Whether you're a researcher exploring human movement, a sports biomechanist tasked with unleashing athletes' full potential, or a product developer working on the next generation of athletic equipment, Theia3D empowers you to:

• Obtain motion data without the need for suits, sensors, or markers.

• Transform raw video footage into precise, 3D models of human movement.

• Integrate effortlessly with leading motion capture software.

• Dive into the future of motion capture and elevate your project to new heights.

Using this Documentation

The reference guides contain detailed information about the interface, settings, and tools available in Theia3D. They are excellent resources for advanced users looking to fine-tune workflows and optimize Theia3D for their specific use cases.

Contact Us

GPU Selection‍

Select which graphics card(s) to use when running Theia3D.

Person Detector Size

Select which person detector to use when running Theia3D, and set the skip frames count.

• Small: fastest, may be slightly less accurate than Default or Large.

• Default: recommended.

• Large: slowest, may be slightly more accurate than Default or Small (currently disabled).

Skip frames sets the frame interval used during the analysis process to identify and track people. Default is 1, Maximum is 5. Increasing the Skip Frames value will speed up the analysis. The Skip frames value should not be used for fast movements recorded at relatively low frame rates, or movements with multiple closely-interacting individuals, as the person identification may be affected.

Joint Detector Size

Select which joint detector to use when running Theia3D.

• Small: fastest, may be slightly less accurate than Large (currently disabled).

• Large: (default) slowest, may be slightly more accurate than Small.

• Add Hands: (available as a license add-on by special request) Include hand model, which improves hand tracking through localization. Requires camera views that are positioned specifically to capture the hands and upper body. Please reach out to support@theiamarkerless.com for more information.

Listen Mode

Available as a license add-on by special request.

Listen Mode allows a directory to be selected where data is saved during data collection. The selected directory will be actively monitored for new, unprocessed data which will be automatically analyzed as it is detected. This allows data to be processed during an active data collection session, enabling all processing to be completed by the end of the collection session.

Overview

Theia3D is an advanced deep learning algorithm-based software which runs and processes data on your local PC. This means that there are specific requirements in order for it to operate properly and perform as intended, which are laid out below.

GPU

One or more CUDA-capable NVIDIA graphics card(s) with the following:

• At least 8 GB of memory

CPU

RAM

Intrinsic lens calibration trials are used to determine parameters associated with the camera lenses and are used to correct for distortion and other visual effects. Lens calibration trials are required for all camera systems that make use of a chessboard calibration method, except for Sony RX0-ii cameras. Users with wand calibration-capable systems (e.g. Qualisys Miqus, Vicon Vue or FLIR Blackfly S systems) are not required to complete lens calibration unless they are planning to use the chessboard calibration method.

Recommendations:

• Record lens calibration trials at a relatively low frame rate (20-30 Hz) to reduce file size and processing time.

• Keep the chessboard as flat as possible during the calibration trial.

• Use a computer monitor facing the person performing the calibration to provide visual feedback during the calibration.

• Move slowly and deliberately to prevent chessboard blur.

• Lens calibrations can be performed for smaller groups or individual cameras and merged afterwards. If you are finding it challenging to calibrate all at once, try recording separate groups.

Recording Lens Calibrations:

1. Place all cameras side by side on a desk or table, facing the same direction and capturing as similar views as possible. It is generally easier to calibrate the lenses with the cameras in 'landscape' orientation.

2. Stand at a distance where the chessboard occupies approximately 1/4 of the camera views.

3. Begin the recording.

4. Slowly move the chessboard in a systematic grid pattern, covering the entire field of view for every camera. Ensure the chessboard goes slightly beyond every edge of every camera field of view, and every corner.

1. Take a step back, and repeat Step 4., covering the camera fields of view again.

2. While covering the field of view, angle the chessboard slightly in multiple directions, varying its orientation throughout.

3. When you are confident that the entire field of view has been covered for all cameras, end the recording.

Extrinsic chessboard calibration trials are used to determine the position and orientation of every camera in your camera system relative to your desired global coordinate system. As with lens calibration trials, users with wand calibration-capable systems (e.g. Qualisys Miqus, Vicon Vue or FLIR Blackfly S systems) are not required to complete chessboard calibration. A minimum of one chessboard calibration trial should be collected every time you set up your camera system, but collecting multiple is recommended. A new calibration is required any time a camera is moved, so having multiple calibration trials collected throughout a long data collection session can help prevent data loss due to accidental or unnoticed camera movements.

Recommendations:

• Record chessboard calibration trials at a relatively low frame rate (20-30 Hz) to reduce file size and processing time.

• Keep the chessboard as flat as possible during the calibration trial.

• Focus on achieving groupings of 3+ cameras that can see the chessboard at all times.

• Align the chessboard with marks on the ground so you can confirm that the global coordinate system is positioned correctly when processing movement trials.

Recording Chessboard Calibrations:

2. Choose or place visible marks on the ground which will be aligned with the global coordinate system, for confirmation when processing data.

3. Check that the chessboard is fully visible in at least 3 camera views when it is placed at the desired global coordinate system origin position.

4. Begin the recording.

5. Slowly ‘show’ the chessboard to groupings of 3 or more cameras while varying the position and orientation of the chessboard slightly. Ensure the chessboard is visible to cameras that overlap between groupings, ideally so that no fewer than 3 cameras can see the chessboard at all times. Focus on achieving groupings of 3+ cameras at all times.

6. When you are confident that all cameras have had sufficient views of the chessboard, slowly place the chessboard on the ground, aligned with your preselected marks.

7. Ensure you are not obstructing the view of the chessboard in any cameras during this localization phase.

8. End the recording.

Camera Setup

Camera system setups differ from location to location, and may be subject to challenging data collection environment constraints. General recommendations for setting up your camera system include:

• Cameras as close as possible to the capture volume, while ensuring the entire capture volume remains within view for all cameras.

• Cameras 4-8 feet (1-2.5 m) above the ground.

• Avoid partial views of subjects, such as lower body or upper body only.

• Avoid unusual camera views, especially those positioned very high or very low, or extremely tilted.

• Aim for a symmetrical camera setup that surrounds the entire capture volume, such as a circle, oval, or rectangle.

Camera Settings

The camera settings you choose will depend on the movements being collected. Selection of appropriate camera settings is crucial for collecting clear, crisp video images. The most important camera settings to consider include:

• Shutter speed / Exposure: should be set to ensure the subject and their body segments are not blurry during the movement. Faster shutter speeds or shorter exposures will capture images with less movement blur, but may reduce the image brightness.

If you are collecting high-speed movements, you may need to consider introducing additional light into your capture volume in order to capture videos that are adequately bright. In general, faster movements require higher frame rates, faster shutter speeds / shorter exposures, and more light.

Subject Attire

Theia3D provides 3D pose estimates that are robust to changes in attire. General recommendations for subject attire include:

• Body-fitting clothing. Each limb should be discernible from the rest of the body.

• Clothing should provide rich visual features, such as visible creasing, shadows, or other textures.

• Lighting is often more important than attire color, as it impacts the visual richness of the attire. Under adequate lighting conditions black attire is acceptable, but lighter colors generally provide more visual features, especially in low-light environments.

Theia3D Basics

Theia3D is a markerless motion capture solution that utilizes synchronized video data to produce accurate and reliable 3D pose estimates of humans visible within the video data. It leverages deep learning algorithms trained to identify humans and accurately predict the 2D positions of 124 keypoints on the human body, in every video frame of every camera. By fitting a scaled subject-specific inverse kinematic model to the keypoint predictions, the human’s pose is reconstructed in 3D and tracked throughout their movement. This data-driven approach results in a robust solution that is generalizable across environments and movements, allowing the accessible collection of high quality 3D motion capture data where it was previously impossible.

Here, we describe the basic framework for collecting markerless motion capture data using Theia3D; for more detailed instructions, please refer to the appropriate section of this documentation and any accompanying videos.

Calibration

The calibration methods supported by Theia3D include:

• Chessboard calibration, using a large printed chessboard pattern to automatically obtain intrinsic and extrinsic camera parameters.

• Third-party calibrations, such as wand calibrations implemented by third-party motion capture hardware suppliers.

• Object calibration, using an object with known dimensions or with precisely measured key point positions to manually obtain extrinsic camera parameters.

Movement Data Collection

‍Data Processing

1. 2D Views

The videos from each camera view are shown here and the 3D scene is projected onto each view. When all views cannot fit in the available area, arrows are shown and can be used to move through the available views. Double-click on a view to maximize it. When maximized, the mouse is used to zoom (scroll) and pan (right-click and drag) the view, and clicking on the arrows moves between views. Double-click the maximized view to restore it and show all views. Use Alt + right-click to inspect a view through a magnifying class - the level of magnification is controlled by the mouse wheel and reset with the ‘R’ key.

2. 3D View

The 3D scene is rendered here. The mouse is used to rotate (left-click and drag), pan (right-click and drag), zoom (scroll), and inspect (Alt + right-click) the scene. The 3D scene contains the global coordinate system, cameras, and skeletons of each identified and tracked person. The origin of the global coordinate system is a white sphere, the x-axis is a red arrow, the y-axis is a green arrow, and the z-axis is a blue arrow. The scene enviroment, ground, and lighting can be modified from the menu accessed using the button at the top left of the 3D view.

3. Quick Tools

The quick tools are shortcuts to the most common Theia3D commands. The entire processing pipeline can be executed using these shortcuts.

4. Menu

The menu is divided into eight sections.

5. Playback Controller

The video data can be played and paused using the play button on the left, or you can scroll through the trial using the timeline slider. The number to the right of the timeline indicates the current frame number, and the playback dropdown allows you to adjust the steps between frames that are displayed. When set to one, every frame is displayed, when set to two, every other frame is displayed, and so on.

‍6. Status Bar

Below the playback timeline controller is a status bar which indicates the current status and any ongoing processes.

Extrinsic object calibration trials are used to determine the position and orientation of every camera in your system relative to the desired global coordinate system. Extrinsic object calibration uses the static position of a calibration object with known dimensions or known positions of specific key points on the object. These 3D dimensions or positions must be measured with high precision. The calibration object should be sufficiently large or the calibration key points should be spaced far apart within the capture volume and most points should be visible in every camera view. The key points can be coplanar or can vary in all three global dimensions.

A calibration object file must be created for your calibration object, containing 3D coordinates of the object key points. Each line of the file must contain three comma-separated values representing the global x-, y-, and z-coordinates of the key point, in millimeters. Each key point should be on its own line.

Object calibrations must be performed at the resolution that will be used to record your movement data. If you will be using more than one resolution during your data collection, you must record object calibration trials at each resolution.

Recommendations:

• Record object calibrations trials at a relatively low frame rate (20-30 Hz) to reduce file size and processing time.

• Object calibration trials only need to be a few seconds long.

Recording Object Calibrations:

1. Place your calibration object within the capture volume, at the desired position and orientation to define the global coordinate system.

2. Ensure the key points on your calibration object are visible in every camera view, and there are no obstructions.

3. Begin the recording.

4. After a few seconds, end the recording.

Shortcut: Ctrl + F4

Clear all loaded data and analysis results.

Shortcut: Ctrl + O

Shortcut: Ctrl + Shift + O

Open a previously saved workspace. To load the workspace, browse to and select the directory containing the workspace.

Shortcut: Ctrl + S

Save the videos, camera calibration, and analysis results in the current workspace to a directory in a format that can be opened by Theia3D. Select the frames to include in the workspace and press the Save button. Browse to the desired save location and enter the name of the directory that will be created to store the workspace files. To overwrite the current workspace, use the dialog to select a .t3d file in the existing workspace folder. Note that this only overwites the results (.t3d and .p3d files) and leaves the videos untouched. In this case, all frames in the workspace must be saved. It is not possible to save to an existing workspace folder other than the currently open one.

Save the current calibration. To save the calibration browse to the desired save location and enter the desired filename.

Save the scaled 3D model. To save the scaled model browse to the desired save location and enter the desired filename. Note that if multiple people are tracked a separate model will be saved for each of them. The files are automatically named as filename_personID.tmb, where personid is automatically assigned.

Shortcut: Ctrl + Shift + S, 1

For .c3d files there is an option to save a multi-subject file. If this option is selected, two .c3d files will be created - one with the unfiltered pose and one with the filtered pose. Both files will contain all tracked subjects. If the multi-subject option is not selected, separate files will be created for each tracked subject.

‍

For .fbx files there is an option to select the exported cooridinate system. Preset options are available for common software programs or custom coordinate system can be defined. Saving an .fbx file requires the full body model to be used when solving the pose.

‍

For .json files the user can select which tracked individuals and which frames of the trial to include in the output file. The .json output files are easily readible in a text editor or used as input for a custom analysis script.

‍

With the desired options selected, press the Save button, browse to the save location and enter the filename. The person ID and filtered tag are appended to the filename as needed.

Opens the pose data for all tracked people in Visual3D. Note: The filtered poses are loaded into Visual3D, and the path to the Visual3D executable must be set correctly in the settings dialog. We strongly recommend an active support agreement for Visual3D, as it is frequently updated to support the latest Theia model changes.

Shortcut: Ctrl + Shift + S, 2

Graph Joint Angles

• Subject: The person to plot Euler angles for.

• Reference Segment & Segment: The Euler angles describe the orientation of the Segment relative to the Reference Segment, resolved in the Reference Segment coordinate system using a ZYX cardan sequence.

• Flip X/Y/Z: Flip the X, Y, or Z angle plots.

Show Batch Progress

View Command Results

The view command results window shows the commands that have completed by Theia3D while handling and processing data, including specific inputs and outputs in the dropdown menus when expanded. The text color indicates the step status, where green indicates success and yellow indicates a warning is present.

Show all will show hidden, non-critical commands in the list.

Clear Results will clear any commands currently in the Commands list.

Most of the Display options shown here can also be toggled ON and OFF using the Display toggle buttons in the sidebar.

Show/Hide 3D View

Toggle showing the 3D View in the application. If toggled ON but the 3D View is not visible, drag the 3D View open from the right border of the application window.

Person identification boxes can be toggled on and off using the sidebar button.

Toggle the boxes around all people found in each 2D view. The color of the box around each identified person is unique to that person and the same in all views. The boxes around people who have not been identified are grey. Person identifications are also printed in the upper left corner of the boxes.

3D segments can be toggled on and off using the sidebar button.

Toggle the 3D segments of each identified person in the 2D views and in the 3D view. The color of the segments is unique to the identifed person and matches the color of the 2D boxes corresponding to that person.

Skeletons can be toggled on and off using the sidebar button.

Toggle the skeletons of each identified person in the 2D views and in the 3D view. The color of each skeleton is unique to the identified person and matches the color of the 2D boxes corresponding to that person.

Local segment coordinate systems can be toggled on and off using the sidebar button.

Toggle the local coordinate systems of segments and cameras in the 2D views and in the 3D view. Note that the local coordinate systems of the segments are only shown if the segments or skeleton are visible. The origin of each coordinate system is a white sphere, the x-axis is a red arrow, the y-axis is a green arrow, and the z-axis is a blue arrow.

Cancel the currently running analysis. This option is available (and replaces all other options) only when analysis is running.

Detect people in each of the loaded videos. When complete, boxes are drawn around the detected people in the video overlays if Show Boxes is selected in the Display menu. Video data and calibration must be loaded before running Run 2D.

Identify people across views (i.e., determine “who is who” in each of the views). Identification requires the person to be clearly visible in at least three views simultaneously. Each identified person is assigned a unique color that is applied to their boxes, segments, and skeleton. Run 2D must be completed before running Track People.

Runs: Track People and Solve Skeleton. Run 2D must be completed.

Shortcut: Ctrl + F

Runs: Run 2D, Track People, and Solve Skeleton. Video data and calibration must be loaded.

The extrinsic camera calibration can be checked after the trial has been analyzed fully. For each view, the 2D representation of the skeleton in that view is compared to the 3D skeleton computed from all views. If the alignment can be improved in at least 30% of the frames by translating the entire skeleton within the 2D image for that view, this information is reported in the dialog. You can then choose to deactivate the cameras that are out of calibration and clear the entire analysis.

Load Videos

Frame Grab Step

The step between frames searched for chessboards. Smaller values require longer to process, but may provide improved calibration results compared to larger values.

Use Custom Chessboard

Use a custom size chessboard for the calibration. Only select this option if not using the chessboard provided by Theia Markerless.

• Square Size: The width and height of each square on the chessboard. Measured in mm.

• Number of Squares High: The number of inner squares in the vertical direction of the chessboard.

• Number of Squares Wide: The number of inner squares in the horizontal direction of the chessboard.

Calibrate Lenses

Perform the intrinsic calibration for all of the cameras.

Save Intrinsics

Save the intrinsic calibration parameters in a format that can be loaded during future calibrations. This is useful when the intrinsic parameters do not change between calibrations. To save the intrinsic calibration, browse to the desired save location and enter the desired filename.

Load Intrinsics

Load the intrinsic calibration parameters from a previous calibration. This is useful when the intrinsic parameters do not change between calibrations. To load the intrinsic calibration, browse to and select the previously saved intrinsic lens calibration file.

Merge Intrinsics

Merge the intrinsic calibration parameters from the current calibration trial with those from a previous calibration. This is useful when new cameras have been acquired that need to be added to an existing intrinsic calibration file, or if a new intrinsic calibration trial has been recorded for a subset of cameras that are already in an existing intrinsic calibration file and the old camera parameters should be replaced. To add or replace intrinsic parameters for a set of cameras within an existing intrinsic calibration file, load the intrinsic calibration trial for those cameras, select the Frame Grab Step, and click Calibrate Lenses.

Lens Calibration Metrics

After the lenses have been calibrated, click Merge Intrinsics, then navigate to and open the existing intrinsic calibration file to which the new parameters should be merged. A new, merged intrinsic calibration file will be saved next to the existing file.

After the lens calibration is complete, a dialog will appear with the results of the intrinsic lens calibration trial.

‍The result metrics can be interpreted as follows:

Chessboards

The number of video frames used to calibrate each camera.

Coverage

The proportion (maximum value 1) of the camera view covered during the intrinsic lens calibration trial. Recommend >0.9 as a quality threshold.

Angle

The maximum angle of the chessboard relative to the camera image plane during the intrinsic lens calibration trial. Recommend 30-60 degress as a quality threshold.

Lens Calibration Review

After acknowledging the lens calibration results dialog window, the camera views will be updated to visualize the results of the lens calibration process. The green shading is a heatmap of the detected chessboard across the camera view, where the green area indicates the portion of the view that was covered during the calibration and the intensity of the green area indicates the number of frames in which the chessboard was detected while covering that part of the view.

Load Videos

Frame Grab Step

The step between frames searched for chessboards. Smaller values take longer to process, but may provide slightly improved calibration results compared to larger values.

Camera Type

Detects and indicates the camera manufacturer based on the videos loaded. This allows the automatic application of default Sony RX0 II camera intrinsic parameters when the calibration videos were recorded using this camera system (see below).

Use Default RX0 II Intrinsics

Automatically selected after loading videos recorded using a Sony RX0 II camera system. This option should be selected to utilize the built-in intrinsic parameters for the Sony RX0 II cameras. Note: These parameters are only valid for Sony RX0 II cameras with standard lenses. For any other cameras or lenses these default parameters are invalid.

‍Load Custom Intrinsics

Loads intrinsic lens parameters for the camera views loaded.

Origin Frame

Frame to use to set the global coordinate system of the capture volume. If the chessboard is not detected in at least three views in this frame, the closest frame in which it can be detected will be used.

Min 2 Cams for Origin Triangulation

Uses two (instead of the default of three) cameras to locate the origin frame. Use this option if the board is difficult to see in the triangulation frame from three cameras.

Normal Axis

Set the coordinate system axis that is defined by the normal axis of the chessboard.

Long Axis

Set the coordinate system axis that is defined by the long axis of the chessboard.

Use Custom Chessboard

Use a custom size chessboard for the calibration. Only select this option if not using the chessboard provided by Theia Markerless.

• Square Size: The width and height of each square on the chessboard. Measured in mm.

• Number of Squares High: The number of inner squares in the vertical direction of the chessboard.

• Number of Squares Wide: The number of inner squares in the horizontal direction of the chessboard.

Calibrate Cameras

Perform the extrinsic calibration for all of the cameras.

Chessboard Calibration Metrics

After the camera calibration is complete, a dialog will appear with the results of the chessboard calibration trial. The result metrics can be interpreted as follows:

The result metrics can be interpreted as follows:

Frames

The number of video frames used to calibrate each camera.

RMSE Reprojection

RMSE error of the reprojected 3D chessboard points relative to the detected chessboard points in 2D, for each camera view. Measured in mm. Since this is an error relative to 0 mm for all chessboards regardless of size, a 1 mm RMSE reprojection has the same meaning for all sizes of chessboard.

RMSE Diagonal

RMSE error of the length of the diagonals of the 4 outer corners of the chessboard relative to their known lengths, for all triangulated chessboard detections. Measured in mm. Provides an absolute error measure, not normalized to the size of the chessboard. Therefore, a 1 mm RMSE diagonal for a large board is a smaller percentage error than a 1 mm RMSE diagonal for a small board. Not affected by the number or size of the chessboard squares. Recommend using <1 mm as ‘Excellent’ and <2 mm as ‘Acceptable’; calibrations with RMSE Diagonal above 2 mm are not recommended for use.

RMSE Angle

RMSE error of the angle of the 4 outer corners of the chessboard relative to their known angle of 90 degrees, for all triangulated chessboard detections. Measured in degrees. Since this is an error relative to 90 degrees for all chessboards regardless of size, a 1 degree RMSE angle has the same meaning for all sizes of chessboard. Not affected by the number or size of the chessboard squares.

RMSE Flat

RMSE error of the normal distance from each detected chessboard point in 3D to the flat plane formed by the outer four 3D chessboard points. Measured in mm. Provides an absolute error measure, not normalized to the size of the chessboard. Therefore, a 1 mm RMSE flat for a large board indicates a lesser degree of bend in the chessboard than a 1 mm RMSE flat for a small board.

Origin Triangulation Frame

The video frame used to initialize the position and orientation of the global coordinate system.

After reviewing the calibrations results, the folling options are available:

Save allows the calibration to be saved.

Save & Assign initiates two steps:

1. Opens the Save Calibration window, allowing the calibration to be saved as a .txt file.

2. Opens the Assign Calibration tool, allowing the previously saved calibration .txt file to be immediately assigned to movement trials.

Ok acknowledges and closes the results dialog window.

Chessboard Calibration Review

After the camera calibration is complete, the calibration trial videos can be reviewed for feedback. The following visual cues projected onto the camera views can be useful in determining which portions of the trial contributed to the calibration, and those that did not. This can be useful for optimizing your calibration trial technique.

The Organize Videos tool converts a folder of videos into a structure that can be used by Theia3D. This functionality is useful when collecting large data sets.

Basic

The Basic tab is the best option when organizing videos that have user-defined file names, such as those collected using OptiTrack, Qualisys, or Vicon camera systems. To use this feature, the video files must be named using the following reserved terms, separated by underscores or periods:

• Cam ID: (Required) The ID of the camera corresponding to the video file. This must be the same ID as in the calibration file.

• Subject: (Recommended) Unique subject identifier (i.e. sub001).

• Action: (Recommended) The action the subject is performing (i.e. walk).

• Trial: (Recommended) The trial number (i.e. 001 or walk01).

• N/A: (Optional) Additional terms in the filename not used by Theia3D.

‍To format the videos, select the delimiter used in the file name and then select the folder containing the videos using the Browse button. When the directory is selected the dialog will parse the name of a sample file. Note that the delimiter can be changed after the folder is selected and the sample file name will be re-parsed. Use the drop down boxes to indicate the reserved term that each part of the filename corresponds to. Click Format Data to convert the folder to the required structure. The hierachy of the created directory is Subject > Action > Trial > Cam ID. Cam ID is the only reserved term that must be included in the filename; however, it is recommended to include as many of the other reserved terms as possible.

For example, consider a folder with the following video names:

• S0001_Walk_001_8_21375.avi

• S0001_Walk_001_8_21376.avi

• S0002_Walk_001_8_21375.avi

To format these videos, the reserved terms selected are: Subject, Action, Trial, N/A, and Cam ID. The data structure after formatting is:

Advanced

The Advanced tab is the best option when organizing videos that do not have user-defined file names, such as those collected using Sony RX0 II camera systems. To use this feature, the names of the video files must end with camid_trialid, where camid is the ID of the camera that recorded the video and trialid is a unique identifier for the trial.

‍

To format the videos, select the folder containing the videos using the Browse button. The trial grid is then created with one row for each trial. The File column of each row is filled with the trialid for that trial.

Use the Load Subject and Action Assignments button to load the subject and action IDs for each of the trials from a csv file. Each row of the csv file must contain the trialid, subjectid, and actionid of a single trial, in that order, separated by commas. For example, the csv file used to assign the subject and action IDs in the above figure was:

Press the Format Data button to format the videos in the required folder structure.

Subject and action IDs can also be manually assigned to the trials without using a csv file.

Use the Build Subjects button to open another dialog and create the subject IDs for the trials. A list of previously saved subject IDs can also be loaded and the current list of subjects can be saved using this dialog. Press the Accept button and close the dialog when the subject list is complete. To assign a subject ID to the trials, select the subject ID from the dropdown menu and press the Add button. Select the trials corresponding to the subject from the trial grid (multi-select is possible using Ctrl or Shift modifiers and by click-and-drag) and press the Finish button.

Use the Build Actions button to open another dialog and create the action IDs for the trials. A list of previously saved action IDs can also be loaded and the current list of actions can be saved using this dialog. Press the Accept button and close the dialog when the actions list is complete. To assign an action ID to the trials, select the action ID from the dropdown menu and press the Add button. Select the trials corresponding to the action from the trial grid (multi-select is possible using Ctrl or Shift modifiers and by click-and-drag) and press the Finish button.

Press the Format Data button to format the videos in the required folder structure.

The Adjust Calibration tool can be used to modify the position and orientation of the global coordinate system (GCS) after completing a chessboard or object calibration, or after loading an existing calibration file. The GCS projection on each 2D camera view and within the 3D View is updated live as the Position and Angle sliders are used to modify these parameters of the GCS localization.

This tool can be used to reposition the GCS to a more desirable position or orientation if the desired origin frame could not be used from the chessboard calibration trial, for any other methodological reason that requires the GCS in a specific position.

Position (X, Y, Z)

X, Y, Z sliders can be used to change the position of the GCS relative to its original position, measured in mm.

Angle (X, Y, Z)

X, Y, Z sliders can be used to change the orientation of the GCS relative to its original orientation, about the respective axis of the GCS, in degrees.

Reset

Reset the position and angle sliders to 0.0.

Apply

Apply the selected position and angle adjustments to the current camera system calibration.

Apply and Save

Apply the selected position and angle adjustments to the current camera system calibration, and save the result as a new calibration .txt file.

Synchronization of the video images can be checked after the trial has been analyzed fully. For each view, the 2D representation of the skeleton in that view is compared to the 3D skeleton computed from all views. If the alignment for a given view can be improved by shifting its video sequence forward or backward in time, this frame offset is reported in the dialog. You can then choose to correct the synchronization by applying these offsets to the out-of-sync views and clearing the entire analysis.

Load Videos

Camera Type

Detects and indicates the camera manufacturer based on the videos loaded. This allows the automatic application of default Sony RX0 II camera intrinsic parameters when the calibration videos were recorded using this camera system (see below).

Load Default RX0 II Intrinsics

Load default lens calibration parameters for Sony RX0 II cameras and apply them to all cameras. This can be used to avoid performing a lens calibration when using the RX0 II cameras. Loading default intrinsics overwrites any existing lens calibration parameters. Note: These parameters are only valid for Sony RX0 II cameras with standard lenses. For any other cameras or lenses these default parameters are invalid.

Load Object

Load a .txt file defining the calibration object. Each line of the file contains comma-separated x-y-z coordinates of one of the key points on the object in mm. To load the object, browse to and select the object file. Once loaded, the object points will be displayed in the table.

Save Object

Save the current object definition. To save the object, browse to the desired save location and enter the desired filename.

Add

Add a calibration object point definition (x, y, z coordinate values).

Remove

Add a calibration object point definition (x, y, z coordinate values).

Auto-Step

If checked, the selected object point will advance each time a point is identified in one of the 2D views.

Cam ID

The ID of the camera corresponding to the current 2D view. Use the drop-down to select a different 2D view. Maximizing a view will automatically select it.

Reset Point

Remove the currently selected point from the current view.

Reset Camera

Remove all points from the current view.

Reset All Cameras

Remove all points from all views.

Calibrate Cameras

Perform the extrinsic calibration for all of the cameras. After performing the calibration, the position of any camera points that were not identified before the calibration are calculated and drawn in the 2D views as an ‘*’.

Adjust Origin

When using the checkerboard extrinsic calibration, sometimes the origin may appear at an undefined location because the board was not recognized in the origin frame. This feature allows you to adjust the reference frame by clicking on an object (typically four squares on the checkerboard). Since the frame is just being adjusted (not computed), a calibration needs to be loaded to use this feature, and the relative pose of the cameras will be unchanged. Note: To adjust the origin, you need to manually identify the object points for three cameras only.

To use this tool, the raw video data must be recorded using the Sony RX0 II camera system, and should be downloaded using the options: Add the Camera Label (prefix) and Do not add the Shooting Date/Time. The raw video files must be accompanied by a data spec .csv file, which can have any name, however it must be the only .csv file within the directory alongside the video files. The data spec .csv file must have a header row of: Trial,SubjectId,Type,Action,CalibrationId. Each subsequent row should contain the relevant information under each header column that corresponds to the trial.

For example, a data collection consisting of the following recordings in this order:

• subject 1, walk

• subject 1, run

• subject 2, walk

• subject 2, run

• calibration

• subject 3, walk

• subject 3, run

would require a data spec .csv as follows:

Trial,SubjectId,Type,Action,CalibrationId
1,N/A,Calibration,N/A,N/A
2,S0001,Motion,Walk,1
3,S0001,Motion,Run,1
4,S0002,Motion,Walk,1
5,S0002,Motion,Run,1
6,N/A,Calibration,N/A,N/A
7,S0003,Motion,Walk,2
8,S0003,Motion,Run,2

/Chessboard
    /Calibration
        /001
            /[cam ID folders]
            /…
        /002
            /…
    /S0001
        /Walk
            /001
            /…
        /Run
            /001
            /…
    /S0002
        /Walk
            /001
            /…
        /Run
            /001
            /…
    /S0003
        /Walk
            /001
            /…
        /Run
            /001
            /…

After successful formatting of the data, an additional file changelog.csv is created which documents the mapping of raw video files to the organized file structure.

The Modify People IDs tool allows you to swap the unique person ID numbers assigned to each person when multiple people are tracked in a trial. Click and drag an ID and drop it onto another ID to swap those two ID numbers. This change to the IDs is immediately visible in the 2D and 3D views. You can also right-click on an ID to display a context menu with options for removing that tracked person or making it person 0.

The Display Video Metadata tool can be used to easily review relevant metadata for the videos currently loaded, including:

Camera Id

Camera ID dropdown, allowing separate cameras to be selected and reviewed.

Video Path

Path to the video file for the currently selected camera.

Trial

Name of the lowest folder level.

Size

Video image resolution, in pixels.

Number of Frames

Video file length, in frames.

Frame Rate

Video file frame rate, in frames per second.

Frame Offset

Number of frames offset between the current video file and other synchronized video files from the current recording. Generally a value of 0 is expected for most camera systems that record frame synchronized videos using on a start/stop trigger signal. Sony RX0 II camera systems may have non-zero values here, as these systems use timecode video synchronization for post-hoc alignment of videos between cameras.

‍The Toggle Views tool can be used to turn on and off individual camera views for the currently loaded trial. When turned off, camera views are greyed out in the 2D viewer area, and are not used when analyzing the trial.

Toggle Views adjustments are included when saving preferences files.

The assign calibration file tool is used to copy and nest an extrinsic calibration file within individual trial folders, one level above the video files. This is best practice for storing calibration files with their associated movement trials, and is a requirement for running batch analyses.

Use the Browse button to navigate to and select your extrinsic calibration .txt file.

Use the plus button to add trials to the Trials list by selecting a folder containing movement trials. All trials within the selected folder will be added to the Trials list, regardless of their depth within the selected folder.

Use the minus button to remove currently selected trials from the Trials list. Use shift+click or ctrl+click to select groups or individual trials to remove from the Trials list.

Just Add to Selection

Assign the calibration file to only the currently selected files within the Trials list.

Cancel

Close the Assign Calibration Files tool without assigning any calibration files.

Assign

Assign the calibration file to the trials in the Trials list or only the currently selected trials if Just Add to Selection is selected.

The Video Enhancement tool allows you to improve video quality before processing data, which can be especially useful for calibration under challenging lighting conditions. Changes made using this tool are previewed for the loaded 2D views, and any enhancements are applied before running the 2D inference, lens calibration, or chessboard calibration processes. Therefore, what you see in the 2D views is how the video data will be processed.

All Views

Apply the selected enhancements to all camera views. Alternatively, select an individual camera from the dropdown menu to apply individual camera enhancements.

Brightness

Increase or decrease the brightness of the videos. Default (no enhancement) is 0.

Contrast

Increase the contrast of the videos. Default (no enhancement) is 1.00.

White Balance

Adjust the white balance of the videos by performing a color correction using a baseline white color. Enable White Balance by selecting the square, and click on the white rectangle to open the color picker. You can select a specific color, input color values, or use the Pick Screen Color option to select a pixel from within your loaded videos of a white surface such as the chessboard.

Blue Mask

Shows which pixels are detected as blue by the chessboard calibration algorithm. This is useful for adjusting your videos before processing a chessboard calibration to ensure the blue squares are detected. The global coordinate system origin is defined at the intersection of the blue squares, which must be detected in order to define the origin.

Reset

Reset all changes to default values, removing any video enhancements.

The Analysis preferences pane contains advanced options and parameters for adjusting how the movement is analyzed, including subject identification and person tracking, model parameters, and 3D reconstruction parameters.

Analysis Frame Range

First and last frame to analyze (inclusive). Modified frame range values are included when preferences are saved. (Default: full trial length)

Image Matte Percentage

Image exclusion border size. Image matte area is displayed as a grey border around the camera views when active. The greyed-out portions of the videos will not be used when running the analysis, allowing poorly calibrated image borders to be ignored. Image matte area is not applied during lens calibration.

Max People

Maximum number of people to track. (Default: No max)

Person Tracking Mode

Select the method for determining person tracking priority. Run Analysis (without 2D) must be performed in order to update person tracking.

• Most Visible: Person identification and tracking performed by prioritizing people who are visible in >75% of the total frames from all cameras throughout the entire trial, and ordering them by their distance to the global coordinate system origin.

• Closest to Origin: Person identification and tracking performed by ordering all people by their minimum distance to the global coordinate system origin throughout the entire trial. For example, if a person walks directly over the origin, they will likely be identified as person 0.

• Order of Appearance: Person identification and tracking performed by numbering people in order of appearance within the capture volume.

Remove stale IDs

Remove any person IDs that are missing for 100 consecutive frames or more. This allows a person who is tracked repeatedly within a single trial (e.g. during several passes into and out of the volume) to be tracked and therefore exported separately, for instance as multiple separate c3d files.

Track Rotating People (beta)

Improve tracking of rotating and non-upright people at the expense of speed. (Default: Off)

If not selected, the generic model will be scaled during Solve Skeleton. If selected, a previously saved model can be used by the inverse kinematics algorithm. Use the Use Saved Model button on the next line to select the model to be used. Note that this option is only available when Max People is 1. (Default: Off)

Large Lens Distortion

Uses a larger grid for lens distortion correction. Only use if there is a lot of distortion and loading is slow. (Default: Off)

Default Model (neither Full Body nor Separate Arm and Head model selected)

‍Use Full Body Model

‍Use Separate Arm and Head Models

Enable 3 DOF Knee

If selected, the model used by the inverse kinematics algorithm will have three degrees of freedom at the knee (flexion/extension, ab/adduction, and internal/external rotation). If not selected, the model will have two degrees of freedom at the knee (flexion/extension and ab/adduction). (Default: Off)

Smoothing Freq. (Hz)

Cutoff frequency of the GCVSPL lowpass filter used to smooth the pose from the inverse kinematics. (Default: 20Hz)

Display/Use

Enables the 3D Analysis Bounding Box for use, which restricts person tracking to within a specified 3D volume. This allows people who are visible but not of interested to be ignored, such as the experimenter or other observers. Run Analysis (without 2D) must be performed in order to update person tracking and skeleton visibility.

Use Camera Locations

The 3D camera locations will be used to establish the edges of the analysis bounding box, rather than the x, y, and z origin positions and length, width, and height dimension definitions.

• Origin X: Position of the center of the bounding box along the global coordinate system X-axis.

• Origin Y: Position of the center of the bounding box along the global coordinate system Y-axis.

• Origin Z: Position of the center of the bounding box along the global coordinate system Z-axis. Minimum value is half the bounding box height.

• Length: Bounding box dimension along the global coordinate system X-axis.

• Width: Bounding box dimension along the global coordinate system Y-axis.

• Height: Bounding box dimension along the global coordinate system Z-axis.

The rendering preferences pane contains advanced options and parameters for how the data should be played back and rendered.

Playback

Playback rate during continuous playback. A value of 1 will show every frame, 2 will show every other frame, etc. (Default: 1)

Skeleton Alpha

Set the opacity of the skeletons, 3D segments, and local coordinate systems rendered in the 2D views. The slider ranges from 0 (transparent) to 100 (opaque).

Render Smooth IK

If selected, the filtered pose of the skeletons will be rendered. If not selected, the unfiltered pose will be rendered. (Default: Off)

Shortcut: Ctrl + ,

‍

The settings available at the top of the preferences pane enable the user to load, save, and save default preferences.

Loaded Preferences File

Displays the path to the preferences file from which the current preferences were loaded.

• Load: Allows the selection of a new preferences .pxt file.

• Save As: Allows the current preferences to be saved as a new .pxt file.

Default Preferences

• Load: Allows previously specified default preferences to be selected and loaded.

• Save As Default: Allows the current preferences to be saved as a new set of default preferences.

‍

The Setup preferences pane contains options and parameters for the software setup and startup.

Visual3D Path

Path to Visual3D.exe location. Must be set in order to load data in Visual3D from Theia3D. Use the Visual3D Path button to browse to and select your Visual3D.exe file.

Select GPUs on launch.

Show or hide the startup/GPU selection dialog when Theia3D launches.

Opens the online documentation for the software.

The About dialog shows the following information:

• Software version number

• Release date

• License key

• License counts

• Support expiry

• GPU utilization information

‍

There are currently three available models in Theia3D:

1. • Consists of two kinematic chains: lower body (pelvis and legs) and upper body (torso, arms, and head).

   • No abdomen or neck segments.

   • Shoulder joints and head segment are allowed 6 degrees of freedom.

2. 1. Consists of one, whole-body kinematic chain.

   2. Abdomen and neck segments included.

   3. Shoulder joints are allowed 6 degrees of freedom.

   4. Head is allowed 3 degrees of freedom.

3. • Consists of five kinematic chains: lower body (pelvis and legs), torso, left arm, right arm, and head.

   • No abdomen or neck segments.

   • Shoulder joints and head segment are allowed 6 degrees of freedom.

Pose

Model pose can be exported to .c3d, .fbx, and .json files.

The .c3d files contain the 4x4 pose matrices for each model segment and the local coordinates of the anatomical landmarks of the distal segments of the model (feet, hands, head). These files can be processed using Visual3D.

The .fbx files contain the hierarchical skeleton, pose, and bone meshes of the animation model. The first frame of the file contains the model in a “T-Pose”. The skeleton must be solved using the Full Body Model in order to save pose files in FBX format.

The .json files contain information about how the trial was processed (Theia3D version, model, preferences, etc.) and the 4x4 pose matrices for each body segment for every frame of the trial.

Kinematic Model Tables

Torso Kinematic Chain

Head Kinematic Chain

Right Arm Kinematic Chain

Left Arm Kinematic Chain

Lower Body Kinematic Chain

The full body kinematic model consists of one kinematic chain comprised of the lower body, upper body, and head, with the pelvis as the root segment. Abdomen and neck segments are included. The Full Body Model must be used in order to save skeleton poses as FBX format.

Full Body Kinematic Chain

Video Data

‍

The video data for a single trial must be contained in its own folder, and each video file must be in its own subfolder. The name of each subfolder must be the ID of the corresponding camera. There are no requirements on the structure of the names of the video files, but the names must be unique and videos must be .avi or .mp4 format. For example, the figure above shows video data for a walking trial collected using four cameras with IDs 21375, 21379, 21380, and 21381.

The Default kinematic model consists of two kinematic chains for the lower body (pelvis and legs) and upper body (torso, arms, and head).

Upper Body Kinematic Chain

Lower Body Kinematic Chain

A workspace folder contains the video and data files of a saved workspace. The video files are named according to their unique camera ID. The .t3d and .p3d files contain the analysis results. It is important that the contents of the workspace folder are not modified, including moving files in or out of the folder.

Theia3D workspaces can be used to save data at various stages of analysis, but are most useful for saving analyzed data that will be reviewed later. When a fully analyzed movement trial is saved as a Theia3D workspace, the saved data includes the 2D videos, calibration file, and all data associated with the analysis. Therefore, when a saved workspace is loaded, you can immediately review the analyzed data in the 2D and 3D viewers.

Top open a Theia3D workspace while Theia3D is closed, you can double-click on the results.p3d file.

Camera Calibration

‍

The camera calibration file contains the calibrations for all of the cameras using a structure similar to XML. An example calibration file is provided with the sample data. Key elements and attributes of the calibration file are outlined here. Note that Qualisys calibrations exported from QTM as .txt files and Vicon calibrations (.xcp) can also be used.

‍<calibration>

Top level element with no attributes.

Grouping element that holds <camera> child elements. It has no attributes.

<results>

<camera>

Element that holds the calibration information for a single camera. Required attributes are: active, serial, and viewrotation. Required child elements are: <transform> and <intrinsic>.

• active: 1 if the camera is used, 0 if unused.

• serial: The camera ID. Must be unique to the camera and match the camera ID used to name the video subfolders.

• viewrotation: Rotation of the camera. 0 if upright, 180 if upside down, 90 or 270 if sideways. This is calculated from the rotation matrix of the calibration.

<transform>

Translation and rotation components of the transformation from global to camera coordinates (p→cam=Rp→global+t→). Required attributes are x, y, z and r11 through r33.

• x, y, z: Elements of the translation component (t→) of the transformation. Expressed in mm.

• rij: Elements of row i and column j of the rotation component (R) of the transformation.

<intrinsic>

Intrinsic camera parameters. Required attributes are: focallength, sensorMinU, sensorMaxU, sensorMinV, sensorMaxV, focalLengthU, focalLengthV, centerPointU, centerPointV, skew, radialDistortion1-3, and tengentialDistortion1-2.

• focallength: Focal length of the lens in mm.

• sensorMinU: Minimum u coordinate of the sensor in pixels.

• sensorMaxU: Maximum u coordinate of the sensor in pixels.

• sensorMinV: Minimum v coordinate of the sensor in pixels.

• sensorMaxV: Maximum v coordinate of the sensor in pixels.

• focalLengthU: Focal length along the u axis in pixels.

• focalLengthV: Focal length along the v axis in pixels.

• centerPointU: Principal point u coordinate in pixels.

• centerPointV: Principal point v coordinate in pixels.

• skew: Skew coefficient. Non-zero if the image axes are not perpendicular. Note that non-zero skew is currently unsupported.

• radialDistortion1-3: Radial distortion coefficients.

• tangentialDistortion1-2: Tangential distortion coefficients.

Output .json files contain:

• Theia3D version number, engines, and kinematic model information

• Trial frame rate

• Preferences used during process, such as start/end frame, max people, smoothing frequency, etc.

• Tracking data for each tracked individual, including ID number, segment names and 4x4 pose matrices, and parameters such as segment length.

FBX file outputs can be saved with a variety of different segment coordinate system conventions for your convenience.

One of the benefits offered by Theia3D markerless motion capture is automated tracking, which allows it to analyze large datasets without human intervention or supervision. This is achieved using the Theia3D Batch companion application to Theia3D, which allows a list of trials to be curated and batch analyzed sequentially. While batch processing is efficient and does not require supervision, we always recommend that you manually examine and check the quality of your markerless data and calibrations using Theia3D, before setting up a batch analysis. This can prevent poor calibrations or other issues with the data from going unnoticed until after the batch analysis has been completed.

‍

The Open into options allow you to easily open the currently selected batch processing directory, in either Windows Explorer or the command line.

Opens the online documentation for the software.

Opens the pdf documentation to the software

The About dialog shows the following information:

• Software version number

• Release date

• Contact information

Settings

Data Path

Currently selected batch analysis root folder.

Select the batch analysis root folder.

Refresh the batch analysis root folder to update the Trials list with changes to the folder.

Save Workspace

If selected, a Theia3D workspace is saved for each trial in a folder called inputdirectory_workspace. This folder is in the same location and has the same structure as inputdirectory.

Save JSON

If selected, the pose data for all individuals tracked in each trial will be saved in a folder called inputdirectory_json. This folder is in the same location and has the same structure as intputdirectory.

Save C3D

If selected, the pose .c3d files will be saved for each trial in a folder called inputdirectory_c3d. This folder is in the same location and has the same structure as inputdirectory.

Save Fbx

If selected, the pose .fbx files will be saved for each trial in a folder called inputdirectory_fbx. This folder is in the same location and has the same structure as inputdirectory. The coordinate system convention used in the .fbx file can be selected from the dropdown box.

Use Hierarchical Names

(Only applicable when Save C3D is selected) If selected, the pose .c3d files will be created with file names that combine the folder names from the lowest n levels of the batch analysis folder hierarchy, where n is the selected Level dropdown value. For example, if selected and Level=3 for data that is structured as [subject] / [action] / [trial] / [camID], the output files will be named: [subject]_[action]_[trial]_pose_filt_#.c3d.

The Trials list is populated with all valid trials within the selected batch analysis root folder, once a folder has been selected. Above the trials list are the following options:

Search

‍Filters trials via string compare and regular expression. Built-in filter options based on trial states can be accessed using the @ symbol, including those shown below.

After searching, the selection box next to ‘Trials’ can be toggled to select or deselect all trials that were returned based on the string search. Helps with batching a subset of the root folder.

Details

Show trial metadata like assigned calibrations, trial length (# frames), and frame rate (FPS).

Right-clicking on any trial within the Trials list will provide the following options:

Reset Analysis Range will reset the Analysis Frame Range preference to be reset so that the entire trial will be analyzed from the first to last frame of data.

The Trials list has the following features within each row. Right-click on the Trials list header to enable or disable specific details columns as desired.

Dropdown triangle icon that expands to show the processing steps for each trial. Indicates success, warnings, or failures for each step where required.

Selection square that indicates whether the trial is included in the batch analysis. Initially, all trials are selected.

Trial Path

The full path to the trial.

State

Circle that indicates the status of the trial:

Calibration Indicator

Icon that indicates the status of the calibration file for the trial

Calibration Filename

Filename for the calibration file assigned to the trial. Allows easy distinction when different calibration files should be used for different movement trials within one batch analysis.

Preferences Indicator

Icon that indicates the status of the preferences file for the trial.

Preferences Filename

Filename for the preferences file assigned to the trial. Allows easy distinction when different preferences should be used for different movement trials within one batch analysis.

# Frames

Length of the trial, expressed as the number of frames.

FPS

Video frame rate of the trial, expressed as Frames Per Second (FPS).

Cutoff Freq

GCVSPL Cutoff Frequency that will be used to analyze the trial as selected in the preference file for the trial, expressed in Hz. See details below for how to edit the preferences file to change the Cutoff Freq.

Excluded Cameras

List of cameras that will be excluded when processing the trial, based on the list of excluded cameras in the preferences file.

Tracking BBox

Maximum and minimum 3D corner positions of the tracking bounding box, when applied. Expressed as a set of six values: (min_X, min_Y, min_Z, max_X, max_Y, max_Z), where the minimum corner is located at (origin_X-length/2, origin_Y-width/2, origin_Z-height/2) and the maximum corner is located at (origin_X+length/2, origin_Y+width/2, origin_Z+height/2).

At the bottom of the TMBatch window, the Run Batch button is used to start the batch analysis. After clicking Run Batch, the TMBatch performs a check to identify trials that have already been analyzed. If it finds any, a dialog is shown offering the user to skip or re-run these trials. This allows data to be added to a directory and that directory to be batch processed again without reprocessing completed trials. Once the batch has started, Theia3D will run in the background a full analysis on each checked trial in the list. After the batch has finished, the results of each trial can be viewed in Theia3D by loading the saved workspace belonging to the trial.

Modifying the Trials List

The Trials list can be easily modified using to include or exclude groups of trials to allow portions of the trial list to be left out of the batch analysis. This can be useful if you are only interested in analyzing specific portions of your data, for example only analyzing one action. The easiest way to make modifications to the Trials list is using the Search bar above the Trials list in combination with the selection box next at the top of the Trials list, which selects or deselects all currently displayed trials. This can be performed to exclude specific subsets of trials, or include specific subsets of trials from the batch analysis.

An example procedure for using this technique for modifying the Trials list is as follows:

1. Ensure all trials are currently selected using the ‘select/deselect all’ box at the top of the Trials list.

2. Use a specific search term in the Search bar to display a subset of trials you wish to exclude.

3. Confirm the currently displayed trials are those that you wish to exclude.

4. Use the ‘select/deselect all’ box at the top of the Trials list to deselect these trials.

5. Clear the Search bar.

6. Confirm that the subset of trials is now deselected within the complete Trials list.

Note: The list of trials displayed after using the search bar is not the current batch analysis list, but just a list of the trials from within the batch analysis directory that meet the search criteria.

The preferences file associated with each trial determines the prefences used when that trial is analyzed. This allows you to customize the preferences used across different trials within a single batch analysis, providing greater control over the analysis process. If there are not any preferences .pxt files assigned to the trials within your selected batch analysis root folder, the Preferences column will indicate that the default preferences will be used for your trials.

Right-clicking on the preferences column displays a context menu for the preferences, with the following options:

Edit

Opens the Settings dialog to change the preferences for the current trial.

Copy

Copies the preferences of the current trial to the clipboard.

Paste

Pastes the preferences in the clipboard to the selected trials, or if no trials are selected, to the current trial.

Modifying the Preferences

An example procedure for modifying the preferences assigned to a subset of trials is as follows:

1. Select one of the trials in the subset whose preferences you wish to modify.

2. Right-click on the Preferences column for that trial, and select Edit.

3. Modify the preferences in the Settings window to your desired preferences, and click Save.

4. Right-click on the Preferences column for the same trial, and select Copy.

5. Select the remaining trials in the subset whose preferences you wish to modify, using shift+click to select sequential groups of trials or ctrl+click to select individual trials.

6. Right-click on the Preferences column for one of the currently selected trials, and select Paste.

Number of Cameras

Theia3D requires a minimum of six cameras for tracking; however, we recommend a minimum of eight cameras for most capture volumes. The number of cameras required will increase with movement complexity, capture volume size, capture volume complexity, and the number of people to be tracked. The field of view and joint visibility requirements outlined below are the best guides when determining the number of cameras required for a specific capture volume.

Synchronization and Video Duration

The cameras must capture synchronous videos with identical start times and durations to be used with Theia3D.

Field of View

Person identification and tracking perform best when the people of interest are fully visible and cover a large percentage of the fields of view of the cameras. In most applications a height of 500 pixels is adequate - this requirement may increase based on camera setup, focus, and image quality. It is important that the people of interest are in focus and clearly visible.

Joint Visibility

Ideal tracking conditions are achieved when all joints are visible in all cameras; however, this is rarely attainable due to occlusions from other limbs, people, and the environment. At a minimum, each joint must be visible in at least three cameras. When setting up the cameras it is important to position and orient the cameras in a way that minimizes occlusions while simultaneously providing views of the subject(s) from varying angles to improve joint depth and position calculations. For example, a purely sagittal view results in several occlusions and poor tracking from that view but provides useful information for identify the depth of joints tracked in a more frontal view.

Subject Clothing

Theia3D does not require people to wear specific clothing to be tracked. However, loose or baggy clothing may result in lower quality tracking. As a general rule, if you can easily identify joints in the camera images, then Theia3D can infer the joint positions as well.

Sony RX0 II

Each Sony RX0 II camera includes the following:

• Camera (1)

• Rechargeable Battery Pack (1)

• Micro USB Cable (1)

• AC Adaptor (1)

• Wrist Strap (1)

• Memory Card Protector (1)

• Startup Guide (1)

• Reference Guide (1)

microSD Card

• A Video Speed Class V60 microSD card that exceeds the Class 10 requirement of the cameras and provides increased download speeds.

Sony Camera Control Box

Each Sony Camera Control Box includes the following:

• Camera Control Box (1)

• Multi Terminal Connecting Cable - Short (1)

• Multi Terminal Connecting Cable - Long (1)

• Micro USB Cable (1)

• AC Adaptor (1)

• Cable Protector (1)

• Instruction Manual (1)

• Reference Guide (1)

PoE Network Switch

• 16-Port Gigabit Switch (with power over ethernet)

• The switch provides 1 Gbps connections for each of the cameras and a 10 Gbps connection to the collection computer. When connected to a 10 Gbps NIC card in the collection computer, this provides increased download speeds from the cameras.

• Power Cord

Cables

• 6ft Cat6 Ethernet Cord (1)

• 50ft Cat6 Ethernet Cord (1 per camera)

• Gigabit PoE Splitter (1 per camera)

Additional Hardware

• Male-to-Male 1/4” Thumb Screw (1 per camera)

• Rubber Washer (1 per camera)

The following video provides step-by-step instructions for setting up your Sony RX0 II camera system for the first time. This includes configuring the cameras and control boxes, setting up the multi-camera system, and controlling the system through a web browser.

Camera

1. Remove the camera, battery, memory card protector, and microSD card from the packaging.

2. Open the battery cover and insert the battery into the camera.

3. Remove and detach the memory card/connector cover from the camera.

4. Insert the microSD card into the camera.

5. Attach the memory card protector to the camera.

6. Turn the camera on.

7. Follow the on-screen prompts to set the language, area, date, and time.

9. Set the shoot mode to (Movie) Manual Exposure. This allows you to manually set the Shutter Speed to achieve crisp images without movement blur. Navigate to MENU > Shoot Mode/Drive > Shoot Mode and select (Movie) Manual Exposure.

10. Turn off audio capture. Navigate to MENU > Movie2 > Audio Recording and select Off.

11. Set the auto power off temperature to high. Navigate to MENU > Setup1 > Auto Power OFF Temp and select High.

12. Set the timecode settings. Navigate to MENU > Setup2 > TC/UB Settings > TC Format and select NDF. Navigate to MENU > Setup2 > TC/UB Settings > TC Run and select Free Run. (Note that the NTSC/PAL selector must be set to NTSC for the timecode settings to be available MENU > Setup2 > NTSC/PAL Selector.)

13. Activate remote control. Navigate to MENU > Setup3 > USB Connection and select PC Remote.

14. Turn the camera off.

15. Repeat steps 1-14 for each camera.

PoE Network Switch

1. Remove the switch and power cord from the packaging.

2. Connect the power cord to the switch and plug it in.

3. Remove the 6ft ethernet cord from the packaging.

4. Connect one end of the ethernet cord to a numbered port on the switch.

5. Connect the other end of the ethernet cord to an ethernet port in a dedicated network card in your computer.

Control Box

1. Remove the control box, cable protector, short multi-terminal connecting cable, and a gigabit PoE splitter from the packaging.

2. Remove the back cover from the control box.

3. Pass the thick end of the multi-terminal cable and the male micro USB and Ethernet branches of the PoE splitter through the cable protector.

4. Plug the thick end of the multi-terminal cable into the MULTI port on the control box.

5. Plug the male micro USB branch of the PoE splitter into the DC IN port on the control box.

6. Plug the male ethernet branch of the PoE splitter into the data port on the control box.

7. Attach the cable protector to the control box using its attached thumbscrews.

8. Set the control box MASTER/CLIENT and ON/OFF switches to CLIENT and OFF respectively.

9. Repeat steps 1-8 for each control box. One control box is required per camera.

Connecting the System

1. Start with a camera and control box, each set up as outlined above.

2. Remove a male-to-male 1/4” thumbscrew, a rubber washer, and a 50ft ethernet cord from the packaging.

3. Physically connect the camera to the control box using the thumbscrew and rubber washer. Thread one end of the thumb screw into the hole on the top of the control box. Place the rubber washer over the other end of the thumb screw and thread it into the hole in the bottom of the camera.

4. Plug the thin end of the multi-terminal cable (the thick end is already connected to the control box) into the MULTI port of the camera.

5. Plug one end of the ethernet cord into the female port of the PoE splitter connected to the control box.

6. Plug the other end of the ethernet cord into a numbered port on the switch.

7. Repeat steps 1-6 for each camera and control box pair.

8. Set the MASTER/CLIENT switch of one of the control boxes to MASTER. All other control boxes must be set to CLIENT.

9. Set the ON/OFF switch of all control boxes to ON.

10. Mount the cameras in the collection volume. Note that hardware to mount the cameras (tripods, wall mounts, suction mounts, etc.) is not provided.

Connecting to and Initializing the System

3. Select all control boxes.

4. Open the Box tab of the Control Area and select Update.

5. Browse to and select the CCB-WD1 firmware update file (previously downloaded).

6. Wait for the update to be applied to all selected boxes.

7. A warning stating that the update failed will be shown for any boxes that are already up to date. If the boxes have already been updated, ignore this warning.

8. Once the update is complete, open the Camera tab of the Control Area.

9. With all control boxes selected, turn the cameras on.

10. With all cameras on and selected, link the date/time of all cameras.

11. Open the Box tab of the Control Area.

12. With all cameras/control boxes selected, select Initialize.

Collecting Data with the System

The following video provides step-by-step instructions for setting up and using your cameras to collect data for use with Theia3D. This includes changing camera settings, synchronizing the cameras, recording data, and downloading videos from the cameras through the web interface. This tutorial demonstrates the basic functionality and recommended settings of the Sony camera control interface. Refer to the Sony documentation for information about other settings and more advanced functionality.