ChimeraX docs icon

ChimeraX Tutorial: DICOM in Virtual Reality

Sample Data
Manipulation Outside of VR
Manipulation in VR
Viewing Planes
Cropping and Slabbing
Windowing

This tutorial covers basics of using ChimeraX virtual reality to view DICOM medical imaging data. It assumes you have the necessary equipment for virtual reality already installed and set up for the room. ChimeraX works with virtual reality (VR) systems supported by SteamVR, such as HTC Vive, Oculus Rift and Samsung Odyssey headsets. Descriptions below are for Vive hand controllers, but other hand controllers have buttons that are generally similar in position and function.

Other than the manipulation in VR, however, the tutorial can be done without VR just using your computer and mouse (or trackpad) instead of a headset and hand controllers.

See a video of this tutorial (Feb 2019). See also: ChimeraX DICOM Reference, ChimeraX tutorials, ChimeraX for Medical Image Analysis

Sample Data

4-24533.zip – one chest scan of many in the RIDER Lung CT collection from the Cancer Imaging Archive:

Clark K, Vendt B, Smith K, et al. The Cancer Imaging Archive (TCIA): Maintaining and Operating a Public Information Repository. Journal of Digital Imaging. 2013; 26(6): 1045-1057.

The data have been anonymized and are freely available. This sample data folder should be downloaded and unzipped. Although this tutorial does not include clinical analysis, a significant feature of the scan is a tumor in the right lung.

screenshot 1

Initial Display and Manipulation Outside of VR

Start ChimeraX by clicking its icon. After the ChimeraX window appears, open the sample data:

Menu: File... Open DICOM Folder (dialog shows Format: DICOM image), browse to location, select directory, click Choose

Just one Z-plane is shown initially. The Volume Viewer tool shows a histogram of the data with plane as the chosen display style. Below the histogram is a slider for viewing different planes along Z. Try dragging the slider. The tumor in the right lung can be seen at approximately Z-planes 146-185.

The small squares connected by a yellow line on the histogram are thresholds, the control points for mapping values to colors and intensities. They can be moved by dragging in the Volume Viewer histogram, and added and deleted using the context menu (right-click or Ctrl-click depending on platform).

Try moving the model around:

rotate
left mouse button
trackpad click-drag
translate in 2D
middle or right mouse button
trackpad + Alt
zoom
mouse scroll wheel
trackpad 2-finger drag (except pinch on Mac)

Some useful commands for viewing DICOM data outside of VR:

To turn off perspective, which makes closer parts appear larger on the screen: camera ortho
To center and scale the display to fit in the graphics window: view     ...or click Graphics icon
To center and scale as above, plus reset to initial orientation: view orient     ...or click Graphics icon

Manipulation in VR

In virtual reality, the hand controllers are used to click icons and move models. There is currently no virtual keyboard in VR for typing commands.

The icon Toolbar across the top of the ChimeraX window is organized into several tabs. Icons in the Map tab act on volume models:

Vive hand controllers
click image to enlarge, or see diagram at Vive website

Icons in the in the following tabs of the Toolbar assign functions or “modes” to hand-controller buttons:

SteamVR status window
The SteamVR status window shows icons for the goggles, two controllers, and two base stations in green when they are detected as ready for use.

Take a moment to look at the hand controllers: Vive controllers each have a trigger to “pull” with the index finger, a grip button on the side, and a larger round trackpad on the top surface that can be pressed with the thumb. Above the trackpad is a smaller menu button marked with horizontal lines, and below it is a power button and a green light indicating when the controller is turned on. Buttons on other controllers are similar.

Start VR:

Command: vr on

(SteamVR starts automatically, providing you had previously signed in to it from the same computer and chose to have your password remembered.)

On the screen, you can see that the icon bars have been moved to the right side of the graphics window. In the process, other tool panels may have been vertically compressed; drag to make the Volume Viewer panel tall enough to show the full height of the histogram.

Put on the headset, and try moving the model. The hand-controller positions are shown in the headset as cones, and icons on the cones indicate button assignments.

rotate
 rotate hand controller with trigger pressed underside of cone

The icon on the cone “underside” shows that the trigger is assigned to translation (and rotation), and the icon on the side shows that the grip button is assigned to center and rescale.
translate in 3D
 move hand controller with trigger pressed
zoom
 move hand controllers farther apart or closer together with both triggers pressed
– or –
move hand controller vertically with trackpad pressed
center and rescale
(recover the view)
 click grip button
show, hide, move control panel  use menu button

Continue moving the model as you wish throughout the tutorial.

It is important to avoid flickering in the VR headset, as this may cause severe and long-lasting nausea. Flickering generally indicates that rendering is too slow to keep up with head movements or other changes in the scene. Rendering planes or a cropped volume is faster than rendering the full volume. Another way to decrease computational demands is to subsample the data; clicking    changes to step 2 (using half as many points in each dimension), whereas clicking    returns to full resolution. Step size may increase automatically when the amount of data displayed increases, but it can still be adjusted manually.

Click the menu button to show the ChimeraX control panel in the headset. The control panel is just the set of tool windows to the right of the graphics window in the desktop view, including icon bars. In VR, it can be moved around with the menu button pressed, or hidden by just clicking the button again.

A Map icon can be clicked with any controller button (trigger, trackpad, or grip) to give the same result. Icons in this tutorial are Map icons except where stated otherwise.

Clicking a Right Mouse icon with a trigger, trackpad, or grip button assigns whichever controller button was clicked to the corresponding function.

An icon will pop up slightly when the cone tip is in position to click it.

Leaving triggers with their default movement modes (rotation, translation, zooming) is recommended, but up to four other functions can be assigned, to the grip and trackpad on each controller. If a trigger is accidentally reassigned, using it to click either the rotate    or translate    Right Mouse icon returns it to the default behavior.

Viewing Planes

A single Z-plane is displayed. To view different planes along Z:

To view planes along X, Y, or Z:

screenshot 2
Desktop view including the VR scene in the graphics window: orthoplanes, control panel, and cones with icons for current mode assignments.

To view planes along X, Y, and Z at the same time:

Cropping and Slabbing

With 3D display, cropping is useful for hiding data that may obscure or distract from a region of interest.

The 3D presets are based on definitions in the Horos program for medical image visualization and analysis. Several are available from the command line, but only “Airways II” is currently available as an icon. If you later go back to viewing planes or thin slabs, it may be helpful to restore the initial thresholds by clicking .

screenshot 3
VR scene during cropping.

Similar to planes, you can move a slab along X, Y, or Z:

Windowing

A 3D preset like “Airways II” sets the thresholds to specific values. They can also be adjusted continuously by hand:

If you want to return to the “Airways II” threshold settings, click . For settings better suited to planes or thin slabs, click .


UCSF Resource for Biocomputing, Visualization, and Informatics / July 2019