ChimeraX docs icon

ChimeraX Virtual Reality

Warning: ChimeraX virtual reality (VR) capabilities have many shortcomings. A few years of development effort will be needed to make these features work well, and only a small fraction of that work has been done.
using HTC Vive VR system

ChimeraX allows display and analysis of structures and density maps using virtual reality headsets such as HTC Vive, Vive Pro, Oculus Rift, Samsung Odyssey and Windows Mixed Reality. For multi-person VR sessions, see the meeting command. ChimeraX can also record 360° movies.

See also the VR at UCSF website and the following reference:
Molecular visualization on the holodeck.
Goddard TD, Brilliant AA, Skillman TL, Vergenz S, Tyrwhitt-Drake J, Meng EC, Ferrin TE.
J Mol Biol. 2018 Oct 19;430(21):3982-3996.

Starting VR Mode

Any models displayed in ChimeraX can be shown in virtual reality headsets supported by SteamVR, such as HTC Vive, Oculus Rift and Samsung Odyssey headsets. Descriptions below are for Vive hand controllers, but different hand controllers have buttons, triggers, etc. that are generally similar and function similarly.

  1. start SteamVR
  2. start ChimeraX and open some data, for example, a PDB model:
    open 2gbp
  3. enter the ChimeraX command:
    vr on

Models in ChimeraX should then appear in the VR headset. With the hand controllers (positions shown as colored cones in the headset):

The physical room bounds are shown as a blue mesh when the headset or a hand controller comes close to a physical wall (within about a meter). Models can be scaled to sizes larger than the physical room.

To turn off rendering in the VR headset and re-enable rendering on the desktop display, use the ChimeraX command:

vr off

Hand Controller Modes

Most of the ChimeraX mouse modes can be used with the VR hand controllers. To assign a mouse mode to a hand controller button:

  1. press the Vive menu button to show the ChimeraX desktop user interface (by default, including the Mouse Modes toolbar) in the headset. Holding the menu button allows moving the panel in the VR scene.
  2. put the tip of a hand controller cone on a mouse mode icon and click with trigger, trackpad, or grip hand controller buttons to assign that mouse mode to the clicked button

There are three assignable buttons on each controller, so six modes can be assigned. However, keeping the default trigger assignment (rotation or translation, equivalent in VR) is recommended to allow manipulation as described above.

Functions include:

Modes that require clicking on an atom such as selection or labeling require pointing at the atom with the hand controller cone. This is easiest to do at close range, although it can work at a distance but requires accurate pointing. Modes that involve dragging on the desktop such as changing the contour level of a map or zooming are operated by clicking and moving the hand controller vertically. The bond rotation mouse mode operates by dragging on the desktop, but in VR operates by rotating the hand controller. Translation and rotation are separate mouse modes but are a single hand controller mode.

Hand Controller Problems

Several factors make it difficult to use the hand controllers. These problems can be remedied but we have not yet had time to fix them.

No click feedback. There is no visual indication when you click a mouse mode icon. The icons are small and easy to miss. It helps to move the panel close by holding the menu button on the hand controller. The Log will show if the mode was changed.

No mode indicator. There is no visual indication of what modes are currently assigned to the buttons. It is easy to forget the six modes assigned to the two controllers. In the future we plan to put the mode icon on the cone at the location of each of the three assignable buttons.

Blackout. Modes that perform calculations taking longer than 1/90th second cause the graphics not to update, and the VR scene will gray out after a fraction of a second without graphics updates. Changing map contour level and running molecular dynamics are prone to this problem. In the future these calculations will be done in parallel so the graphics will not be slowed down.

Flickering labels. Atom labels or residue labels crated by the swap amino acid mode can cause scene flicker because the labels are slow to render. Graphics optimization is needed.

Selection outlines. Green selection outlines can slow rendering and cause flicker.

Limitations of Interactive Viewing

Motion sickness warning: If the model is stuttering or flickering in the headset (due to insufficient rendering speed), then you should stop viewing it, as this can easily and suddenly induce nausea that could last as long as 24 hours. Even without the stuttering, if you feel any motion sickness, stop using the headset. Progression from slight nausea to needing to throw up can happen quickly.

Operating systems. We have tested ChimeraX VR on Windows 10 with an Nvidia Geforce GTX 1080 graphics card and with an AMD Radeon RX 480 graphics card. As of November 2018, SteamVR works well on Windows on Windows 10, has some support on macOS High Sierra and Mojave, and can work on Linux.

Small models only. Large molecular structures (more than a few thousand atoms) render too slowly and will cause stuttering in the headset. Large density map surfaces also cause stuttering; a 2003 map at full resolution should work, but larger sizes may require subsampling (step 2). Current headsets render at 90 frames per second in two eyes, so the effective rendering speed needed to avoid flickering is about 200 frames per second. By comparison, a rendering speed of 10 frames per second is adequate on a conventional desktop display.

Simple lighting. By default ChimeraX switches to simple lighting when VR is enabled -- other lighting modes render too slowly and cause stuttering.

Desktop rendering. The ChimeraX graphics on your normal computer display will show what the user sees in the VR headset. Rendering to the conventional display (generally 60 frames per second) can slow the VR rendering if ChimeraX is not able to switch to a mode where graphics is not synchronized to vertical refresh. To test if this is the cause of stuttering you can use the vr command option display blank to prevent graphics from updating on the desktop display. An alternative mirroring can be done by the SteamVR runtime using SteamVR menu entry “Mirror Display.”

Side View should be closed. The ChimeraX Side View tool will slow headset rendering and cause stuttering, so it should be closed during VR viewing.

Difficult to change model display. Changing molecule or map display can be done by displaying the desktop in the headset using the Vive controller menu button. Then use the hand controllers to “click” icons in the ChimeraX Molecule Display and Density Maps toolbars to change the display. Alternatively the display can be changed by removing the headset and using the ChimeraX interface on the conventional display with mouse and keyboard.

Clipping when models far away. If models are scaled to a very large size, parts more than 500 meters away (in physical room dimensions) will be clipped. This may seem like a large distance, but as a model is scaled up it may appear to stop increasing in size because stereoscopic depth perception is not effective beyond about 10 meters. As parts of the model get both larger and farther away at the same rate, the change in size is not evident. To gauge model size, it can be useful to move a hand controller near a physical wall so that the blue mesh will provide a reference distance.

Dynamic scenes may render too slowly. Dynamic (time-varying) scenes are more interesting to view than static scenes. We have successfully displayed 3D optical microscopy time series of crawling cells using the command vseries play with the cacheFrames option so that all surface depictions are precomputed; without that option, the VR headset rendering is too slow. It may also help to change the scene at a slower pace, for example using vseries play with the maxFrameRate option. A molecular dynamics trajectories can be played with the coordset command. Use the coordset pauseFrames 9 option to slow the playback rate from the 90 frames per second of the headset to 10 frames per second.

SteamVR room boundaries wrong. Sometimes tracing room boundaries using room-scale room setup in SteamVR produces incorrect results (November 2018). We have observed this when the traced space is not a simple rectangle because of a desk against one wall. Tracing around the desk joins the 6 corners of the bounds in the wrong order -- a different order than they were traced. It is unclear what is wrong with the SteamVR room setup algorithm. But we have found that clicking the room tracing "Advanced Mode" checkbutton which allows you to click at each corner of the room reliably produces the correct SteamVR bounds.

UCSF Resource for Biocomputing, Visualization, and Informatics / December 2018