home overview research resources outreach & training outreach & training visitors center visitors center search search

Head Tracking via Camera Mouse

Introduction

UCSF Chimera is a modeling application for displaying molecules, volumes and other three-dimensional objects. To manipulate the objects, users move the mouse. However, a natural tendency when viewing 3D objects is to move one's head to see the side of the object (rather than moving the object). So, we are interested in devices for tracking the user location and movements.

Product

Camera Mouse 2007 is a free program that enables you to control the mouse pointer on your computer screen just by moving your head. Head tracking is done by using a standard USB webcam and monitoring the position of a selected pattern, eg user's nose, in the field of view.

Evaluation

Logitech QuickCam Pro 4000 is the only additional hardware that was used in this experiment.

For hardware, we used a Logitech QuickCam Pro 4000. Some new software code was also written for this experiment. First, Chimera was modified to support asymmetric viewing. The line of sight is still perpendicular to the screen (projection plane, in graphics lingo), but it does not need to go through the center of view. Second, a custom extension, HeadTracker, was implemented to adjust Chimera viewing parameters to match the user head position. For example, if the user head location moves rightward, then the line of sight will be adjusted accordingly so that objects in the scene will appear to move leftward.

A severe limitation with using the original Camera Mouse implementation with Chimera is that the user can either use the mouse (controlled by Camera Mouse) for viewing, or use the mouse (controlled manually) for rotating and translating the models. Switching between the two modes required toggling a checkbox in the HeadTracker user interface.

Prof. James Gips and Donald Green were kind enough to modify Camera Mouse so that it produced coordinates via an IP network port instead of modifying the mouse position. The new application, Streaming Camera Mouse 2008, made is possible to update the HeadTracker extension so that the user can both move his head to alter the view, and move the mouse to manipulate the object.

Results

Overall, our approach works, but does not substantially add to the user experience. This is because most users are already manually manipulating the models in Chimera, and a simple drag of the mouse will provide the views generated by head motion. As expected, head tracking is most effective when viewing a static scene. Head tracking using Camera Mouse does provide a low-cost (but relatively low-performance) alternative to stereo viewing hardware such as Crystal Eyes.

We discovered several limitations during our experiment.

• The head tracking approach is not appropriate when more than one user is present since the displayed view reflects the scene as seen by the tracked user.
• Camera Mouse is primarily developed to help people with disabilities and therefore is very good at tracking relatively small head motions rather than large head movements. For our use, we expect large movements and Camera Mouse occasionally loses track of the head position. In addition, Camera Mouse only tracks the horizontal and vertical positions from the camera but not the distance from the camera. Thus, we cannot alter the view when the user moves either closer to or farther from the camera.
• Finally, Camera Mouse is available only for Microsoft Windows, and not for either Linux or Mac OS X.

Laboratory Overview | Research | Outreach & Training | Available Resources | Visitors Center | Search