3D Selection Vault

GG Interaction

Also known as Gaze–grasp pose interaction

Look at the object and shape your hand as if to grasp it; the object whose width best matches your grasp, among those near your gaze line, is selected.

How it works

Gaze forms a candidate group from all objects within a threshold distance of the gaze line, updated every frame. The hand's grasp pose is measured as the smallest distance between the thumb and the other fingertips and compared with each candidate's width. Each candidate gets a cost combining its gaze error and grasp error, and the lowest-cost object is selected once it falls below a threshold. Because selection is triggered by the grasp itself, it avoids the eye-hand visibility mismatch of ray-casting and can pick partly overlapped objects that differ in size. Evaluated against ray-casting (n=20) in a toy-block manipulation task and a 3D reciprocal tapping task, on a 40-inch stereoscopic monitor viewed from about 70 cm, with a Tobii Rex eye tracker and an RGBD sensor for hand tracking.

Papers