3D Selection Vault

Gaze-Hand Alignment

Also known as Gaze&Finger, Gaze&Hand

Selection by aligning a hand-controlled pointer with the gaze ray; the target is selected when the two line up.

How it works

Gaze pre-selects the object being looked at, and the selection is triggered when a hand-controlled pointer is brought into alignment with it, so no separate click is needed. Gaze&Finger casts a ray from between the eyes over the raised index fingertip (image-plane pointing), and selection happens when that ray crosses the gaze-pre-selected target. Gaze&Hand opens the menu by gaze dwell and then moves a cursor with relative hand movement onto the gazed item. In an AR menu study (HoloLens 2) both gaze-assisted techniques and Gaze&Pinch beat a hands-only Point&Pinch baseline; Gaze&Pinch was fastest to open menus, Gaze&Finger most efficient for item selection, and Gaze&Hand least physically demanding but most error-prone.

Papers

• Mathias N. Lystbæk, Peter Rosenberg, Ken Pfeuffer, Jens Emil Grønbæk, Hans Gellersen (2022). Gaze-Hand Alignment: Combining Eye Gaze and Mid-Air Pointing for Interacting with Menus in Augmented Reality. Proceedings of the ACM on Human-Computer Interaction 6 (ETRA), Article 145 doi:10.1145/3530886
• Uta Wagner, Mathias N. Lystbæk, Pavel Manakhov, Jens Emil Sloth Grønbæk, Ken Pfeuffer, Hans Gellersen (2023). A Fitts' Law Study of Gaze-Hand Alignment for Selection in 3D User Interfaces. CHI Conference on Human Factors in Computing Systems doi:10.1145/3544548.3581423
• Mohammadreza Amini, Wolfgang Stuerzlinger, Robert J. Teather, Anil Ufuk Batmaz (2025). A Systematic Review of Fitts' Law in 3D Extended Reality. CHI Conference on Human Factors in Computing Systems doi:10.1145/3706598.3713623