FocalPoint
Also known as Adaptive selection cylinder
In smartphone AR with the free hand reaching into the scene, a selection cylinder cast from the screen adapts to where on the screen the user tends to select, a plane attached to the hand makes everything in front see-through, and the best-scoring object snaps to the pinch.
How it works
The phone is held in one hand and the other hand is tracked (Leap Motion on the phone, or MediaPipe). The focal region is a circle fitted to the screen positions of the last ten selections and extruded from the screen into a cylinder; candidates within twice its radius are scored by their screen distance to the focal centre and their distance to the hand, and the best one is outlined in red. A pinch within 3 mm snaps it to the hand. An occlusion plane 3 mm behind the hand makes objects in front of it transparent and unselectable, and object opacity encodes depth. With 12 participants selecting 3 mm cubes from piles, accuracy was 70% against 24% for a direct-manipulation baseline and selection was 5.5 s faster.
Papers
- Jiaju Ma, Jing Qian, Tongyu Zhou, Jeff Huang (2023). FocalPoint: Adaptive Direct Manipulation for Selecting Small 3D Virtual Objects. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 7(1), Article 22 doi:10.1145/3580856