Patrick Perea - Mobile augmented reality interaction: digital objects exploration and pointing

This thesis contributes to the research field of Human-Computer Interaction (HCI). The focus of the research is on user interaction with handheld Augmented Reality (AR) systems.

AR allows the addition of digital content, primarily graphics, to the user's physical environment. The resulting mixed environment includes digital objects registered in the physical world. This mixed environment, partially visible on the mobile device's screen, defines several constraints for interaction. Our work is dedicated to the selection of a digital target in this mixed environment and we address two questions: (1) how to improve the perception of the mixed environment beyond what is perceived by the camera's field of view of the mobile device, for finding off-screen digital targets (2) how to enhance pointing at a digital target registered in the physical environment. Our contributions answer these two questions.

We first propose three interactive off-screen guidance techniques, which we evaluate by conducting two laboratory experiments. These techniques are based on Halo, a visualization technique of off-screen objects based on circles, and differ in the way the aggregation of the off-screen objects is displayed on screen. The results of the two experiments show that our three techniques effectively extend users' knowledge of the mixed environment, and limit the visual intrusion on the mobile device's screen in comparison with the traditional arrow-based visualization technique.

We then define two interaction techniques enhancing the selection of digital targets. These techniques are based on (1) target expansion techniques, which facilitate target selection by allocating a larger active area to each target and (2) a cursor jump, which shortens the distance between the cursor and the content of the digital target. The two techniques differ in the way users manipulate the cursor once it has jumped to the digital target. We propose two possibilities: physical pointing by moving the mobile device to the desired content, or relative pointing using thumb strokes on the screen. The result of a laboratory experiment confirms that target expansion techniques improve the pointing performance, while relative pointing is preferred by participants.

Our contributions are applied to the field of industrial maintenance, in charge of repairing or preventing failures on production machines. As part of a Schneider Electric-CIFRE thesis, the Schneider Electric's Augmented Operator Advisor product (AR maintenance assistance application) includes one of the proposed targeting assistance techniques.