Adapting Interaction Based on Users' Visual Attention

Abstract

Successful interaction with many information systems depends on our ability to visually attend to the system feedback as well as to our own actions. However, at a given time, we are able to attend to only a portion of the available information. Among other constraints, what can we attend is limited by the spatial acuity of our eyes. Aware of this limitation, researchers have long pursued interfaces that decrease our dependence on visual attention during interaction. The newly proliferating sensing technologies such as eye and head tracking as well as methods for user modeling provide a novel venue for addressing this limitation: An information system can utilize users' visual attention information to change how it responds to user actions. This thesis contributes design knowledge about adapting the interaction based on users' level of visual monitoring during input through a series of prototypes that have been developed for different use cases. I first distinguish between different implications of visual attention information for interface design, and identify visual attention as a measure of user awareness as the main focus of the work presented in this thesis. Lack of visual attention during input decreases users' awareness of the environment. In these cases, the system can adapt the interaction through a number of methods such as handling input more flexibly or remediating the lack of visual attention through novel visual feedback techniques. These interaction methods have been formulated as part of a constructive research program and applied to single-user applications that require users to split their visual attention between multiple interface regions during pointing and also to collocated and synchronous multi-user applications. User studies provide evidence for the increased uncertainty during input with low visual attention and also show in which situations these interaction techniques can improve performance. The dissertation discusses these empirical findings in terms of the previously identified trade-offs between time and spatial multiplexing, and between predictability and adaptiveness in interface design. The thesis also makes a theoretical contribution to the general design challenge of building adaptive or context aware systems through an analysis of the concept implicit interaction. Overall, the thesis contributes to the existing line of work on attentive interfaces by developing interaction methods that specifically target handling user input with low visual attention, and contributes to the ongoing discussions about the integration of eye tracking into human–computer interaction.