Julian Andres Galindo Losada - UI adaptation by using user emotions; Application to GUI personalization by facial expression depending on age and gender

User interfaces can be adapted to the user's context of use, i.e. to their characteristics, their interaction device and their environment (brightness, noise, etc.). This thesis focuses on the adaptation of interfaces through user's emotions at interacting with the system. A software architecture has been designed to adapt the user interface thanks to emotions and characteristics of the user (age and gender). This thesis focuses on how to infer a usability or aesthetic problem which has been encountered by the user. It is based on an experiment verifying that: 1) existing tools give similar results in terms of detected emotions which 2) seem to be well influenced by the aesthetics and usability of an interface relying on age and sex. Consequently, these results allow the inference of user interface problems.

Our approach aims to personalize user interfaces with user emotions at run-time. An architecture, Perso2U (Personalize to You), has been designed to adapt the UI according to emotions and user characteristics (age and gender). In order to validate our approach, we conduced an experiment that we analysed according to 3 points of view. The 1st analysis makes it possible to establish that the emotions detected by the facial recognition tools provide similar emotion values with a high emotion detection similarity. A second experimental analysis tends to show that: (1) UI quality factors (aesthetics and/or usability) influences user emotions differently based on age and gender, (2) the level (high and/or low) of UI quality factors seem to impact emotions differently based on age and gender. From these results, we define thresholds based on age and gender that allow the system to infer usability and/or aesthetics problems.