Ky Trung Nguyen - Story Generation from Smartphones Data: A script approach

Script is a structure describing an appropriate sequence of events or actions in our daily life. Stories built on scripts with several possible deviations, which allows us to deeper understand what has happened in the routine behavior of our daily life. Therefore, it is essential in many ambient intelligence applications such as health monitoring and emergency services. Fortunately, in recent years, with the advancement of sensing technologies and embedded systems, it has become possible to collect activities of human beings continuously, by integrating sensors into wearable devices (e.g., smart-phone, smart-watch, etc.). Hence, human activity recognition (HAR) has become a hot topic interest in research over the past decades. In order to do HAR, most researches used machine learning approaches such as Neural networks, Bayesian networks, etc. Therefore, the ultimate goal of our thesis is to generate such kind of stories or scripts from activity data of wearable sensors using a machine learning approach. However, to the best of our knowledge, it is not a trivial task due to the large semantic gaps between a story and raw wearable sensors data. Hence, there is still no approach to generate script/story using machine learning, even though many machine learning approaches were proposed for HAR in recent years (e.g., convolutional neural network, deep neural network, etc.) to enhance the activity recognition accuracy. In order to achieve our goal, first of all in this thesis we proposed a novel framework, which addressed for the problem of imbalanced data, based on active learning combined with oversampling technique so as to enhance the recognition accuracy of conventional machine learning models i.e., Multilayer Perceptron. Secondly, we introduce a novel scheme to automatically generate scripts from wearable sensor human activity data using deep learning models, evaluate the generated method performance. Finally, we proposed a neural event embedding approach that is able to benefit from semantic and syntactic information about the textual context of events. The approach is able to learn the stereotypical order of events from sets of narratives describing typical situations of everyday life.