Faizan Ur Rehman - Towards a Framework for Multiscale Social Event Extraction and Visualization

The population in cities is slated to double by mid-century according to estimates prepared by the World Health Organization. This rapid increase in population will impact transportation and economic growth, and will increase responsibilities of local managing authorities and different stakeholders. It is a need of the hour to convert cities into smart cities in order to provide new service to the public, by using available resources in an optimum manner. From crowd-sourced data and open governmental data to other online sources, a variety of data sources can provide users with smart tools to efficiently manage their daily activities. Moreover, with the advancement in Internet and mobile technologies, social networking platforms such as Facebook and Twitter have become popular modes of communication. They allow users to share a spectrum of information, including spatio-temporal data, both publicly and within their community of interest in real-time. Scrutinizing knowledge from different types of available, rich, geo-tagged, and crowd-sourced data and incorporating it on a map has become more feasible. This presents a real opportunity to enrich traditional maps and enhance conventional spatio-temporal queries with the help of different types of data extracted from a variety of available data sources. In this thesis, we first propose a constraint-aware route recommendation system in lack of physical infrastructure environment that leverages geo-tagged data in social media and user-generated content to identify upcoming traffic constraints and, thus, recommend an optimized path. We have designed and developed a system using a spatial grid index to inform users about upcoming constraints and calculate a new, optimized path in minimal response time. Later, the concept of “smart maps” will be introduced by collecting, managing, and integrating heterogeneous data sources in order to infer relevant knowledge-based layers. Unlike conventional maps, smart maps extract information about live events (e.g., concert, competition, incidents, etc.), online offers, and statistical analysis (e.g., dangerous areas) by encapsulating incoming semi- and un-structured data into structured generic packets. This methodology sets the ground for providing different intelligent services and applications. Moreover, developing smart maps requires an efficient and scalable processing and the visualization of knowledge-based layers at multiple map scales, thus allowing a smooth and clutter-free browsing experience. Finally, we introduce Hadath, a scalable and efficient system that extracts social events from a variety of unstructured data streams. Hadath applies natural language processing and multi-dimensional clustering techniques to extract relevant events of interest at different map scales, and to infer the spatio-temporal extent of detected events. The system comprises a data wrapping component which digests different types of data sources, and prepossesses data to generate structured data packets out of unstructured streams. Hadath also implements a hierarchical in-memory spatio-temporal indexing scheme to allow efficient and scalable access to raw data, as well as to extracted clusters of events. Initially, data packets are processed to discover events at a local scale, then, the proper spatio-temporal extent and the significance of detected events at a global scale is determined. As a result, live events can be displayed at different spatio-temporal resolutions, thus allowing a smooth and unique browsing experience. Finally, to validate our proposed system, we conducted experiments on real-world data streams. The final output of our system named Hadath creates a unique and dynamic map browsing experience.