Modeling Users Interacting with Smart Devices
Seyyed Hadi Hashemi

Abstract:
Personalizing users’ experience and the ability to perform complex tasks on smart devices and environments such as smart speakers and smart homes are changing the way people are doing their daily tasks. Checking the weather and planning to visit a museum is as simple as asking your smart speaker at home to read out loud the weather condition and commanding the Intelligent Assistant (IA) integrated with the smart speaker to book a ticket to visit the museum. To improve user experience in physical spaces such as smart homes, museums, and cities while performing their daily tasks, effective modeling of users interacting with smart devices is required.
The overall goal of this thesis is to improve users’ experience in physical spaces such as smart cities and environments by modeling user interactions with smart devices. In Chapter 2, we model users’ behavior in interacting with smart devices in a smart museum to recommend unseen archaeological objects to visit without asking users to provide any information about their preferences. To understand users’ preferences, we have studied both users’ onsite physical interaction behavior in the physical space and their online digital interaction behavior at the search engine of the museum. We found similarities and differences in users’ online and onsite interaction behaviors, which leads to incorporating both online digital and onsite physical user interactions in training an effective point of interest recommender system for a smart museum.
In Chapter 3 and Chapter 4 of the thesis, we focus on creating and maintaining reusable test collections for the evaluation of contextual suggestion systems to rank tourist attractions for users in a smart city context. Creating and maintaining a reusable test collection for offline evaluation of personalized contextual suggestion systems is challenging due to the personalized and dynamic nature of the test collection. However, personalization is an important aspect of contextual suggestion systems as it can have a direct impact on the user experience. Thus, we create a reusable test collection for the evaluation of personalized contextual suggestion systems and proposed an approach for maintaining the reusability of dynamic test collections.
Furthermore, to measure how satisfied users are in using smart devices such as smart speakers in their smart homes, Chapter 5 of this thesis is allocated to identifying tasks and sessions on smart speaker IAs using a time-oriented approach in analyzing smart speaker IA interaction logs. Then, Chapter 6 details our proposed user satisfaction prediction model on smart speaker IAs to measure user satisfaction while performing a task to improve users’ experience in using smart speakers. In this thesis, we show how different contextual factors such as the learning curve of users have impacts on users’ behavior on smart speaker IAs that lead to different task and session boundaries estimation on different contextual situations. Furthermore, we propose users’ utterance intent as a signal to measure user satisfaction on smart speaker IAs and show how incorporating users’ intent in query representation learning can improve a user satisfaction prediction model and consequently users’ experience with smart speakers.