Neural Models of Language Use: Studies of Language Comprehension and Production in Context
Mario Giulianelli

Abstract:
This thesis explores novel ways of using artificial neural networks as models of human language use, with the goal of establishing new methods and enabling new research directions for a wide variety of language scientists: from historical linguists, sociolinguists, and lexicographers to cognitive scientists and neuroscientists. It consists of a series of studies on language comprehension and language production. with an emphasis on how their modelling is affected when linguistic contexts are properly taken into account. It is divided into three parts.

Part 1 presents two novel methods to study word usage as a function of a word's sentential context of occurrence: the first consists of extracting, grouping, and analysing contextualised neural representations from language models; the second uses human-readable word definitions generated by language models prompted with word usage examples. Lexical semantic change analysis is taken as an example application, as it requires dynamically capturing word meaning with its nuanced context-determined modulations.

Part 2 focuses on neural models as contextually-aware simulations of language comprehenders. I obtain surprisal estimates of information rate from neural language models and use these to test psycholinguistic theories of utterance production, which postulate speaker monitoring of information rate and, in turn, of comprehension costs. Findings challenge established hypotheses of rational use of the communication channel, especially in dialogic settings---but, overall, they confirm that strategies of utterance production can be described as efficiently containing the comprehension effort of interlocutors.

Part 3 investigates the potential of neural text generators as models of language production. I test whether generators produce language with statistical properties aligned to those of human productions, and then use them to obtain interpretable measures of information rate which are complementary to those used in Part 2. I conclude by collecting insights from the rest of the thesis into a formal framework for artificial simulations of human-like---efficient, communicatively effective, and audience-aware---language production behaviour.