Context-Sensitive Modeling of Metaphoric Potential with Large Language Models
Shihui Li

Abstract:
Psycholinguistic variables such as concreteness, imageability, familiarity, and iconicity are commonly considered to influence a word’s metaphorical usages. However, existing studies usually rely on static lemma-level norms, although metaphorical meaning is strongly context-dependent. Recent work suggests that Large Language Models (LLMs) can approximate human psycholinguistic ratings, making them a possible tool for generating context-sensitive estimates at scale.
In this thesis, we construct a human-annotated dataset of in-context ratings for the four variables using metaphorical and literal sentences from the VU Amsterdam Metaphor Corpus. We then evaluate four LLMs on both isolated and contextual ratings, and use lexical and LLM-generated contextual features to model metaphoric potential. We find that GPT-4o achieves the strongest agreement with human judgments, while performance is higher for concreteness and imageability than for familiarity and iconicity. Model-human agreement also improves for items with more consistent human ratings.
In metaphor modelling, contextual and lexical ratings provide complementary information. Contextual ratings generally outperform lexical ratings alone, while combining both yields the best overall performance and generalization. Contextual ratings capture how psycholinguistic properties are realized in a particular sentence: odds ratio analyses show that concreteness decreases with a shift towards metaphorical meaning, whereas imageability exhibits the opposite pattern. In stepwise regression, familiarity emerges as the strongest predictor.
Overall, our dataset provides a resource for studying contextual variation in psycholinguistic properties and evaluating LLMs’ ability to approximate human lexical judgments. The thesis also provides corpus-based evidence for cognitive accounts of metaphor and contributes a contextual psycholinguistic dataset at the intersection of NLP and psycholinguistics.