11 January 2022, Computational Linguistics Seminar, Arthur Bražinskas

Opinion summarization is the automatic creation of text reflecting subjective information expressed in multiple documents, such as user reviews of a product. These short summaries can help users make better purchasing decisions by condensing useful information in hundreds or even thousands of reviews. However, due to the high cost of summary production, datasets large enough for supervised learning were absent until recently. This lead to a variety of extractive methods that construct summaries from review sentences. However, these methods often produce incoherent summaries with unimportant details. This presentation will focus on abstractive approaches that generate summaries using a free vocabulary and thus can yield more coherent texts. We will discuss summarizers trained in unsupervised, few-shot, and supervised regimes. These models combine principles of latent probabilistic models, variational inference, and reinforcement learning. In our unsupervised model (Copycat), we treat the product and review representations as latent continuous variables. At test time, we induce summarizing representations and map them to summarizing texts. In the supervised model (SelSum), we decompose the system into a selector (posterior) and summarizer. The selector treats reviews as latent categorical variables and selects a summary-relevant subset in training. Only the small subset is passed to the summarizer, which results in computational and memory savings. The system is trained end-to-end using variational inference and reinforcement learning. Finally, we fit another selector (prior) that selects subsets of informative reviews to summarize in test time.