Adjunction in Hierarchical Phrase-Based Translation
Sophie Arnoult

Abstract:
Hierarchical Phrase-Based SMT models are compositional by reliance on a formal, Synchronous Context-Free Grammar. There is however no guarantee that target-side translations are compositional. Linguistic enrichment methods often exploit syntactic cues for better target rewritings or source-side rule selection, but these cues generally describe one side of the data, and have in turn little bearing on compositional translation equivalence.

Besides, while reliance on lexicalized rules allows for modelling complex reorderings, unconstrained extraction of these rules potentially leads to very large grammars. This risk is normally circumvented by applying span-length constraints, under the assumption that short-range rules capture the most useful translation relations. These constraints naturally limit the expressiveness of hierarchical phrase-based models in the long range. Structuring the reordering space remains an open problem for these models and for SMT in general. 

This dissertation takes adjunction as a source of linguistic information for hierarchical phrase-based models. Adjunction is regarded as the main driver of recursion in Tree-Adjoining Grammar (Joshi et al., 1975; Joshi and Schabes, 1997), allowing to explain linguistic phenomena like wh-fronting without resorting to transformations and, generally, to abstract syntactic modification away. Synchronous Tree-Adjoining Grammar proposes to view adjunction as applying synchronously, thus driving recursion in translation. While synchronous adjunction has been applied in Syntax-Based SMT, it has been left unexploited in Hierarchical Phrase-based SMT, possibly because of the apparent contradiction between the formal character of adjunction and the asyntactic nature of phrase-based models.   

This dissertation considers the constituents involved in adjunction--adjuncts, in the broad sense of syntactic modifiers--and investigates which part they play in compositional, phrase-based translation. This dissertation makes the following contributions:

- a corpus study of adjunct alignment in French-English data, to assess the degree to which adjunction is synchronous in translation data. This study further relates gold measures of synchronous adjunction to empirical measures based on word alignments and parse-based adjunct-identification heuristics. 
- an extension to Hiero (Chiang, 2005), where adjuncts are utilized to guide extraction, by fully relaxing span constraints around adjuncts. We further exploit adjunct optionality by factoring out adjuncts at rule extraction. 
- an extension to the latent reordering PCFG grammar of (Stanojevic and Sima'an, 2015) for translation preordering, where adjuncts are utilized to inform reordering.    

Our corpus study confirms that adjuncts are synchronous to a high degree in translation data, at least for English-French. We show that synchronous adjunction is not solely indebted to syntactic similarity however, but also has a semantic basis and reflects translation compositionality.

The proposed extension to Hiero shows that adjuncts are useful for driving recursion in hierarchical phrase-based models, as adjunct-crossing constraints not only allow for useful long-range rules, but also effectively filter short-range rules. Experiments with adjunct optionality further provide promising results for English-Japanese, showing that adjunction can be applied outside of Syntax-Based SMT. Comparing adjuncts and constituents for extraction constraints reveals however that constituency tends to be the better guide. 

Our reordering modelling experiments confirm this result, showing notably for English-Japanese that adjuncts explain most, but not all translation reordering when compared to constituents.

On the whole, this dissertation shows that adjunction need not be confined to syntax-based models of translation, as properties of adjunction like optionality can be exploited in asyntactic models as well. 
But it also calls for a more extensive reliance of data-based translation models on core linguistic principles of recursion--starting with constituency.