Universiteit van Amsterdam

Please note that this newsitem has been archived, and may contain outdated information or links.

BEER@ILLC-UvA Best at WMT 2014

The BEER@ILLC-UvA metric (Miloš Stanojević and Khalil Sima'an) for evaluating Machine Translation system output (per sentence) obtains a top position in the recent international competition WMT2014.

The WMT2014 competition is a yearly international competition both for MT systems and evaluation metrics where a large number of research teams showcase their research findings in terms of system performance on a real-world task. For MT evaluation, the challenge is to produce an automatic evaluation metric that ranks different translations by different systems of every input sentence according to quality. Success is measured by statistical correlation against human judgements of the same system outputs.

The BEER@ILLC-UvA metric introduces a new statistical approach for MT evaluation based on an linear interpolation of adequacy and fluency features. Particularly, the fluency features are mostly based on a novel permutation factorization approach. The BEER@ILLC-UvA metric won the first place (from 16 submissions) in evaluating translations from English into six languages (French, Spanish, Russian, Czech, German, Hindi), and it won the second place in translating from these six languages into English.

For more details contact Milos Stanojevic and Khalil Sima'an.

Please note that this newsitem has been archived, and may contain outdated information or links.