[HumEval 2024] Quality and Quantity of Machine Translation References for Automatic Metrics

Higher-quality references lead to better metric correlations with humans at the segment-level. The references from vendors of different qualities can be mixed together and improve metric success. Higher quality references, however, cost more to create and we frame this as an optimization problem: given a specific budget, what references should be collected to maximize metric success.
Read the full paper here: arxiv.org/abs/2401.01283
The paper is presented at HumEval 2024, part of LREC-COLING.