Man or machine? Comparing the difficulty of human translation versus neural machine translation post-editing

This study aims to compare neural machine translation (NMT) post-editing and human translation in terms of task difficulty while considering source text (ST) complexity and machine translation (MT) quality levels, two factors that have been rarely examined in previous comparison studies. Data were obtained from 60 trainee translators concerning the perceived and objective difficulties of post-editing and human translation tasks and the participants’ performance. It was found that (1) the difficulty of the NMT post-editing task, compared to human translation, was significantly influenced by both NMT quality and ST complexity; the difficulty of the post-editing task was significantly lower than that of the human translation task only in the case of high-quality NMT paired with complex ST, while the results were mixed for other interactions between NMT quality and ST complexity levels; (2) no strong correlations were found between the participants’ perceived difficulty and the measurements of objective difficulty and task performance for both post-editing and human translation tasks. Practical and research implications were discussed.