Evaluating Google Neural Machine Translation from Chinese to English: technical vs. literary texts
As the global need for translation increases, machine translation (MT) has significantly enhanced the efficiency in facilitating information dissemination and cross-cultural communication. However, its quality remains bound by intrinsic limitations among language pairs and text genres. These discrep...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Penerbit Universiti Kebangsaan Malaysia
2025
|
| Online Access: | http://psasir.upm.edu.my/id/eprint/120866/ http://psasir.upm.edu.my/id/eprint/120866/1/120866.pdf |
| _version_ | 1848868233070772224 |
|---|---|
| author | Zhang, Zhongming Syed Abdullah, Syed Nurulakla Redzuan Abdullah, Muhammad Alif Duan, Wenqi |
| author_facet | Zhang, Zhongming Syed Abdullah, Syed Nurulakla Redzuan Abdullah, Muhammad Alif Duan, Wenqi |
| author_sort | Zhang, Zhongming |
| building | UPM Institutional Repository |
| collection | Online Access |
| description | As the global need for translation increases, machine translation (MT) has significantly enhanced the efficiency in facilitating information dissemination and cross-cultural communication. However, its quality remains bound by intrinsic limitations among language pairs and text genres. These discrepancies lead to distinct MT performance when processing technical and literary texts, forming the core gap and focus. This study aims to compare the quality of Google Neural Machine Translation (GNMT) in literary and technical texts, investigating error disparities and establishing the abilities and limits of MT across diverse linguistic contexts. The research was concerned with the English-Chinese language pair with the Multidimensional Quality Metrics (MQM) framework for manual annotation. The COMET automatic evaluation metric was also applied for validation and confirmation of quality differences observed. This study selected five excerpts from Apple product manuals (33 aligned units) and the novel, the Old Man and Sea (32 aligned units), respectively. Findings included (1) GNMT performed well with technical texts, but acted less effective with literary texts and technical texts exhibited notable terminology errors, whereas literary texts showed more stylistic inconsistencies; (2) MQM scores demonstrated a remarkable difference, with technical texts outperforming literary texts by 18.57%; and (3) COMET evaluation validated the above observations, confirming a significant difference between the two text styles. Although GNMT faced challenges with both texts, the quality remained acceptable within this study. Results recommend improving GNMT algorithms to enhance accuracy and remedy error patterns and distributions. |
| first_indexed | 2025-11-15T14:49:08Z |
| format | Article |
| id | upm-120866 |
| institution | Universiti Putra Malaysia |
| institution_category | Local University |
| language | English |
| last_indexed | 2025-11-15T14:49:08Z |
| publishDate | 2025 |
| publisher | Penerbit Universiti Kebangsaan Malaysia |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | upm-1208662025-10-14T03:55:48Z http://psasir.upm.edu.my/id/eprint/120866/ Evaluating Google Neural Machine Translation from Chinese to English: technical vs. literary texts Zhang, Zhongming Syed Abdullah, Syed Nurulakla Redzuan Abdullah, Muhammad Alif Duan, Wenqi As the global need for translation increases, machine translation (MT) has significantly enhanced the efficiency in facilitating information dissemination and cross-cultural communication. However, its quality remains bound by intrinsic limitations among language pairs and text genres. These discrepancies lead to distinct MT performance when processing technical and literary texts, forming the core gap and focus. This study aims to compare the quality of Google Neural Machine Translation (GNMT) in literary and technical texts, investigating error disparities and establishing the abilities and limits of MT across diverse linguistic contexts. The research was concerned with the English-Chinese language pair with the Multidimensional Quality Metrics (MQM) framework for manual annotation. The COMET automatic evaluation metric was also applied for validation and confirmation of quality differences observed. This study selected five excerpts from Apple product manuals (33 aligned units) and the novel, the Old Man and Sea (32 aligned units), respectively. Findings included (1) GNMT performed well with technical texts, but acted less effective with literary texts and technical texts exhibited notable terminology errors, whereas literary texts showed more stylistic inconsistencies; (2) MQM scores demonstrated a remarkable difference, with technical texts outperforming literary texts by 18.57%; and (3) COMET evaluation validated the above observations, confirming a significant difference between the two text styles. Although GNMT faced challenges with both texts, the quality remained acceptable within this study. Results recommend improving GNMT algorithms to enhance accuracy and remedy error patterns and distributions. Penerbit Universiti Kebangsaan Malaysia 2025 Article PeerReviewed text en http://psasir.upm.edu.my/id/eprint/120866/1/120866.pdf Zhang, Zhongming and Syed Abdullah, Syed Nurulakla and Redzuan Abdullah, Muhammad Alif and Duan, Wenqi (2025) Evaluating Google Neural Machine Translation from Chinese to English: technical vs. literary texts. GEMA Online Journal of Language Studies, 25 (3). pp. 732-754. ISSN 1675-8021; eISSN: 2550-2131 https://ejournal.ukm.my/gema/article/view/85687 10.17576/gema-2025-2503-09 |
| spellingShingle | Zhang, Zhongming Syed Abdullah, Syed Nurulakla Redzuan Abdullah, Muhammad Alif Duan, Wenqi Evaluating Google Neural Machine Translation from Chinese to English: technical vs. literary texts |
| title | Evaluating Google Neural Machine Translation from Chinese to English: technical vs. literary texts |
| title_full | Evaluating Google Neural Machine Translation from Chinese to English: technical vs. literary texts |
| title_fullStr | Evaluating Google Neural Machine Translation from Chinese to English: technical vs. literary texts |
| title_full_unstemmed | Evaluating Google Neural Machine Translation from Chinese to English: technical vs. literary texts |
| title_short | Evaluating Google Neural Machine Translation from Chinese to English: technical vs. literary texts |
| title_sort | evaluating google neural machine translation from chinese to english: technical vs. literary texts |
| url | http://psasir.upm.edu.my/id/eprint/120866/ http://psasir.upm.edu.my/id/eprint/120866/ http://psasir.upm.edu.my/id/eprint/120866/ http://psasir.upm.edu.my/id/eprint/120866/1/120866.pdf |