Abstract
This article reports on a study from the first author’s honours research project in Language Practice titled “’n Vergelykende gevallestudie oor die gehalte van tegniese vertaling deur ChatGPT-4o” (A comparative case study on the quality of technical translation by ChatGPT-4o) (Van den Bergh 2025). The motivation for this study was the popularity that AI, especially ChatGPT, has gained in recent years, and the questioning of human intervention that has accompanied this development. Although AI programs such as ChatGPT are not explicitly designed for translation, their underlying architecture enables them to perform such tasks. Consequently, doubts regarding the relevance of humans in certain tasks, such as translation, have surfaced.
Against this background, the aim of the research project was to investigate the technical translation quality of ChatGPT-4o from English into Afrikaans at two different reader levels, namely an expert reader level and a lay reader level. More specifically, the study sought to determine to what extent human intervention is still necessary to achieve a high-quality translation. To address this objective, a qualitative research approach was employed to analyse the translation quality of ChatGPT-4o’s translations.
Translation quality is a complex concept to define, and it cannot be measured solely in terms of stylistic and linguistic errors; it is also measured by the extent to which the translation brief has been fulfilled and how fluent and accurate the translation is (Fields, Hague, Koby, Lommel and Melby 2014). In Van den Bergh’s study, the translation quality of ChatGPT-4o was examined qualitatively through text analysis, allowing for an in-depth evaluation of ChatGPT-4o’s translation performance.
The theoretical framework of the study draws on the discussion of neural machine translation (NMT), AI translation, the functionalist approach to translation and ChatGPT as technical translator. NMT affords effects similar to those of AI through the use of artificial neural networks, enabling automated translation from one language into another without human intervention (Sakamoto 2022). NMT uses machine learning techniques that utilise input data and predict output data accordingly. However, it is questioned whether this represents the final and optimal paradigm of machine translation, and raises the question whether we are moving towards a new era of machine translation – the AI era (Melby 2019). Although NMT and AI are similar in various ways, one key difference supports the notion that the field may be moving towards a new machine translation paradigm, namely the ability to correct errors. ChatGPT-4o can, on request, correct its own input, whereas traditional NMT systems are unable to perform this task.
In addition, the study adopts a functionalist approach to translation. This approach postulates that the goal of the target text (TT) takes precedence over that of the source text (ST). However, it is important for translators to have a bilateral loyalty to both texts, a principle that is crucial to consider during the translation process (Nord 2002). Furthermore, Nord (1997) proposes a set of translation problems that can be used to analyse texts, while Pym (2016) provides a typology of translation solutions that may assist translators when they encounter such problems. The translation quality was also assessed by investigating how ChatGPT-4o addresses translation problems, using Pym’s typology of translation solutions as a framework.
The popularity of ChatGPT has given rise to numerous recent studies aimed at examining and evaluating its translation quality (Lyu, Tan, Zapadka, Ponnatapura, Niu et al. 2023). These studies tested the translation quality of ChatGPT across various settings and text types. However, relatively few studies have investigated technical translation in the South African context. This gap in the literature further motivated the study. Some studies in the international context highlight the fact that ChatGPT-4o falls short in areas such as common translation errors and inaccuracies, terminological issues and unjustified text shortening. ChatGPT-4o has, however, excelled in areas such as concision and clarity (Lyu, Tan et al. 2023).
Technical texts are among the most commonly translated texts in practice, and are characterised by features that distinguish them from other texts, such as professional jargon and abbreviations, and neutral, objective language. Given that technical texts are constructed with a specific goal and function in mind, it further motivates the reasoning for employing the functionalist approach to translation.
The study therefore employed a qualitative research design incorporating text analysis as the primary data collection technique. The data for the study was collected by means of two prompts functioning as translation briefs, instructing ChatGPT-4o to produce a translation at an expert reader level and at a lay reader level. ChatGPT-4o therefore received two prompts from the first author – one for each reading level – specifying the target reader, the target language and more detailed expectations for the translated text.
The data was organised according to Nord’s four types of translation problems, namely pragmatic, cultural, linguistic and text-specific problems. Within each category, ChatGPT-4o’s handling of these problems was discussed using Pym’s typology of translation solutions, which consists of copying, expression change and material change, in each of which more specific solutions are subsumed. To ensure a thorough analysis, other aids such as specialised dictionaries and resources were used to evaluate the output.
The analysis revealed that ChatGPT-4o demonstrated greater consistency in addressing pragmatic and intercultural translation problems than in the case of linguistic and text-specific problems. The linguistic and text-specific problems were solved rather inconsistently compared with the pragmatic and intercultural problems, indicating an area where ChatGPT-4o’s translation quality falls short. According to the translation brief, ChatGPT-4o provided a better translation for the expert reader than for the lay reader, as it required fewer adjustments and compensations. The analysis indicates that with thorough post-editing, the translation of the expert reader was deemed of higher quality than that of the lay reader’s.
Although the reception of the two target texts was not empirically tested, the analysis nevertheless provides valuable insights into the technical translation quality of ChatGPT-4o from English into Afrikaans for two different reader levels. Overall, the study indicates that while ChatGPT-4o shows translation potential, human intervention remains essential, if not crucial, to ensuring a translation that effectively communicates with the target readers. Human translators are, therefore, not as readily replaceable by AI as growing presumptions suggest, and remain essential in producing high-quality translations.
Keywords: AI; artificial intelligence; ChatGPT-4o; technical translation in Afrikaans; translation problems; translation solutions; translation quality
- This article’s featured image was created by Matheus Bertelli and obtained from Pexels.
Lees die volledige artikel in Afrikaans
’n Ondersoek na die gehalte van tegniese vertaling deur ChatGPT-4o