Evaluating the Arabic–Indonesian Translation Quality of ChatGPT on Quranic and Hadith Texts
DOI:
https://doi.org/10.25217/jf.v10i2.6629Keywords:
ChatGPT, Religious Texts, Translation Evaluation, Translation QualityAbstract
This study examines the reliability of ChatGPT in translating Arabic–Indonesian religious texts, addressing the concern that AI often struggles with culturally embedded, legal, and theological expressions as highlighted in recent translation studies. The issue is significant because Qur’anic and hadith texts carry normative meanings whose alterations may lead to doctrinal misinterpretation. Data were collected from ten Qur’anic legal verses and ten hadiths from al-Arbaʿīn al-Nawawiyyah, translated by ChatGPT using a standardized prompt. The translations were evaluated through Nababan et al.’s accuracy–acceptability–readability model by expert reviewers and general readers. Findings show that ChatGPT performs well in accuracy and readability but displays limitations in rendering legal terminology, cultural nuance, and doctrinally sensitive expressions. This research contributes empirical evidence on the strengths and risks of AI-assisted translation for Islamic texts and underscores the necessity of expert post-editing to ensure theological precision and culturally appropriate interpretation.
References
Abdalhussein, H. F. (2025). Comparative Analysis Of AI And Human Translations Of Qur’anic Legal Verses In Surah Al-Ma’idah. Ijaz Arabi Journal of Arabic Learning, 8(1). https://doi.org/10.18860/ijazarabi.v8i1.28804
Abidah, N. K. K., Hikmawati, R., & Erawanto, V. (2024). ChatGPT in Indonesia-Arabic Translation: A Quality Analysis. Al-Arabi: Jurnal Bahasa Arab Dan Pengajarannya = Al-Arabi: Journal of Teaching Arabic as a Foreign Language, 8(1), 33–52. https://doi.org/10.17977/um056v8i1p33-52
AlJahsh, M. A. I. (2023). Influence Of Cultural Context On Qur’ānic Translation: Analyzing Social Justice Interpretations In Sura An-Nisā’ Verse 58. Maʿālim Al-Qurʾān Wa al-Sunnah, 19(2), 366–387. https://doi.org/10.33102/jmqs.v19i2.446
AlSukhni, E., Al-Kabi, M. N., & Alsmadi, I. M. (2016). An Automatic Evaluation for Online Machine Translation: Holy Quran Case Study. International Journal of Advanced Computer Science and Applications (Ijacsa), 7(6), Article 6. https://doi.org/10.14569/IJACSA.2016.070614
Arju, S., Husein, M. F., Sa’adah, B., & Rejeki, A. M. (2025). Analisis Keakuratan Hasil Terjemahan Chatgpt Menggunakan Teknik Back Translation. El-Tsaqafah : Jurnal Jurusan PBA, 24(1), Article 1. https://doi.org/10.20414/tsaqafah.v24i1.10604
Dahia, I., & Belbacha, M. (2024). Machine-Learning-based English Quranic Translation: An Evaluation of ChatGPT. International Journal of Linguistics, Literature and Translation, 7, 128–136. https://doi.org/10.32996/ijllt.2024.7.8.17
Dani, A. R. (2023). Gaya Bahasa Dialog Al-Qur’an “Analisis Stilistika Atas Dialog Al-Qur’an Mengenai Hari Kebangkitan. Jurnal Sosial Dan Sains, 3(3), 258–270. https://doi.org/10.59188/jurnalsosains.v3i3.703
El Ganadi, A., Aftar, S., Gagliardelli, L., & Ruozzi, F. (2025). Generative AI for Islamic Texts: The EMAN Framework for Mitigating GPT Hallucinations: Proceedings of the 17th International Conference on Agents and Artificial Intelligence, 1221–1228. https://doi.org/10.5220/0013312800003890
El-Saadany, M. R. (2024). A Comparative Study between Chat GPT and Human Translation in Translating English Proverbs into Arabic. مجلة البحث العلمي في الآداب. https://doi.org/10.21608/jssa.2024.257874.1592
Framesthia, L. M., Khoerunnisa, S., Izzah, S. A., Farisi, M. Z. A., & Supriadi, R. (2024). Analisis Perbandingan Teknik Penerjemahan Arab-Indonesia pada Google Translate dan ChatGPT. Ihtimam : Jurnal Pendidikan Bahasa Arab, 7(02), 114–127. https://doi.org/10.36668/jih.v7i02.716
Gaanoun, K., & Alsuhaibani, M. (2025). Sentiment preservation in Quran translation with artificial intelligence approach: Study in reputable English translation of the Quran. Humanities and Social Sciences Communications, 12. https://doi.org/10.1057/s41599-024-04181-00
Gunawan, F., Batmang, B., & Boulahnane, S. (2023). Translating the Qur’an in Indonesia: Marrying the Concept of Transcreation and Critical Discourse Analysis. IJELTAL (Indonesian Journal of English Language Teaching and Applied Linguistics), 8(2). https://doi.org/10.21093/ijeltal.v8i2.1523
Hassan Saleh Mahdi. (2022). The use of web-based translationsoftware in translatingreligious terms. Saudi Journal Of Language Studies. https://doi.org/10.1108/SJLS-11-2021-0023
Huda, K. (2018). Problematika Kebudayaan dalam Penerjemahan Bahasa Arab ke Bahasa Indonesia. Al-Fathin: Jurnal Bahasa Dan Sastra Arab, 1(02), 136–149. https://doi.org/10.32332/al-fathin.v1i2.1270
Istiqomah, S. N., Nurhaliza, T. N., Nafis, Z., & Supriadi, R. (2023). Teknik Penerjemahan Bahasa Arab ke Bahasa Indonesia: Taqniyyāt al-Tarjamah min al-‘Arabiyyah ila al-Indūnīsiyyah. Aphorisme: Journal of Arabic Language, Literature, and Education, 4(2), Article 2. https://doi.org/10.37680/aphorisme.v4i2.4500
Jiang, Z., Lv, Q., Zhang, Z., & Lei, L. (2024). Convergences and Divergences between Automatic Assessment and Human Evaluation: Insights from Comparing ChatGPT-Generated Translation and Neural Machine Translation (No. arXiv:2401.05176). arXiv. https://doi.org/10.48550/arXiv.2401.05176
Jiao, W., Wang, W., Huang, J., Wang, X., Shi, S., & Tu, Z. (2023). Is ChatGPT A Good Translator? Yes With GPT-4 As The Engine (No. arXiv:2301.08745). arXiv. https://doi.org/10.48550/arXiv.2301.08745
Kalla, D., Smith, N., Samaah, F., & Kuraku, S. (2023). Study and Analysis of Chat GPT and its Impact on Different Fields of Study (SSRN Scholarly Paper No. 4402499). Social Science Research Network. Google
Latifah, S. N., & Djamilah, W. I. F. (2024). Penggunaan Chat Gpt Dalam Penerjemahan Bahasa Arab. Al-Dirosah : Jurnal Pendidikan Agama Islam, 1(02). Google
M. Faidh Fasyani, Abdul Muqsid, & Abdal Wahab Mujtaba. (2024). Artificial Intelligence And Digital Tafsīr: Assessing the Interpretive Accuracy of ChatGPT’s Engagement with Tafsīr al-Qurṭūbī. Journal of Ushuluddin and Islamic Thought, 2(1). https://doi.org/10.15642/juit.2024.2.1.86-118
Majid, M. N., Akmaliyah, A., & Hermawan, M. B. (2024). Perbandingan Hasil Terjemah Arab – Indonesia Antara ChatGPT dan Gemini AI. Ihtimam : Jurnal Pendidikan Bahasa Arab, 7(02), 1–21. https://doi.org/10.36668/jih.v7i02.1004
Maulana, M. A. (2022). Problematika Penerjemahan Arab-Indonesia. Al Ibrah: Journal of Arabic Language Education, 5(2), Article 2. https://doi.org/10.24256/jale.v5i2.3417
Mustofa, R., Mustofa, A., & Surur, M. (2025). Variations of Arabic-Indonesian Translation Strategies in Automatic Machine Translation: A Comparative Study. Journal of Language and Literature Studies, 5(3), 522–540. https://doi.org/10.36312/jolls.v5i3.3075
Mu’tashim Billah, Susiknan Azhari, Nurdhin Baroroh, & Vivi Rahma. (2023). Evaluating AI-Generated Fatwas: A Quality Assessment of ChatGPT and Google Bard Against Authoritative Islamic Rulings. Asy-Syir’ah, 57(1). https://doi.org/10.14421/ajish.v57i1.1032
Nababan, M., & Nuraeni, A. (n.d.). Pengembangan Model Penilaian Kualitas Terjemahan. Kajian Linguistik dan Sastra, 24(1). Google
Nababan, M., Nuraeni, A., & Sumardiono. (2012). Pengembangan Model Penilaian Kualitas Terjemahan. Kajian Linguistik Dan Sastra, 24(1), Article 1. https://doi.org/10.23917/kls.v24i1.101
Nugraha, T. C., Lukman, F., & Amalia, R. M. (2022). Learning Model for Local Wisdom-based Prophet’s Hadith Translation. Islamic Research, 5(2), Article 2. https://doi.org/10.47076/jkpis.v5i2.170
Nurfaiza. (2024). Pengaruh Penggunaan Chat Gpt Dalam Pembelajaran Terjemah Indonesia-Arab. Ibtidaiyah Jurnal Pendidikan Guru Madrasah Ibtidaiyah, 70–89. https://doi.org/10.52491/ibtidaiyah.v2i2.228
Ruhmadi, A., & Farisi, M. Z. A. (2023). Analisis Kesalahan Morfologi Penerjemahan Arab–Indonesia pada ChatGPT: Taḥlīl al-Akhtā’ al-Ṣarfiyyah ‘ibra Tarjamah al-Arabiyyah wa al-Indūnīsiyyah fī ChatGPT. Aphorisme: Journal of Arabic Language, Literature, and Education, 4(1), Article 1. https://doi.org/10.37680/aphorisme.v4i1.3148
Ruth Suci Putri, Rudy Sofyan, & Ely Hayati Nasution. (2022). Translation Quality Assessment on Medan City Tourism Official Web Pages. LingPoet: Journal of Linguistics and Literary Research, 2(3). https://doi.org/10.32734/lingpoet.v3i1.6500
Sabrin Azzam Abdulrahman Fadhel. (2025). The Use of the AI Model Chat GPT in Translating Religious texts and Compare it with Expert Translation (Comparative Study). Albaydha University Journal, 7(1). https://doi.org/10.56807/buj.v7i1.758
Sadiq, S. (2025). Evaluating English-Arabic translation: Human translators vs. Google Translate and ChatGPT. Journal of Languages and Translation, 12(1), 67–95. https://doi.org/10.21608/jltmin.2025.423147
Saimin, A. A., Supriadi, R., & Farisi, M. Z. A. (2024). Analisis Kesalahan Penerjemahan Teks Bahasa Indonesia ke dalam Bahasa Arab pada ChatGPT (Studi Analisis Morfologi dan Sintaksis). Jurnal Naskhi Jurnal Kajian Pendidikan Dan Bahasa Arab, 6(2), 1–12. https://doi.org/10.47435/naskhi.v6i1.2668
Setyawan, C. E., Zulaeha, Rauhillah, S., & Mu’at. (2025). Analisis Penerjemahan Indonesia-Arab Google Translate dan CHAT-GPT di Tinjau dari Linguistik Struktural dan Semantik. Bilingua, 2(2), 1–11. https://doi.org/10.33752/bilingua.v2i2.8946
Shormani, M. Q., & Alfahad, A. (2025). Artificial Intelligence or Human: The Use of ChatGPT in the Academic Translation for Religious Texts. Sage Open, 15(3), 21582440251343954. https://doi.org/10.1177/21582440251343954
Siregar, R., Subagiharti, H., Handayani, D. S., Hutagaol, D., Hasibuan, A. L., Nuraida, N., & Sipahutar, E. (2025). A Comparative Analysis of Google Translate and ChatGPT Results Based on Translation Quality Assessment (TQA). International Journal of Educational Research Excellence (IJERE), 4(1), 328–337. https://doi.org/10.55299/ijere.v4i1.1175
Syihabuddin, S. (2016). Penerjemahan Teks Suci: Analisis Ketepatan Terjemahan Istilah Kecendekiaan Dalam Alquran dan Terjemahnya. Adabiyyāt: Jurnal Bahasa Dan Sastra, 15(1), Article 1. https://doi.org/10.14421/ajbs.2016.15105
Trisakti Agriani, M. R. Nababan, & Djatmika Djatmika. (2018). Translation Quality on Words Containing Spelling, Punctuation and Grammatical Error. International Journal of Multicultural and Multireligious Understanding, 5(5). https://doi.org/10.18415/ijmmu.v5i5.314
Zitouni, M., Alzahrani, A., Kous, N. A., Almutlaq, S., Abdul-Ghafour, A.-Q., & Zemni, B. (2022). The Translation of Selected Cultural Items in Al-Nawawi’s Forty Hadiths: A Descriptive and Analytical Study. Journal of Intercultural Communication, 22(3), 43–53. https://doi.org/10.36923/jicc.v22i3.74
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 M. Syamsul Ma'arif, Ruston Nawawi, Moh. Badrus Sholichin, Umar Faruq, Hilmy Baihaqy Yussof, Prabowo Adi Widayat
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.
