Optimizing Indonesian-Sundanese Bilingual Translation with Adam-Based Neural Machine Translation

Anita Qotrun Nada; Aji Prasetya Wibawa; Dhea Fanny Putri Syarifa; Erliana Fajarwati; Fadia Irsania Putri

doi:10.29207/resti.v8i6.6116

Anita Qotrun Nada Universitas Negeri Malang
Aji Prasetya Wibawa Universitas Negeri Malang
Dhea Fanny Putri Syarifa Universitas Negeri Malang
Erliana Fajarwati Universitas Negeri Malang
Fadia Irsania Putri Universitas Negeri Malang

DOI: https://doi.org/10.29207/resti.v8i6.6116

Keywords: Adam, BLEU, indonesian, LSTM, sundanese

Abstract

This research seeks to construct an automatic translation between Indonesian and Sundanese languages based on the Neural Machine Translation (NMT) method. The model used in this study is the Long Short-Term Memory (LSTM) type, which carries out an encoder-decoder structure model learned with Bible data. The text translation here was conducted in different epochs to optimize the process, followed by the Adam optimization algorithm. Testing the Adam optimizer with different epoch settings yields a BLEU score for Indonesian to Sundanese translations of 0.991785, higher than the performance of the None optimizer. Experimental results demonstrate that Indonesian to Sundanese translation using Adam optimization with 1000 epochs consistently performed better in BLEU - Bilingual Evaluation Understudy - scoring than Sundanese to Indonesian translation. Limitations of the research were also put forth, particularly technical issues related to the collection of data and the Sundanese language’s complex grammatical features, that the model can only partially express, honorifics, and the problem of polysemy. Also, it must be mentioned that no special hyperparameter selection was performed, as parameters were chosen randomly. In future studies, transformer-based models can be investigated since these architectures will better deal with complex language via their self-attention mechanism.

Downloads

Download data is not yet available.

References

R. Yoseptry, “The Management of Sundanese Cultural Local Wisdom Learning in developing Early Childhood Nationalist Character,” AL-ISHLAH J. Pendidik., vol. 14, no. 4, pp. 5035–5050, Sep. 2022, doi: 10.35445/alishlah.v14i4.1732.

W. Wongso, H. Lucky, and D. Suhartono, “Pre-trained transformer-based language models for Sundanese,” J. Big Data, vol. 9, no. 1, p. 39, Dec. 2022, doi: 10.1186/s40537-022-00590-7.

Y. Heryadi, B. D. Wijanarko, D. Fitria Murad, C. Tho, and K. Hashimoto, “Indonesian-Sundanese Language Machine Translation using Bidirectional Long Short-term Memory Model,” in 2023 International Conference on Computer Science, Information Technology and Engineering (ICCoSITE), Jakarta, Indonesia: IEEE, Feb. 2023, pp. 945–950. doi: 10.1109/ICCoSITE57641.2023.10127691.

B. D. Wijanarko, Y. Heryadi, D. F. Murad, C. Tho, and K. Hashimoto, “Recurrent Neural Network-based Models as Bahasa Indonesia-Sundanese Language Neural Machine Translator,” in 2023 International Conference on Computer Science, Information Technology and Engineering (ICCoSITE), Jakarta, Indonesia: IEEE, Feb. 2023, pp. 951–956. doi: 10.1109/ICCoSITE57641.2023.10127817.

Y. Heryadi, B. D. Wijanarko, D. F. Murad, C. Tho, and K. Hashimoto, “Neural Machine Translation Approach for Low-resource Languages using Long Short-term Memory Model,” in 2023 International Conference on Computer Science, Information Technology and Engineering (ICCoSITE), Jakarta, Indonesia: IEEE, Feb. 2023, pp. 939–944. doi: 10.1109/ICCoSITE57641.2023.10127724.

B. D. Wijanarko, D. Fitria Murad, Y. Heryadi, C. Tho, and K. Hashimoto, “Exploring the Effect of Activation Function on Transformer Model Performance for Official Announcement Translator from Indonesian to Sundanese Languages,” in 2023 International Conference on Computer Science, Information Technology and Engineering (ICCoSITE), Jakarta, Indonesia: IEEE, Feb. 2023, pp. 827–831. doi: 10.1109/ICCoSITE57641.2023.10127770.

K. Azizah and M. Adriani, “Hierarchical Transfer Learning for Text-to-Speech in Indonesian, Javanese, and Sundanese Languages,” in 2020 International Conference on Advanced Computer Science and Information Systems (ICACSIS), Depok, Indonesia: IEEE, Oct. 2020, pp. 421–428. doi: 10.1109/ICACSIS51025.2020.9263086.

K. Resiandi, Y. Murakami, and A. H. Nasution, “Neural Network-Based Bilingual Lexicon Induction for Indonesian Ethnic Languages,” Appl. Sci., vol. 13, no. 15, p. 8666, Jul. 2023, doi: 10.3390/app13158666.

R. Dabre, C. Chu, and A. Kunchukuttan, “A Survey of Multilingual Neural Machine Translation,” ACM Comput. Surv., vol. 53, no. 5, pp. 1–38, Sep. 2021, doi: 10.1145/3406095.

S. Ranathunga, E.-S. A. Lee, M. Prifti Skenduli, R. Shekhar, M. Alam, and R. Kaur, “Neural Machine Translation for Low-resource Languages: A Survey,” ACM Comput. Surv., vol. 55, no. 11, pp. 1–37, Nov. 2023, doi: 10.1145/3567592.

Z. Tan et al., “Neural machine translation: A review of methods, resources, and tools,” AI Open, vol. 1, pp. 5–21, 2020, doi: 10.1016/j.aiopen.2020.11.001.

F. Stahlberg, “Neural Machine Translation: A Review,” J. Artif. Intell. Res., vol. 69, pp. 343–418, Oct. 2020, doi: 10.1613/jair.1.12007.

X. Zheng et al., “Adaptive Nearest Neighbor Machine Translation,” in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), Online: Association for Computational Linguistics, 2021, pp. 368–374. doi: 10.18653/v1/2021.acl-short.47.

B. Zhang, D. Xiong, and J. Su, “Neural Machine Translation with Deep Attention,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 42, no. 1, pp. 154–163, Jan. 2020, doi: 10.1109/TPAMI.2018.2876404.

G. Ramesh et al., “Samanantar : The Largest Publicly Available Parallel Corpora Collection for 11 Indic Languages,” Trans. Assoc. Comput. Linguist., vol. 10, pp. 145–162, Feb. 2022, doi: 10.1162/tacl_a_00452.

H. Wang, H. Wu, Z. He, L. Huang, and K. W. Church, “Progress in Machine Translation,” Engineering, vol. 18, pp. 143–153, Nov. 2022, doi: 10.1016/j.eng.2021.03.023.

C. Lalrempuii and B. Soni, “Attention-Based English to Mizo Neural Machine Translation,” in Machine Learning, Image Processing, Network Security and Data Sciences, vol. 1241, A. Bhattacharjee, S. Kr. Borgohain, B. Soni, G. Verma, and X.-Z. Gao, Eds., in Communications in Computer and Information Science, vol. 1241. , Singapore: Springer Singapore, 2020, pp. 193–203. doi: 10.1007/978-981-15-6318-8_17.

F. A. Ahda, A. P. Wibawa, D. Dwi Prasetya, and D. Arbian Sulistyo, “Comparison of Adam Optimization and RMS prop in Minangkabau-Indonesian Bidirectional Translation with Neural Machine Translation,” JOIV Int. J. Inform. Vis., vol. 8, no. 1, p. 231, Mar. 2024, doi: 10.62527/joiv.8.1.1818.

D. A. Sulistyo, “LSTM-Based Machine Translation for Madurese-Indonesian,” J. Appl. Data Sci., vol. 4, no. 3, pp. 189–199, Sep. 2023, doi: 10.47738/jads.v4i3.113.

R. Ratawa and P. Nanda, “Artificial neural networks based damage severity assessment of structures using modal analysis,” in Sensors and Smart Structures Technologies for Civil, Mechanical, and Aerospace Systems 2022, D. Zonta, Z. Su, and B. Glisic, Eds., Long Beach, United States: SPIE, Apr. 2022, p. 36. doi: 10.1117/12.2616710.

B. Eikema and W. Aziz, “Is MAP Decoding All You Need? The Inadequacy of the Mode in Neural Machine Translation,” in Proceedings of the 28th International Conference on Computational Linguistics, Barcelona, Spain (Online): International Committee on Computational Linguistics, 2020, pp. 4506–4520. doi: 10.18653/v1/2020.coling-main.398.

M. Fu, C. Tantithamthavorn, T. Le, V. Nguyen, and D. Phung, “VulRepair: a T5-based automated software vulnerability repair,” in Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, Singapore Singapore: ACM, Nov. 2022, pp. 935–947. doi: 10.1145/3540250.3549098.

C. S. Xia and L. Zhang, “Less training, more repairing please: revisiting automated program repair via zero-shot learning,” in Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, Singapore Singapore: ACM, Nov. 2022, pp. 959–971. doi: 10.1145/3540250.3549101.

P. He, C. Meister, and Z. Su, “Structure-invariant testing for machine translation,” in Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering, Seoul South Korea: ACM, Jun. 2020, pp. 961–973. doi: 10.1145/3377811.3380339.

S. Li et al., “Hidden Backdoors in Human-Centric Language Models,” in Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, Virtual Event Republic of Korea: ACM, Nov. 2021, pp. 3123–3140. doi: 10.1145/3460120.3484576.

C. Watson, M. Tufano, K. Moran, G. Bavota, and D. Poshyvanyk, “On learning meaningful assert statements for unit test cases,” in Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering, Seoul South Korea: ACM, Jun. 2020, pp. 1398–1409. doi: 10.1145/3377811.3380429.

T. Meinnel, C. Dian, and C. Giglione, “Myristoylation, an Ancient Protein Modification Mirroring Eukaryogenesis and Evolution,” Trends Biochem. Sci., vol. 45, no. 7, pp. 619–632, Jul. 2020, doi: 10.1016/j.tibs.2020.03.007.

K. Dedes, A. B. Putra Utama, A. P. Wibawa, A. N. Afandi, A. N. Handayani, and L. Hernandez, “Neural Machine Translation of Spanish-English Food Recipes Using LSTM,” JOIV Int. J. Inform. Vis., vol. 6, no. 2, p. 290, Jun. 2022, doi: 10.30630/joiv.6.2.804.

Y. Hu, W. Tao, Y. Xie, Y. Sun, and Z. Pan, “Token-level disentanglement for unsupervised text style transfer,” Neurocomputing, vol. 560, p. 126823, Dec. 2023, doi: 10.1016/j.neucom.2023.126823.

M. Işik and H. Dağ, “The impact of text preprocessing on the prediction of review ratings,” Turk. J. Electr. Eng. Comput. Sci., vol. 28, no. 3, pp. 1405–1421, May 2020, doi: 10.3906/elk-1907-46.

M. Singh, R. Kumar, and I. Chana, “Corpus based Machine Translation System with Deep Neural Network for Sanskrit to Hindi Translation,” Procedia Comput. Sci., vol. 167, pp. 2534–2544, 2020, doi: 10.1016/j.procs.2020.03.306.

G. Van Houdt, C. Mosquera, and G. Nápoles, “A review on the long short-term memory model,” Artif. Intell. Rev., vol. 53, no. 8, pp. 5929–5955, Dec. 2020, doi: 10.1007/s10462-020-09838-1.

S. A. Mohamed, A. A. Elsayed, Y. F. Hassan, and M. A. Abdou, “Neural machine translation: past, present, and future,” Neural Comput. Appl., vol. 33, no. 23, pp. 15919–15931, Dec. 2021, doi: 10.1007/s00521-021-06268-0.

L. Benkova, D. Munkova, Ľ. Benko, and M. Munk, “Evaluation of English–Slovak Neural and Statistical Machine Translation,” Appl. Sci., vol. 11, no. 7, p. 2948, Mar. 2021, doi: 10.3390/app11072948.

C. Su, H. Huang, S. Shi, P. Jian, and X. Shi, “Neural machine translation with Gumbel Tree-LSTM based encoder,” J. Vis. Commun. Image Represent., vol. 71, p. 102811, Aug. 2020, doi: 10.1016/j.jvcir.2020.102811.

W.-J. Ko et al., “Adapting High-resource NMT Models to Translate Low-resource Related Languages without Parallel Data,” in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Online: Association for Computational Linguistics, 2021, pp. 802–812. doi: 10.18653/v1/2021.acl-long.66.

S. Mostafi, T. Alghamdi, and K. Elgazzar, “Interconnected Traffic Forecasting Using Time Distributed Encoder-Decoder Multivariate Multi-Step LSTM,” in 2024 IEEE Intelligent Vehicles Symposium (IV), Jeju Island, Korea, Republic of: IEEE, Jun. 2024, pp. 2503–2508. doi: 10.1109/IV55156.2024.10588573.

Y. Fan, F. Tian, Y. Xia, T. Qin, X.-Y. Li, and T.-Y. Liu, “Searching Better Architectures for Neural Machine Translation,” IEEEACM Trans. Audio Speech Lang. Process., vol. 28, pp. 1574–1585, 2020, doi: 10.1109/TASLP.2020.2995270.

S. Edunov, M. Ott, M. Ranzato, and M. Auli, “On The Evaluation of Machine Translation Systems Trained With Back-Translation,” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online: Association for Computational Linguistics, 2020, pp. 2836–2846. doi: 10.18653/v1/2020.acl-main.253.

N. Mathur, T. Baldwin, and T. Cohn, “Tangled up in BLEU: Reevaluating the Evaluation of Automatic Machine Translation Evaluation Metrics,” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online: Association for Computational Linguistics, 2020, pp. 4984–4997. doi: 10.18653/v1/2020.acl-main.448.

B. Zhang, P. Williams, I. Titov, and R. Sennrich, “Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translation,” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online: Association for Computational Linguistics, 2020, pp. 1628–1639. doi: 10.18653/v1/2020.acl-main.148.

S.-P. Chuang, T.-W. Sung, A. H. Liu, and H. Lee, “Worse WER, but Better BLEU? Leveraging Word Embedding as Intermediate in Multitask End-to-End Speech Translation,” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online: Association for Computational Linguistics, 2020, pp. 5998–6003. doi: 10.18653/v1/2020.acl-main.533.

D. Tay, “Turning metaphor on its head: a ‘target-to-source transformation’ approach in statistics education,” Front. Psychol., vol. 14, p. 1162925, Jun. 2023, doi: 10.3389/fpsyg.2023.1162925.

A. Dalkıran et al., “Transfer learning for drug–target interaction prediction,” Bioinformatics, vol. 39, no. Supplement_1, pp. i103–i110, Jun. 2023, doi: 10.1093/bioinformatics/btad234.