An In-depth Exploration of Sentiment Analysis on Hasanuddin Airport using Machine Learning Approaches

Lilis Nur Hayati; Fitrah Yusti Randana; Herdianti Darwis

doi:10.29207/resti.v9i2.6253

Lilis Nur Hayati Universitas Muslim Indonesia
Fitrah Yusti Randana Universitas Muslim Indonesia
Herdianti Darwis Universitas Muslim Indonesia

DOI: https://doi.org/10.29207/resti.v9i2.6253

Keywords: Sentiment Analysis, Support Vector Machine, Naive Bayes, K-Nearest Neighbor, SMOTE, Sultan Hasanuddin Airport

Abstract

Machine learning-based sentiment analysis has become essential for understanding public perceptions of public services, including air transportation. Sultan Hasanuddin Airport, one of the main gateways in eastern Indonesia, faces the challenge of improving services amid changing user needs due to the COVID-19 pandemic. This study aims to compare the effectiveness of three machine learning algorithms- Support Vector Machine (SVM), Naive Bayes Multinomial, and K-Nearest Neighbor (KNN)-in analyzing the sentiment of user reviews related to airport services. The research also explores data splitting techniques, text preprocessing, data balancing using SMOTE, model validation, and method parameterization to ensure optimal results. The review data was retrieved from Google Maps (2021-2024) and underwent manual labelling. Text preprocessing includes normalization, stemming using Sastrawi, and stopword removal. The data-balancing technique uses SMOTE, while model evaluation is done with stratified k-fold cross-validation. SVM with a linear kernel showed the best performance, achieving an F1-score of 98.4%. Naive Bayes performed optimally, achieving an F1-score of 93.9%, while KNN recorded the best F1-score of 92.0%. SMOTE was shown to improve Naive Bayes' performance on unbalanced datasets, although it did not significantly impact SVM. The findings of this study provide data-driven recommendations to improve services at Sultan Hasanuddin Airport, such as the management of cleaning facilities, waiting room comfort, and passenger flow efficiency. In addition, this research opens up opportunities for developing real-time sentiment analysis systems that can be applied in other air transportation sectors.

Downloads

Download data is not yet available.

References

N. Putu, O. Intan, Y. Putri, and S. Widiyanesti, “Sentiment Analysis of West Java International Airport (BIJB) Kertajati on Twitter,” Jurnal Manajemen dan Bisnis, vol. 4, no. 2, 2020, doi: 10.36555/almana.v4i2.1348.

I. Imelda and Arief Ramdhan Kurnianto, “Naïve Bayes and TF-IDF for Sentiment Analysis of the Covid-19 Booster Vaccine,” Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi), vol. 7, no. 1, pp. 1–6, Jan. 2023, doi: 10.29207/resti.v7i1.4467.

M. S. M. Alanazi, J. Li, and K. W. Jenkins, “Evaluating Airport Service Quality Based on the Statistical and Predictive Analysis of Skytrax Passenger Reviews,” Applied Sciences (Switzerland), vol. 14, no. 20, Oct. 2024, doi: 10.3390/app14209472.

A. Nurdina and A. B. I. Puspita, “Naive Bayes and KNN for Airline Passenger Satisfaction Classification: Comparative Analysis,” Journal of Information System Exploration and Research, vol. 1, no. 2, Jul. 2023, doi: 10.52465/joiser.v1i2.167.

L. Li, Y. Mao, Y. Wang, and Z. Ma, “How has airport service quality changed in the context of COVID-19: A data-driven crowdsourcing approach based on sentiment analysis,” J Air Transp Manag, vol. 105, Oct. 2022, doi: 10.1016/j.jairtraman.2022.102298.

A. Nieuwborg, M. Melles, S. Hiemstra-van Mastrigt, and S. Santema, “How can airports prepare for future public health disruptions? Experiences and lessons learned during the COVID-19 pandemic from a systemic perspective based on expert interviews,” Transp Res Interdiscip Perspect, vol. 23, Jan. 2024, doi: 10.1016/j.trip.2023.101000.

A. W. Sari, T. I. Hermanto, and M. Defriani, “Sentiment Analysis Of Tourist Reviews Using K-Nearest Neighbors Algorithm And Support Vector Machine,” Sinkron, vol. 8, no. 3, pp. 1366–1378, Jul. 2023, doi: 10.33395/sinkron.v8i3.12447.

E. Pujo Ariesanto Akhmad, K. Adi, and A. Puji Widodo, “Machine learning approach to customer sentiment analysis in twitter airline reviews,” in E3S Web of Conferences, EDP Sciences, Nov. 2023. doi: 10.1051/e3sconf/202344802044.

R. Song, W. Shi, W. Qin, X. Xue, and H. Jin, “Exploring Passengers’ Emotions and Satisfaction: A Comparative Analysis of Airport and Railway Station through Online Reviews,” Sustainability (Switzerland) , vol. 16, no. 5, Mar. 2024, doi: 10.3390/su16052108.

S. Szeghalmy and A. Fazekas, “A Comparative Study of the Use of Stratified Cross-Validation and Distribution-Balanced Stratified Cross-Validation in Imbalanced Learning,” Sensors, vol. 23, no. 4, Feb. 2023, doi: 10.3390/s23042333.

A. M. Sarhan, H. Ayman, M. Wagdi, B. Ali, A. Adel, and R. Osama, “Integrating machine learning and sentiment analysis in movie recommendation systems,” Journal of Electrical Systems and Information Technology, vol. 11, no. 1, p. 53, Nov. 2024, doi: 10.1186/s43067-024-00177-7.

R. Budiarto, P. Siber, and S. Negara, “Text Preprocessing for Optimal Accuracy in Indonesian Sentiment Analysis Using a Deep Learning Model with Word Embedding,” 2021. doi: 10.1063/5.0126116.

N. C. Dang, M. N. Moreno-García, and F. De la Prieta, “Sentiment analysis based on deep learning: A comparative study,” Electronics (Switzerland), vol. 9, no. 3, Mar. 2020, doi: 10.3390/electronics9030483.

Imamah and F. H. Rachman, “Twitter sentiment analysis of Covid-19 using term weighting TF-IDF and logistic regresion,” in Proceeding - 6th Information Technology International Seminar, ITIS 2020, Institute of Electrical and Electronics Engineers Inc., Oct. 2020, pp. 238–242. doi: 10.1109/ITIS50118.2020.9320958.

L. Cahyaningrum, A. Luthfiarta, and M. Rahayu, “Sentiment Analysis on the Impact of MBKM on Student Organizations Using Supervised Learning with Smote to Handle Data Imbalance,” Inform : Jurnal Ilmiah Bidang Teknologi Informasi dan Komunikasi, vol. 9, no. 1, pp. 58–66, Jan. 2024, doi: 10.25139/inform.v9i1.7484.

Hermanto, A. Y. Kuntoro, T. Asra, E. B. Pratama, L. Effendi, and R. Ocanitra, “Gojek and Grab User Sentiment Analysis on Google Play Using Naive Bayes Algorithm and Support Vector Machine Based Smote Technique,” in Journal of Physics: Conference Series, IOP Publishing Ltd, Nov. 2020. doi: 10.1088/1742-6596/1641/1/012102.

M. Hadwan, M. Al-Sarem, F. Saeed, and M. A. Al-Hagery, “An Improved Sentiment Classification Approach for Measuring User Satisfaction toward Governmental Services’ Mobile Apps Using Machine Learning Methods with Feature Engineering and SMOTE Technique,” Applied Sciences (Switzerland), vol. 12, no. 11, Jun. 2022, doi: 10.3390/app12115547.

A. Fadlil, I. Riadi, and F. Andrianto, “Improving Sentiment Analysis in Digital Marketplaces through SVM Kernel Fine-Tuning,” International Journal of Computing and Digital Systems, vol. 16, no. 1, pp. 159–171, Jul. 2024, doi: 10.12785/ijcds/160113.

A. Bhalla, “Comparative Analysis of Text Classification using SVM, Naïve Bayes, and KNN Models,” in 2023 5th International Conference on Inventive Research in Computing Applications (ICIRCA), 2023, pp. 878–883. doi: 10.1109/ICIRCA57980.2023.10220728.

K. Munawaroh, “Performance Comparison of SVM, Naïve Bayes, and KNN Algorithms for Analysis of Public Opinion Sentiment Against COVID-19 Vaccination on Twitter,” Journal of Advances in Information Systems and Technology, vol. 4, no. 2, 2022, doi: 10.15294/jaist.v4i2.59493.

V. B. Shtino and M. Muça, “Comparative Study of K-NN, Naive Bayes and SVM for Face Expression Classification Techniques,” Balkan Journal of Interdisciplinary Research, vol. 9, no. 3, pp. 23–32, Dec. 2023, doi: 10.2478/bjir-2023-0015.

H. Darwis, A. N. P. Pagala, S. Anraeni, T. Amaliah, I. As’ad, and A. U. Tenripada, “Analysis of Public Sentiment about Childfree in Indonesia using Support Vector Machine Methods,” in 2025 19th International Conference on Ubiquitous Information Management and Communication (IMCOM), 2025, pp. 1–8. doi: 10.1109/IMCOM64595.2025.10857551.

S. Akter et al., A Comprehensive Study Of Machine Learning Approaches For Customers Sentiment Analysis In Banking Sector, vol. 6, no. 10. 2024, pp. 100–111. doi: 10.37547/tajet/Volume06Issue10-11.

N. Umar and M. Adnan Nur, “Application of Naïve Bayes Algorithm Variations On Indonesian General Analysis Dataset for Sentiment Analysis,” Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi), vol. 6, no. 4, pp. 585–590, Aug. 2022, doi: 10.29207/resti.v6i4.4179.

Azwan Triyadi, “Public Sentiment Analysis About Neuralink from Twitter Using Naïve Bayes: Multinomial, Gaussian and Complement,” The Indonesian Journal of Computer Science, vol. 13, no. 5, Oct. 2024, doi: 10.33022/ijcs.v13i5.4278.

J. Muliawan and E. Dazki, “Sentiment Analysis of Indonesia’s Capital City Relocation Using Three Algorithms: Naive Bayes, KNN, and Random Forest,” Jurnal Teknik Informatika (JUTIF), vol. 4, no. 5, pp. 1227–1236, 2023, doi: 10.52436/1.jutif.2023.4.5.347.

F. M. J. M. Shamrat et al., “Sentiment analysis on twitter tweets about COVID-19 vaccines using NLP and supervised KNN classification algorithm,” Indonesian Journal of Electrical Engineering and Computer Science, vol. 23, no. 1, pp. 463–470, Jul. 2021, doi: 10.11591/ijeecs.v23.i1.pp463-470.

M. F. Fakhrezi, Adian Fatchur Rochim, and Dinar Mutiara Kusomo Nugraheni, “Comparison of Sentiment Analysis Methods Based on Accuracy Value Case Study: Twitter Mentions of Academic Article,” Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi), vol. 7, no. 1, pp. 161–167, Feb. 2023, doi: 10.29207/resti.v7i1.4767.

A. R. Isnain, J. Supriyanto, and M. P. Kharisma, “Implementation of K-Nearest Neighbor (K-NN) Algorithm For Public Sentiment Analysis of Online Learning,” IJCCS (Indonesian Journal of Computing and Cybernetics Systems), vol. 15, no. 2, p. 121, Apr. 2021, doi: 10.22146/ijccs.65176.