Modelling Metadata and Data from Censuses and Surveys with Graph Databases

Alya Faradila; Lutfi Rahmatuti Maghfiroh

doi:10.29207/resti.v7i5.5273

Alya Faradila Politeknik Statistika STIS
Lutfi Rahmatuti Maghfiroh Politeknik Statistika STIS

DOI: https://doi.org/10.29207/resti.v7i5.5273

Keywords: graph database, database relational, database modelling

Abstract

Relational database users are switching to non-relational databases because non-relational databases are better able to handle dynamic data storage. One of the institutions that require dynamic data storage is Statistics Indonesia (BPS). Currently, data storage for census and survey activities at BPS is done using a relational database, although there are metadata changes in each activity. Accommodating metadata changes in each activity requires one database, which creates problems when retrieving some raw data. There is an opportunity for convenience if the data collected is stored in a non-relational database, one of which is a graph database. This research discusses the modeling of metadata and data from censuses and surveys at BPS using a graph database. Then we implement the Neo4j DBMS and compare the proposed model with the relational model in the Microsoft SQL Server DBMS. Then, a comparison of the features and characteristics of each DBMS is done, and finally, performance testing is done with Apache JMeter. Modeling has been able to handle dynamic data structure changes, but Neo4j's performance is still lagging behind Microsoft SQL Server.

Downloads

Download data is not yet available.

References

Jatin and Shalini Batra, “MONGODB Versus SQL: A Case Study on Electricity Data,” in Emerging Research in Computing, Information, Communication and Applications, Springer Singapore, 2016, pp. 297–308. doi: 10.1007/978-981-10-0287-8_28.

D. Kunda and H. Phiri, “A Comparative Study of NoSQL and Relational Database,” Zambia Information Communication Technology (ICT) Journal, vol. 1, no. 1, pp. 1–4, 2017.

E. Khatibi and S. L. Mirtaheri, “A dynamic data dissemination mechanism for Cassandra NoSQL data store,” Journal of Supercomputing, vol. 75, no. 11, pp. 7479–7496, Nov. 2019, doi: 10.1007/s11227-019-02959-7.

R. Hogan, “A Practical Guide to Database Design,” 2018.

Badan Pusat Statistik, “Informasi Umum BPS,” https://ppid.bps.go.id/, 2020.

BPS, “Metadata Sensus: Keterangan Umum,” https://sensus.bps.go.id/metadata_kegiatan/index/sp2020/keterangan%20umum, 2020.

BPS, “Istilah,” Jun. 11, 2009. https://www.bps.go.id/istilah/index.html?Istilah%5Bberawalan%5D=S&Istilah_page=5 (accessed Jul. 20, 2023).

Binus University, “PENGGUNAAN DATABASE ORACLE DI BEBERAPA PERUSAHAAN INDONESIA,” Aug. 08, 2018. https://sis.binus.ac.id/2018/08/08/penggunaan-database-oracle-di-beberapa-perusahaan-indonesia/ (accessed Sep. 09, 2023).

W. Ali, M. U. Shafique, M. A. Majeed, and A. Raza, “Comparison between SQL and NoSQL Databases and Their Relationship with Big Data Analytics,” Asian Journal of Research in Computer Science, pp. 1–10, Oct. 2019, doi: 10.9734/ajrcos/2019/v4i230108.

L. R. Maghfiroh and I. G. B. B. Nugraha, “Survey data and metadata modelling using document-oriented NoSQL,” in Journal of Physics: Conference Series, Institute of Physics Publishing, Apr. 2018. doi: 10.1088/1742-6596/971/1/012030.

L. R. Maghfiroh and I. Santoso, “NoSQL Model Data Warehaouse Metadata Survei Dinamis Studi Kasus: Survei Rumah Tangga,” 2019. [Online]. Available: https://silastik.bps.go.id

G. Harrison, Next generation databases : NoSQL, NewSQL, and Big Data. 2015.

A. Gupta, S. Tyagi, N. Panwar, and S. Sachdeva, “NoSQL Databases: Critical Analysis and Comparison,” International Conference on Computing and Communication Technologies for Smart Nation (IC3TSN), 2017.

I. Robinson, J. Webber, and E. Eifrem, “Graph Databases,” 2015.

S. Medhi and H. Baruah, “Relational database and graph database: A comparative analysis,” Journal of Process Management. New Technologies, vol. 5, no. 2, pp. 1–9, 2017, doi: 10.5937/jouproman5-13553.

V. Vojtˇ, V. Kolomičenko, M. Svoboda, and I. Holubová, “Experimental Comparison of Graph Databases *,” 2013. [Online]. Available: http://www.sparsity-technologies.com/dex

D. Fernandes and J. Bernardino, “Graph databases comparison: Allegrograph, arangoDB, infinitegraph, Neo4J, and orientDB,” in DATA 2018 - Proceedings of the 7th International Conference on Data Science, Technology and Applications, SciTePress, 2018, pp. 373–380. doi: 10.5220/0006910203730380.

P. Kotiranta, M. Junkkari, and J. Nummenmaa, “Performance of Graph and Relational Databases in Complex Queries,” Applied Sciences (Switzerland), vol. 12, no. 13, Jul. 2022, doi: 10.3390/app12136490.

C. Kemper, Beginning Neo4j. 2015.

D. G. Feitelson, “Experimental Computer Science: The Need for a Cultural Change,” 2006.

Y. Y. Sahria and D. H. Fudholi, “Pemodelan Pengetahuan Graph Database Untuk Jejaring Penelitian Kesehatan di Indonesia,” Jurnal Media Informatika Budidarma, vol. 4, no. 3, p. 604, Jul. 2020, doi: 10.30865/mib.v4i3.2183.

neo4j.com, “Graph Data Modeling,” Aug. 24, 2018.

E. H. Halili, Apache JMeter : a practical beginner’s guide to automated testing and performance measurement for your websites. Packt Pub, 2008.

H. Garcia-Molina, J. D. Ullman, and J. Widom, “DATABASE SYSTEMS The Complete Book Second Edition,” 2009.

R. McColl, D. Ediger, J. Poovey, D. Campbell, and D. A. Bader, “A performance evaluation of open source graph databases,” in PPAA 2014 - Proceedings of the 2014 Workshop on Parallel Programming for Analytics Applications, Association for Computing Machinery, 2014, pp. 11–17. doi: 10.1145/2567634.2567638.

K. Sahatqija, J. Ajdari, X. Zenuni, B. Raufi, and F. Ismaili, “Comparison between relational and NOSQL databases,” 2018.

Neo4j, “Neo4j Licensing,” Jun. 20, 2016. https://neo4j.com/licensing/ (accessed Jul. 20, 2023).

Micrososft, “Microsoft Licensing Resources,” https://www.microsoft.com/id-id/licensing/product-licensing/sql-server, 2019.

Neo4j, “Neo4j Pricing,” https://neo4j.com/pricing/#graph-database, 2023.

Microsoft, “SQL Server 2019 -- Pricing,” https://www.microsoft.com/en-us/sql-server/sql-server-2019-pricing, 2019. https://www.microsoft.com/en-us/sql-server/sql-server-2019-pricing (accessed May 10, 2023).

J. Stegeman, “Native vs. Non-Native Graph Database,” May 08, 2023. https://neo4j.com/blog/native-vs-non-native-graph-technology/ (accessed Jun. 06, 2023).

Neo4j, “The Neo4j Operations Manual v5,” https://neo4j.com/docs/operations-manual/current/, 2023.

Microsoft, “Security for SQL Server Database Engine and Azure SQL Database,” https://learn.microsoft.com/en-us/sql/relational-databases/security/security-center-for-sql-server-database-engine-and-azure-sql-database?view=sql-server-ver15, 2019.