Enhancing Data Retrieval Efficiency in Geospatial Surveys Using Indexing Techniques for Large-Scale JavaScript Object Notation Datasets

Abstract

The use of JavaScript Object Notation (JSON) format as a Not only Structured Query Language (NoSQL) storage solution has grown in popularity but has presented technical challenges, particularly in indexing large-scale JSON files. In this study, we propose using JSON data sets especially in cases to survey and access large amounts of data, but the devices used for data collection have insufficient memory to process large-sized files. We conducted experiments on 32 Gigabyte data sets with 1,000,000 transactions in JSON format and implemented two types of indexing, Dense and Sparse, to enhance data access efficiency. Additionally, we identified the suitable sample size for both indexing methods. The findings indicated that the use of dense indexing decreased data retrieval time from 26,869.218 milliseconds (Non index) to 382.196 milliseconds, a reduction of 98.58% in one-to-one data retrieval and from 38,300.848 milliseconds to 1.097 when there were no keywords. In contrast, sparse indexing reduced data retrieval time from 55,197.734 milliseconds (Non index) to 854.661 milliseconds, a decrease of 98.45% in one-to-many data retrieval and from 47,203.253 milliseconds to 0.179 milliseconds when keywords were not found. Furthermore, we discovered that for both dense and sparse indexing, all sample size ranges could rapidly manage memory and access keywords.