Comparison of Storages Performance in Big Data Platform

Abstract

From the continuous application of information technology in various fields today. This causes the amount of data in information systems to increase rapidly. The data that comes from the data source is stored sparsely in external systems. Before being imported and linked to store in the Big Data Platform, which is designed to support the storage and processing of large data. Get all types efficiently before being put to use in various dimensions such as data analysis, data service and sharing, and reporting. So that executives can use these data and reports to truly analyze and plan to drive the organization with data.

However, the imported data is stored in the big data platform coming from different external systems resulting in a variety of data storage formats. Each format has a different structure on the storage, such as a row-based or column-based data structure, storing as a binary or text file, and supporting data compression. Because each format has both advantages and disadvantages, we therefore studied and compared the efficiency of data file formats stored on the big data platform in order to find the most suitable data file format for working in various cases as efficiently as possible. The results of the experiment indicated that column-based data structures are better suited for importing and storing data taken from external sources, while row-based data structures are suitable for querying or analyzing the data with more complex commands.