Development of Data Warehouse Using Data Integration Techniques to Reduced Data Redundancy for Workload Evaluation System Based on Academic Staff
Article Sidebar
Main Article Content
Abstract
In the era of big data, data integration to create data warehouses is vital to the development of organizations with a variety of data. There are a lot of redundancies. This redundancy affects the duration of decision-making and the overall efficiency of an information-driven organization. The objectives of the research were to: 1) develop a data warehouse using data integration techniques to reduce data redundancy; and 2) analyze data about redundancy reduction. This paper proposes the data warehouse development, which has five steps. Firstly, the process of data structure study, which uses the SQLyog program as a tool to understand the data structures of KIMs and WE Systems. Secondly, data integration is performed based on the ETL process via the SSIS platform. Thirdly, the data warehouse development process uses the SSMS program to create a new database that consolidates the integrated data. Fourthly, the process of testing on the data warehouse is done using SQL and PHP as tools to create the report systems for decision-making. Finally, the data redundancy analysis process is conducted to determine the rate of data redundancy reduction. This research uses raw data from the WE and KIMs databases, compressing 8,505 entries. The data sets are classified into two groups, which are the research group (5,490 entries) and the academic service group (3,015 entries). According to experimental results, the reduction in data redundancy for the research group between the two systems is 1,381 entries, which is equal to 74.84%. Similarly, the reduction in data redundancy for the academic services group between the two systems is 468 entries, which is equal to 84.47%. The research demonstrates that the development of data warehouses using data integration based on the ETL process can effectively reduce data redundancy within organizations. As a result, the performance of the data warehouse development process can improve the efficiency of decision-making and enhance data-driven operations within the organization.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Article Accepting Policy
The editorial board of Thai-Nichi Institute of Technology is pleased to receive articles from lecturers and experts in the fields of business administration, languages, engineering and technology written in Thai or English. The academic work submitted for publication must not be published in any other publication before and must not be under consideration of other journal submissions. Therefore, those interested in participating in the dissemination of work and knowledge can submit their article to the editorial board for further submission to the screening committee to consider publishing in the journal. The articles that can be published include solely research articles. Interested persons can prepare their articles by reviewing recommendations for article authors.
Copyright infringement is solely the responsibility of the author(s) of the article. Articles that have been published must be screened and reviewed for quality from qualified experts approved by the editorial board.
The text that appears within each article published in this research journal is a personal opinion of each author, nothing related to Thai-Nichi Institute of Technology, and other faculty members in the institution in any way. Responsibilities and accuracy for the content of each article are owned by each author. If there is any mistake, each author will be responsible for his/her own article(s).
The editorial board reserves the right not to bring any content, views or comments of articles in the Journal of Thai-Nichi Institute of Technology to publish before receiving permission from the authorized author(s) in writing. The published work is the copyright of the Journal of Thai-Nichi Institute of Technology.
References
ADPT. “DPT Explains – Data Integration: Data management that businesses in the digital age must know.” (in Thai), ADPT.news. https://www.adpt.news/2021/11/04/adpt-explains-data-integration (accessed Feb. 19, 2024).
J. Awiti, A. A. Vaisman, and E. Zimányi, “Design and implementation of ETL processes using BPMN and relational algebra,” Data Knowl. Eng., vol. 129, Sep. 2020, Art. no. 101837, doi: 10.1016/j.datak.2020.101837.
T. Thammanavasoros. “What is Data Integration?.” (in Thai), MEDIUM.com. https://totdatacom.medium.com/data-integration-คืออะไร-d51605639a86 (accessed Feb. 19, 2024).
J. Janruang et al., “Adding Potential of knowledge management and innovation to local economic development through digital platform for Rajamangala University of Technology Isan,” (in Thai), EAU Heritage J. Sci. Technol., vol. 16, no. 2, pp. 172–185, 2022.
J. Ronanaimuang, J. Janruang, and S. Karnka, “Applying data integration techniques for data of research load within Rajamangala University of Technology Isan,” (in Thai), in Proc. 13th Rajamangala Surin Nat. Conf., Surin, Thailand, Nov. 2022, pp. B582–B596.
K. Jetinai, “Semantic data integration of heterogeneous information systems using ontology mapping,” (in Thai), Inf. Technol. J., vol. 13, no. 2, pp. 29–38, 2017.
P. Chaiyabud. “What is the difference between ETL and ELT? Why do modern data warehouses use ELT?.” (in Thai), BLOG.DATATH.com. https://blog.datath.com/etl-vs-elt (accessed Apr. 25, 2024).
S. Dusadeeviroj. “Review ETL process.” (in Thai), FUSIONSOL.com. https://www.fusionsol.com/blog/review-etl-process/ (accessed Apr. 25, 2024).
N. Fatima. “What is data warehouse architecture.” (in Thai), ASTERA.com. https://astera.com/type/blog/data-warehouse-architecture (accessed Apr. 26, 2024).
E. Sangiamkul and T. Panrungsri, “Development of decision support system in impact and severity from flood in Phuket,” (in Thai), Inf. Technol. J., vol. 15, no. 1, pp. 60–70, 2019.
R. Sherman, “Data integration design and development,” in Business Intelligence Guidebook: From Data Integration to Analytics. Boston, MA, USA: Elsevier, 2015, ch. 11, pp. 275–299.
M. Hendayun, E. Yulianto, J. F. Rusdi, A. Setiawan, and B. Ilman, “Extract transform load process in banking reporting system,” MethodsX, vol. 8, 2021, Art. no. 101260, doi: 10.1016/j.mex.2021.101260.
J. Sreemathy, K. Naveen Durai, E. Lakshmi Priya, R. Deebika, K. Suganthi, and P. Aisshwarya, “Data integration and ETL: A theoretical perspective,” in Proc. 7th Int. Conf. Adv. Comput. and Commun. Syst. (ICACCS), Coimbatore, India, Mar. 2021, pp. 1655–1660.
D. Cho, M. Lee, and J. Shin, “Development of cost and schedule data integration algorithm based on big data technology,” Appl. Sci., vol. 10, no. 24, Dec. 2020, Art. no. 8917.
B. Silva, J. Moreira, and R. L. de C. Costa, “Logical big data integration and near real-time data analytics,” Data Knowl. Eng., vol. 146, Jul. 2023, Art. no. 102185, doi: 10.1016/j.datak.2023.102185.