Enhancing a temporal fusion transformer using GRU-LSTM encoder-decoder for effective solar generation forecasting

Main Article Content

Yeunyong Kantanet
Nattapon Kumyaito

Abstract

Efficient renewable energy management requires precise solar power forecasting. This study enhances prediction performance by integrating a Gated Recurrent Unit (GRU) – Long Short-Term Memory (LSTM) encoder-decoder architecture within a Temporal Fusion Transformer (TFT), enabling more effective modeling of complex temporal dependencies in solar generation data compared to traditional models. The novel contribution lies in the synergy between GRU’s ability to handle vanishing gradients and LSTM’s capability of maintaining long-term dependencies, resulting in improved forecasting accuracy. Additionally, we incorporate relevant meteorological data as supplementary inputs to refine the model's predictive precision. The results using the UNISOLAR and Solcast weather data reveal that our GRU-LSTM encoder-decoder within TFT (GRU-LSTM) model consistently outperforms the standard LSTM encoder-decoder within TFT (LSTM) and GRU encoder-decoder within TFT (GRU) models, achieving superior accuracy across both short-term and long-term forecasting tasks. This GRU-LSTM model exhibits significantly lower Mean Absolute Error (MAE), Mean Squared Error (MSE), and Root Mean Squared Error (RMSE), particularly during periods of high solar output. In short-term forecasting, the model achieved an MAE of 2.687, MSE of 15.603, and RMSE of 3.950 for Campus 1, and an MAE of 0.509, MSE of 0.585, and RMSE of 0.978 for Campus 2. Long-term results followed a similar trend, reinforcing the model’s ability to identify underlying patterns in solar generation data. These findings validate the effectiveness of the proposed GRU-LSTM encoder-decoder within TFT (GRU-LSTM) model for robust and exact solar power forecasting.

Article Details

How to Cite
Kantanet, Y., & Kumyaito, N. (2025). Enhancing a temporal fusion transformer using GRU-LSTM encoder-decoder for effective solar generation forecasting. Engineering and Applied Science Research, 52(6), 659–669. retrieved from https://ph01.tci-thaijo.org/index.php/easr/article/view/259960
Section
ORIGINAL RESEARCH

References

Bamisile O, Dagbasi M, Babatunde A, Ayodele O. A review of renewable energy potential in Nigeria; solar power development over the years. Eng Appl Sci Res. 2017;44(4):242-8.

Alcañiz A, Grzebyk D, Ziar H, Isabella O. Trends and gaps in photovoltaic power forecasting with machine learning. Energy Rep. 2023;9:447-71.

Mei F, Gu J, Lu J, Lu J, Zhang J, Jiang Y, et al. Day-ahead nonparametric probabilistic forecasting of photovoltaic power generation based on the LSTM-QRA ensemble model. IEEE Access. 2020;8:166138-49.

Mishra M, Dash PB, Nayak J, Naik B, Kumar Swain S. Deep learning and wavelet transform integrated approach for short-term solar PV power prediction. Measurement. 2020;166:108250.

Chodakowska E, Nazarko J, Nazarko Ł, Rabayah HS, Abendeh RM, Alawneh R. ARIMA models in solar radiation forecasting in different geographic locations. Energies. 2023;16(13):5029.

Alkahtani H, Aldhyani THH, Alsubari SN. Application of artificial intelligence model solar radiation prediction for renewable energy systems. Sustainability. 2023;15(8):6973.

Abdullah BUD, Khanday SA, Islam NU, Lata S, Fatima H, Nengroo SH. Comparative analysis using multiple regression models for forecasting photovoltaic power generation. Energies. 2024;17(7):1564.

Phyo PP, Jeenanunta C. Electricity load forecasting using a deep neural network. Eng Appl Sci Res. 2019;46(1):10-7.

Beigi M, Beigi Harchegani H, Torki M, Kaveh M, Szymanek M, Khalife E, et al. Forecasting of power output of a PVPS based on meteorological data using RNN approaches. Sustainability. 2022;14(5):3104.

Noh SH. Analysis of gradient vanishing of RNNs and performance comparison. Information. 2021;12(11):442.

Kang Q, Yu D, Cheong KH, Wang Z. Deterministic convergence analysis for regularized long short-term memory and its application to regression and multi-classification problems. Eng Appl Artif Intell. 2024;133:108444.

Ait Mansour A, Tilioua A, Touzani M. Bi-LSTM, GRU and 1D-CNN models for short-term photovoltaic panel efficiency forecasting case amorphous silicon grid-connected PV system. Results Eng. 2024;21:101886.

Ibrahim MS, Gharghory SM, Kamal HA. A hybrid model of CNN and LSTM autoencoder-based short-term PV power generation forecasting. Electr Eng. 2024;106(4):4239-55.

Ho R, Hung K. CEEMD-based multivariate financial time series forecasting using a temporal fusion transformer. The 14th Symposium on Computer Applications & Industrial Electronics (ISCAIE); 2024 May 24-25; Penang, Malaysia. USA: IEEE; 2024. p. 209-15.

Ayhan B, Vargo EP, Tang H. On the exploration of temporal fusion transformers for anomaly detection with multivariate aviation time-series data. Aerospace. 2024;11(8):646.

Islam M, Shuvo SS, Shohan JA, Faruque O. Forecasting of PV plant output using interpretable temporal fusion transformer model. 2023 North American Power Symposium (NAPS); 2023 Oct 15-17; Asheville, USA. USA: IEEE; 2023. p. 1-6.

Hanif MF, Mi J. Harnessing AI for solar energy: emergence of transformer models. Appl Energy. 2024;369:123541.

Mao W, Zhao H, Huang X, Miao J, Wang X, Geng Z. A short-term power prediction method for photovoltaic power generation based on GRU-transformer model. The 7th International Conference on Energy, Electrical and Power Engineering (CEEPE); 2024 Apr 26-28; Yangzhou, China. USA: IEEE; 2024. p. 1365-70.

Zheng P, Zhou H, Liu J, Nakanishi Y. Interpretable building energy consumption forecasting using spectral clustering algorithm and temporal fusion transformers architecture. Appl Energy. 2023;349:121607.

van Heerden L, van Staden C, Vermeulen HJ. Temporal fusion transformer for day-ahead wind power forecasting in the south african context. IEEE International Conference on Environment and Electrical Engineering and 2023 IEEE Industrial and Commercial Power Systems Europe (EEEIC / I&CPS Europe); 2023 Jun 6-9; Madrid, Spain. USA: IEEE; 2023. p. 1-5.

Feng G, Zhang L, Ai F, Zhang Y, Hou Y. An improved temporal fusion transformers model for predicting supply air temperature in high-speed railway carriages. Entropy. 2022;24(8):1111.

Li D, Tan Y, Zhang Y, Miao S, He S. Probabilistic forecasting method for mid-term hourly load time series based on an improved temporal fusion transformer model. Int J Electr Power Energy Syst. 2023;146:108743.

Mazen FM, Shaker Y, Abul Seoud RA. Forecasting of solar power using GRU-temporal fusion transformer model and DILATE loss function. Energies. 2023;16(24):8105.

Wimalaratne S, Haputhanthri D, Kahawala S, Gamage G, Alahakoon D, Jennings A. UNISOLAR: an open dataset of photovoltaic solar energy generation in a large multi-campus university setting. The 15th International Conference on Human System Interaction (HSI); 2022 Jul 28-31; Melbourne, Australia. USA: IEEE; 2022. p. 1-5.

Bright J. Solcast: validation of a satellite-derived solar irradiance dataset. Sol Energy. 2019;189:435-49.

Kantanet Y, Kumyaito N. A comparative analysis of machine learning models for robust multivariate imputation in solar energy datasets. ICIC Express Lett B: Appl. 2025;16(4):397-404.

Ratner B. The correlation coefficient: its values range between +1/−1, or do they?. J Target Meas Anal Mark. 2009;17(2):139-42.

Tang Y, Zhang L, Huang D, Yang S, Kuang Y. Ultra-short-term photovoltaic power generation prediction based on hunter–prey optimized K-Nearest neighbors and simple recurrent unit. Appl Sci. 2024;14(5):2159.

Lim B, Arık SÖ, Loeff N, Pfister T. Temporal fusion transformers for interpretable multi-horizon time series forecasting. Int. J. Forecast. 2021;37(4):1748-64.

Akiba T, Sano S, Yanase T, Ohta T, Koyama M. Optuna: a next-generation hyperparameter optimization framework. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining; 2019 Aug 4-8; Anchorage, USA. USA: Association for Computing Machinery; 2019. p. 2623-31.

Bergstra J, Bardenet R, Bengio Y, Kégl B. Algorithms for hyper-parameter optimization. Proceedings of the 25th International Conference on Neural Information Processing Systems; 2011 Dec 12-15; Granada, Spain. USA: Curran Associates Inc; 2011. p. 2546-54.

Jalali A, Azimi J, Fern X, Zhang R. A lipschitz exploration-exploitation scheme for bayesian optimization. European Conference, ECML PKDD 2013; 2013 Sep 23-27; Prague, Czech Republic. Berlin: Springer; 2013. p. 210-24.

Macêdo D, Zanchettin C, Ludermir T. Sigmoidal learning rate optimizer for deep neural network training using a two-phase adaptation approach. Appl Soft Comput. 2024;167:112264.

Snoek J, Larochelle H, Adams RP. Practical Bayesian optimization of machine learning algorithms. Proceedings of the 26th International Conference on Neural Information Processing Systems - Volume 2; 2012 Dec 3-6; Lake Tahoe, Nevada. USA: Curran Associates Inc; 2012. p. 2951-9.

Prakash U, Chollera A, Khatwani K, Prabuchandran KJ, Bodas T. Practical first-order bayesian optimization algorithms. Proceedings of the 7th Joint International Conference on Data Science & Management of Data (11th ACM IKDD CODS and 29th COMAD); 2024 Jan 4-7; Bangalore, India. USA: Association for Computing Machinery; 2024. p. 173-81.

Kingma DP, Ba JL. Adam: a method for stochastic optimization. International Conference on Learning Representations (ICLR 2015); 2015 May 7-9; San Diego, USA. USA: Ithaca; 2015. p. 1-15.

Mohapatra S, Sasy S, He X, Kamath G, Thakkar O. The role of adaptive optimizers for honest private hyperparameter selection. Proceedings of the AAAI Conference on Artificial Intelligence; 2022 Feb 22 – Mar 1; Online Conference. USA: AAAI; 2022. p. 7806-13.

Lai Y. Application and effectiveness evaluation of bayesian optimization algorithm in hyperparameter tuning of machine learning models. 2024 International Conference on Power, Electrical Engineering, Electronics and Control (PEEEC); 2024 Aug 14-16; Athens, Greece. USA: IEEE; 2024. P. 351-5.

Apaydin Ustun M, Xu L, Zeng B, Qian X. Hyperparameter tuning through pessimistic bilevel optimization [Internet]. arXiv [Preprint]. 2024 [cited 2024 Dec 4]. Available from: https://arxiv.org/abs/2412.03666.

Fetterman AJ, Kitanidis E, Albrecht J, Polizzi Z, Fogelman B, Knutins M, et al. Tune as you scale: hyperparameter optimization for compute efficient training [Internet]. arXiv [Preprint]. 2023 [cited 2024 Dec 4]. Available from: https://arxiv.org/abs/ 2306.08055.

Dinesh LP, Khafaf NA, McGrath B. A gated recurrent unit for very short-term photovoltaic generation forecasting. 2023 IEEE International Conference on Energy Technologies for Future Grids (ETFG); 2023 Dec 3-6; Wollongong, Australia. USA: IEEE; 2024. p. 1-6.

Goui G, Zrelli A, Benletaief N. A comparative study of LSTM/GRU models for energy long-term forecasting in IoT networks. 2023 IEEE/ACIS 23rd International Conference on Computer and Information Science (ICIS); 2023 Jun 23-25; Wuxi, China. USA: IEEE; 2023. p. 60-4.

Zameer A, Jaffar F, Shahid F, Muneeb M, Khan R, Nasir R. Short-term solar energy forecasting: integrated computational intelligence of LSTMs and GRU. PLoS One. 2023;18(10):e0285410.

Liu CH, Gu JC, Yang MT. A simplified LSTM neural networks for one day-ahead solar power forecasting. IEEE Access. 2021;9:17174-95.

Choi JY, Lee B. Combining LSTM network ensemble via adaptive weighting for improved time series forecasting. Math Probl Eng. 2018;2018:1-8.