Effects of Data Standardization on Hyperparameter Optimization with the Grid Search Algorithm Based on Deep Learning: A Case Study of Electric Load Forecasting
DOI:
https://doi.org/10.46604/aiti.2022.9227Keywords:
deep learning, grid search, data standardization method, hyperparameter, electric load forecastingAbstract
This study investigates data standardization methods based on the grid search (GS) algorithm for energy load forecasting, including zero-mean, min-max, max, decimal, sigmoid, softmax, median, and robust, to determine the hyperparameters of deep learning (DL) models. The considered DL models are the convolutional neural network (CNN) and long short-term memory network (LSTMN). The procedure is made over (i) setting the configuration for CNN and LSTMN, (ii) establishing the hyperparameter values of CNN and LSTMN models based on epoch, batch, optimizer, dropout, filters, and kernel, (iii) using eight data standardization methods to standardize the input data, and (iv) using the GS algorithm to search the optimal hyperparameters based on the mean absolute error (MAE) and mean absolute percent error (MAPE) indexes. The effectiveness of the proposed method is verified on the power load data of the Australian state of Queensland and Vietnamese Ho Chi Minh city. The simulation results show that the proposed data standardization methods are appropriate, except for the zero-mean and min-max methods.
References
A. M. Omer, “Energy Use and Environmental Impacts: A General Review,” Journal of Renewable and Sustainable Energy, vol. 1, no. 5, Article no. 053100, 2009.
K. Amasyali, et al., “A Review of Data-Driven Building Energy Consumption Prediction Studies,” Renewable and Sustainable Energy Reviews, vol. 81, pp. 1192-1205, January 2018.
E. Naml, et al., “Artificial Intelligence-Based Prediction Models for Energy Performance of Residential Buildings,” Recycling and Reuse Approach for Better Sustainability, pp. 141-149, 2019.
Y. Jiang, et al., “Stochastic Receding Horizon Control of Active Distribution Networks with Distributed Renewables,” IEEE Transactions on Power Systems, vol. 34, no. 2, pp. 1325-1341, March 2019.
J. C. López, et al., “Parsimonious Short-Term Load Forecasting for Optimal Operation Planning of Electrical Distribution Systems,” IEEE Transactions on Power Systems, vol. 34, no. 2, pp. 1427-1437, March 2019.
S. Khan, et al., “Forecasting Day, Week and Month ahead Electricity Load Consumption of a Building Using Empirical Mode Decomposition and Extreme Learning Machine,” 15th International Wireless Communications and Mobile Computing Conference, pp. 1600-1605, June 2019.
T. T. Ngoc, et al., “Grid Search of Exponential Smoothing Method: A Case Study of Ho Chi Minh City Load Demand,” Indonesian Journal of Electrical Engineering and Computer Science, vol. 19, no. 3, pp. 1121-1130, September 2020.
T. T. Ngoc, et al., “Grid Search of Multilayer Perceptron Based on the Walk-Forward Validation Methodology,” International Journal of Electrical and Computer Engineering, vol. 11, no. 2, pp. 1742-1751, April 2021.
K. Krishnakumari, et al., “Hyperparameter Tuning in Convolutional Neural Networks for Domain Adaptation in Sentiment Classification (HTCNN-DASC),” Soft Computing, vol. 24, no. 5, pp. 3511-3527, March 2020.
Y. Yu, et al., “Short-Term Load Forecasting Using Deep Belief Network with Empirical Mode Decomposition and Local Predictor,” IEEE Power and Energy Society General Meeting, pp. 1-5, August 2018.
X. Wang, et al., “LSTM-Based Short-Term Load Forecasting for Building Electricity Consumption,” IEEE 28th International Symposium on Industrial Electronics, pp. 1418-1423, June 2019.
M. Zahid, et al., “Electricity Price and Load Forecasting Using Enhanced Convolutional Neural Network and Enhanced Support Vector Regression in Smart Grids,” Electronics, vol. 8, no. 2, Article no. 122, 2019.
N. M. Aszemi, et al., “Hyperparameter Optimization in Convolutional Neural Network Using Genetic Algorithms,” International Journal of Advanced Computer Science and Applications, vol. 10, no. 6, pp. 269-278, March 2019.
J. Brownlee, Deep Learning for Time Series Forecasting: Predict the Future with MLPs, CNNs, and LSTMs in Python, New York: Machine Learning Mastery, 2018.
S. Mukhopadhyay, Deep Learning and Neural Networks, Advanced Data Analytics Using Python, Berkeley: Apress, 2018.
J. Moolayil, Learn Keras for Deep Neural Networks: A Fast-Track Approach to Modern Deep Learning with Python, New York: Springer, 2019.
S. Raschka, et al., Python Machine Learning: Machine Learning and Deep Learning with Python, Scikit-Learn, and TensorFlow 2, Hoboken: Wiley, 2019.
L. Yang, et al., “On Hyperparameter Optimization of Machine Learning Algorithms: Theory and Practice,” Neurocomputing, vol. 415, pp. 295-316, July 2020.
S. Motepe, et al., “Power Distribution Networks Load Forecasting Using Deep Belief Networks: The South African Case,” IEEE Jordan International Joint Conference on Electrical Engineering and Information Technology, pp. 507-512, April 2019.
X. Zhang, et al., “Deep Neural Network Hyperparameter Optimization with Orthogonal Array Tuning,” International Conference on Neural Information Processing, pp. 287-295, December 2019.
X. Song, et al., “Time-Series Well Performance Prediction Based on Long Short-Term Memory (LSTM) Neural Network Model,” Journal of Petroleum Science and Engineering, vol. 186, Article no. 106682, March 2020.
N. N. Bon, et al., “Fault Identification, Classification, and Location on Transmission Lines Using Combined Machine Learning Methods,” International Journal of Engineering and Technology Innovation, vol. 12, no. 2, pp. 91-109, February 2022.
U. Michelucci, Applied Deep Learning: A Case-Based Approach to Understanding Deep Neural Networks, New York: Apress, 2018.
S. V. Subramanian, et al., “Deep-Learning Based Time Series Forecasting of Go-Around Incidents in the National Airspace System,” AIAA Modeling and Simulation Technologies Conference, Article no. 0424, January 2018.
T. T. Ngoc, et al., “Support Vector Regression Based on Grid Search Method of Hyperparameters for Load Forecasting,” Acta Polytechnica Hungarica, vol. 18, no. 2, pp. 143-158, January 2021.
A. S. Girsang, et al., “Stock Price Prediction Using LSTM and Search Economics Optimization,” IAENG International Journal of Computer Science, vol. 47, no. 4, pp. 758-764, November 2020.
Published
How to Cite
Issue
Section
License
Submission of a manuscript implies: that the work described has not been published before that it is not under consideration for publication elsewhere; that if and when the manuscript is accepted for publication. Authors can retain copyright in their articles with no restrictions. is accepted for publication. Authors can retain copyright of their article with no restrictions.
Since Jan. 01, 2019, AITI will publish new articles with Creative Commons Attribution Non-Commercial License, under The Creative Commons Attribution Non-Commercial 4.0 International (CC BY-NC 4.0) License.
The Creative Commons Attribution Non-Commercial (CC-BY-NC) License permits use, distribution and reproduction in any medium, provided the original work is properly cited and is not used for commercial purposes.
