Application of Recent Developments in Deep Learning to ANN-based Automatic Berthing Systems
Previous studies on Artificial Neural Network (ANN)-based automatic berthing showed considerable increases in performance by training ANNs with a set of berthing datasets. However, the berthing performance deteriorated when an extrapolated initial position was given. To overcome the extrapolation problem and improve the training performance, recent developments in Deep Learning (DL) are adopted in this paper. Recent activation functions, weight initialization methods, input data-scaling methods, a higher number of hidden layers, and Batch Normalization (BN) are considered, and their effectiveness has been analyzed based on loss functions, berthing performance histories, and berthing trajectories. Finally, it is shown that the use of recent activation and weight initialization method results in faster training convergence and a higher number of hidden layers. This leads to a better berthing performance over the training dataset. It is found that application of the BN can overcome the extrapolated initial position problem.
N. K. Im and K. Hasegawa, “Automatic ship berthing using parallel neural controller,” Proc. International Federation of Automatic Control (IFAC), vol. 34, no. 7, pp. 51-57, 2001.
N. K. Im and K. Hasegawa, “Motion identification using neural networks and its application to automatic ship berthing under wind,” Journal of Ship and Ocean Technology, vol. 6, no. 1, pp. 16-26, 2002.
N. K. Im, “All direction approach automatic ship berthing controller using ANN (artificial neural networks),” Journal of Institute of Control, Robotics and Systems, vol. 13, no. 4, pp. 304-308, 2007.
C. H. Bae, S. K. Lee, S. E. Lee, and J. H. Kim, “A study of the automatic berthing system of a ship using artificial neural network,” Journal of Korean Navigation and Port Research, vol. 32, no. 8, pp. 589-596, 2008.
Y. A. Ahmed and K. Hasegawa, “Automatic ship berthing using artificial neural network trained by consistent teaching data using nonlinear programming method,” Engineering Applications of Artificial Intelligence, vol. 26, no. 10, pp. 2287-2304, 2013.
N. K. Im and V. S. Nguyen, “Artificial neural network controller for automatic ship berthing using head-up coordinate system,” International Journal of Naval Architecture and Ocean Engineering, vol. 10, no. 3, pp. 235-249, 2018.
V. Nair and G. E. Hinton, “Rectified linear units improve restricted Boltzmann machines,” Proc. 27th International Conference on Machine Learning (ICML-10), 2010.
A. L. Maas, A. Y. Hannun, and A. Y. Ng, “Rectifier nonlinearities improve neural network acoustic models,” Proc. International Conference on Machine Learning, vol. 30, no. 1, 2013.
K. Levenberg, “A method for the solution of certain non-linear problems in least squares,” Quarterly of Applied Mathematics, vol. 2, no. 2, pp. 164-168, 1944.
T. Bazzi, R. Ismail, and M. Zohdy, “Comparative performance of several recent supervised learning algorithms,” International Journal of Computer and Information Technology, vol. 7, 2018.
T. Tieleman and G. Hinton, Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude, COURSERA: Neural Networks for Machine Learning, 2012.
D. P. Kingma and J. Ba, “Adam: a method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014.
K. H. Shon, “Hydrodynamic forces and maneuvering characteristics of ships at low advance speed,” Journal of the Society of Naval Architects of Korea, vol. 29, no. 3, pp. 90-101, 1992.
V. S. Nguyen, V. C. Do, and N. K. Im, “Development of automatic ship berthing system using artificial neural network and distance measurement system,” International Journal of Fuzzy Logic and Intelligent Systems, vol. 18, no. 1, pp. 41-49, 2018.
Y. A. Ahmed and K. Hasegawa, “Automatic ship berthing using artificial neural network based on virtual window concept in wind condition,” Proc. International Federation of Automatic Control (IFAC), vol. 45, no. 24, pp. 286-291, 2012.
X. Glorot and Y. Bengio, “Understanding the difficulty of training deep feedforward neural network,” Proc. International Conference on Artificial Intelligence and Statistics, vol. 9, pp. 246-266, 2010.
K. He, X. Zhang, S. Ren, and J. Sun, “Delving deep into rectifiers: surpassing human-level performance on Imagenet classification,” Proc. IEEE International Conference on Computer Vision, pp. 1026-1034, 2015.
H. Robbins and S. Monro, “A stochastic approximation method,” The Annals of Mathematical Stastics, pp. 400-407, 1951.
D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning representations by back-propagating errors,” Nature, vol. 323, no. 6088, pp. 533-536, 1986.
J. Duchi, E. Hazan, and Y. Singer, “Adaptive subgradient methods for online learning and stochastic optimization,” Journal of Machine Learning Research, pp. 2121-2159, July 2011.
C. M. Bishop, Neural networks for pattern recognition. Oxford: Clarendon Press, New York, 1996.
N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and D. Salakhutdinov, “Dropout: A simple way to prevent neural networks from overfitting,” Journal of Machine Learning Research, vol. 15, no.1, pp. 1929-1958, 2014.
S. Loffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” arXiv preprint arXiv:1502.03167, 2015.
J. N. Newman, Marine hydrodynamics. Cambridge, MA: MIT Press, 1980.
C. Nwankpa, W. Ijomah, A. Gachagan, and S. Marshall, “Activation functions: comparison of trends in practice and research for deep learning,” arXiv preprint arXiv:1811.03378, 2018.
S. K. Lee, G. W. Lee, S. J. Lee, and S. R. Jeong, “A study on the automatic berthing control of a ship by artificial neural network,” Journal of Korean Navigation and Port Research, vol. 32, no. 4, pp. 21-28, 1997.
Copyright (c) 2019 Daesoo Lee, Seung-Jae Lee, Yu-Jeong Seo
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Submission of a manuscript implies: that the work described has not been published before that it is not under consideration for publication elsewhere; that if and when the manuscript is accepted for publication. Authors can retain copyright in their articles with no restrictions. Also, author can post the final, peer-reviewed manuscript version (postprint) to any repository or website.
Since Jan. 01, 2019, IJETI will publish new articles with Creative Commons Attribution Non-Commercial License, under Creative Commons Attribution Non-Commercial 4.0 International (CC BY-NC 4.0) License.
The Creative Commons Attribution Non-Commercial (CC-BY-NC) License permits use, distribution and reproduction in any medium, provided the original work is properly cited and is not used for commercial purposes.