Application of Recent Developments in Deep Learning to ANN-based Automatic Berthing Systems

Daesoo Lee; Seung-Jae  Lee; Yu-Jeong Seo

doi:10.46604/ijeti.2020.4354

Authors

Daesoo Lee Division of Naval Architecture and Ocean Systems Engineering, Korea Maritime and Ocean University, Busan, Korea
Seung-Jae Lee Division of Naval Architecture and Ocean Systems Engineering, Korea Maritime and Ocean University, Busan, Korea
Yu-Jeong Seo Division of Naval Architecture and Ocean Systems Engineering, Korea Maritime and Ocean University, Busan, Korea

DOI:

https://doi.org/10.46604/ijeti.2020.4354

Keywords:

automatic ship berthing, artificial neural network, deep neural network, batch normalization

Abstract

Previous studies on Artificial Neural Network (ANN)-based automatic berthing showed considerable increases in performance by training ANNs with a set of berthing datasets. However, the berthing performance deteriorated when an extrapolated initial position was given. To overcome the extrapolation problem and improve the training performance, recent developments in Deep Learning (DL) are adopted in this paper. Recent activation functions, weight initialization methods, input data-scaling methods, a higher number of hidden layers, and Batch Normalization (BN) are considered, and their effectiveness has been analyzed based on loss functions, berthing performance histories, and berthing trajectories. Finally, it is shown that the use of recent activation and weight initialization method results in faster training convergence and a higher number of hidden layers. This leads to a better berthing performance over the training dataset. It is found that application of the BN can overcome the extrapolated initial position problem.

References

N. K. Im and K. Hasegawa, “Automatic ship berthing using parallel neural controller,” Proc. International Federation of Automatic Control (IFAC), vol. 34, no. 7, pp. 51-57, 2001.

N. K. Im and K. Hasegawa, “Motion identification using neural networks and its application to automatic ship berthing under wind,” Journal of Ship and Ocean Technology, vol. 6, no. 1, pp. 16-26, 2002.

N. K. Im, “All direction approach automatic ship berthing controller using ANN (artificial neural networks),” Journal of Institute of Control, Robotics and Systems, vol. 13, no. 4, pp. 304-308, 2007.

C. H. Bae, S. K. Lee, S. E. Lee, and J. H. Kim, “A study of the automatic berthing system of a ship using artificial neural network,” Journal of Korean Navigation and Port Research, vol. 32, no. 8, pp. 589-596, 2008.

Y. A. Ahmed and K. Hasegawa, “Automatic ship berthing using artificial neural network trained by consistent teaching data using nonlinear programming method,” Engineering Applications of Artificial Intelligence, vol. 26, no. 10, pp. 2287-2304, 2013.

N. K. Im and V. S. Nguyen, “Artificial neural network controller for automatic ship berthing using head-up coordinate system,” International Journal of Naval Architecture and Ocean Engineering, vol. 10, no. 3, pp. 235-249, 2018.

V. Nair and G. E. Hinton, “Rectified linear units improve restricted Boltzmann machines,” Proc. 27th International Conference on Machine Learning (ICML-10), 2010.

A. L. Maas, A. Y. Hannun, and A. Y. Ng, “Rectifier nonlinearities improve neural network acoustic models,” Proc. International Conference on Machine Learning, vol. 30, no. 1, 2013.

K. Levenberg, “A method for the solution of certain non-linear problems in least squares,” Quarterly of Applied Mathematics, vol. 2, no. 2, pp. 164-168, 1944.

T. Bazzi, R. Ismail, and M. Zohdy, “Comparative performance of several recent supervised learning algorithms,” International Journal of Computer and Information Technology, vol. 7, 2018.

T. Tieleman and G. Hinton, Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude, COURSERA: Neural Networks for Machine Learning, 2012.

D. P. Kingma and J. Ba, “Adam: a method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014.

K. H. Shon, “Hydrodynamic forces and maneuvering characteristics of ships at low advance speed,” Journal of the Society of Naval Architects of Korea, vol. 29, no. 3, pp. 90-101, 1992.

V. S. Nguyen, V. C. Do, and N. K. Im, “Development of automatic ship berthing system using artificial neural network and distance measurement system,” International Journal of Fuzzy Logic and Intelligent Systems, vol. 18, no. 1, pp. 41-49, 2018.

Y. A. Ahmed and K. Hasegawa, “Automatic ship berthing using artificial neural network based on virtual window concept in wind condition,” Proc. International Federation of Automatic Control (IFAC), vol. 45, no. 24, pp. 286-291, 2012.

X. Glorot and Y. Bengio, “Understanding the difficulty of training deep feedforward neural network,” Proc. International Conference on Artificial Intelligence and Statistics, vol. 9, pp. 246-266, 2010.

K. He, X. Zhang, S. Ren, and J. Sun, “Delving deep into rectifiers: surpassing human-level performance on Imagenet classification,” Proc. IEEE International Conference on Computer Vision, pp. 1026-1034, 2015.

H. Robbins and S. Monro, “A stochastic approximation method,” The Annals of Mathematical Stastics, pp. 400-407, 1951.

D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning representations by back-propagating errors,” Nature, vol. 323, no. 6088, pp. 533-536, 1986.

J. Duchi, E. Hazan, and Y. Singer, “Adaptive subgradient methods for online learning and stochastic optimization,” Journal of Machine Learning Research, pp. 2121-2159, July 2011.

C. M. Bishop, Neural networks for pattern recognition. Oxford: Clarendon Press, New York, 1996.

N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and D. Salakhutdinov, “Dropout: A simple way to prevent neural networks from overfitting,” Journal of Machine Learning Research, vol. 15, no.1, pp. 1929-1958, 2014.

S. Loffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” arXiv preprint arXiv:1502.03167, 2015.

J. N. Newman, Marine hydrodynamics. Cambridge, MA: MIT Press, 1980.

C. Nwankpa, W. Ijomah, A. Gachagan, and S. Marshall, “Activation functions: comparison of trends in practice and research for deep learning,” arXiv preprint arXiv:1811.03378, 2018.

S. K. Lee, G. W. Lee, S. J. Lee, and S. R. Jeong, “A study on the automatic berthing control of a ship by artificial neural network,” Journal of Korean Navigation and Port Research, vol. 32, no. 4, pp. 21-28, 1997.