Format

Send to

Choose Destination
Neural Netw. 2018 Aug;104:26-39. doi: 10.1016/j.neunet.2018.04.004. Epub 2018 Apr 16.

A frequency-domain approach to improve ANNs generalization quality via proper initialization.

Author information

1
Department of Electrical and Computer Engineering, University of Louisiana at Lafayette, P.O. Box 43890, Lafayette, LA 70504, USA. Electronic address: mxc0798@louisiana.edu.
2
Department of Electrical and Computer Engineering, University of Louisiana at Lafayette, P.O. Box 43890, Lafayette, LA 70504, USA. Electronic address: afef.fekih@louisiana.edu.
3
Department of Petroleum Engineering, University of Louisiana at Lafayette, P.O. Box 44690, Lafayette, LA 70504, USA. Electronic address: acs9955@louisiana.edu.
4
Department of Mechanical Engineering, University of Louisiana at Lafayette, P.O. Box 43678, Lafayette, LA 70504, USA. Electronic address: jxb9360@louisiana.edu.

Abstract

The ability to train a network without memorizing the input/output data, thereby allowing a good predictive performance when applied to unseen data, is paramount in ANN applications. In this paper, we propose a frequency-domain approach to evaluate the network initialization in terms of quality of training, i.e., generalization capabilities. As an alternative to the conventional time-domain methods, the proposed approach eliminates the approximate nature of network validation using an excess of unseen data. The benefits of the proposed approach are demonstrated using two numerical examples, where two trained networks performed similarly on the training and the validation data sets, yet they revealed a significant difference in prediction accuracy when tested using a different data set. This observation is of utmost importance in modeling applications requiring a high degree of accuracy. The efficiency of the proposed approach is further demonstrated on a real-world problem, where unlike other initialization methods, a more conclusive assessment of generalization is achieved. On the practical front, subtle methodological and implementational facets are addressed to ensure reproducibility and pinpoint the limitations of the proposed approach.

KEYWORDS:

Artificial intelligence; Frequency response function; Multilayered perceptron; Network generalization; Network initialization

PMID:
29705668
DOI:
10.1016/j.neunet.2018.04.004
[Indexed for MEDLINE]

Supplemental Content

Full text links

Icon for Elsevier Science
Loading ...
Support Center