Format

Send to

Choose Destination
Neural Comput. 2002 Dec;14(12):2791-846.

On different facets of regularization theory.

Author information

1
Adaptive Systems Lab, Communications Research Laboratory, McMaster University, Hamilton, Ontario, Canada L8S 4K1. zhechen@soma.crl.mcmaster.ca

Abstract

This review provides a comprehensive understanding of regularization theory from different perspectives, emphasizing smoothness and simplicity principles. Using the tools of operator theory and Fourier analysis, it is shown that the solution of the classical Tikhonov regularization problem can be derived from the regularized functional defined by a linear differential (integral) operator in the spatial (Fourier) domain. State-of-the-art research relevant to the regularization theory is reviewed, covering Occam's razor, minimum length description, Bayesian theory, pruning algorithms, informational (entropy) theory, statistical learning theory, and equivalent regularization. The universal principle of regularization in terms of Kolmogorov complexity is discussed. Finally, some prospective studies on regularization theory and beyond are suggested.

PMID:
12487794
DOI:
10.1162/089976602760805296

Supplemental Content

Full text links

Icon for Atypon
Loading ...
Support Center