Format

Send to

Choose Destination
Biometrics. 2007 Dec;63(4):1031-7. Epub 2007 Apr 9.

Determining the number of clusters using the weighted gap statistic.

Author information

1
Medtronic Sofamor Danek, 1800 Pyramid Place, Memphis, Tennessee 38132, USA. mingjin.yan@medtronic.com

Abstract

Estimating the number of clusters in a data set is a crucial step in cluster analysis. In this article, motivated by the gap method (Tibshirani, Walther, and Hastie, 2001, Journal of the Royal Statistical Society B63, 411-423), we propose the weighted gap and the difference of difference-weighted (DD-weighted) gap methods for estimating the number of clusters in data using the weighted within-clusters sum of errors: a measure of the within-clusters homogeneity. In addition, we propose a "multilayer" clustering approach, which is shown to be more accurate than the original gap method, particularly in detecting the nested cluster structure of the data. The methods are applicable when the input data contain continuous measurements and can be used with any clustering method. Simulation studies and real data are investigated and compared among these proposed methods as well as with the original gap method.

[Indexed for MEDLINE]

Supplemental Content

Full text links

Icon for Wiley
Loading ...
Support Center