The International Diagnostic Checklists for the assessment of the DSM-III-R and ICD-10 personality disorders (IDCL-P) were compared with a structured interview, the International Personality Disorder Examination (IPDE), using a balanced test-retest design with forty psychiatric inpatients. The results, using pairwise kappa for the calculation of agreement, were as follows: any personality disorder versus no personality disorder 0.52 for DSM-III-R diagnoses and 0.75 for ICD-10 diagnoses. The range for the single personality disorders diagnosed at least five times was from -0.07 to 0.71 for DSM-III-R diagnoses and from 0.38 to 0.68 for ICD-10 diagnoses. Only for DSM-III-R diagnoses do figures exist from three other studies comparing two structured interviews with each other. The results of all four studies suggest that 60% of the variance in personality disorder diagnoses represents variance not attributable to the patients, which is scientifically unacceptable.