Identification of putative biomarkers of prion disease using MALDI FTMS profiling of tryptic peptides of CSF and heat map feature extraction. Representative mass spectra of hamster CSF, infected (A) and uninfected control (B), collected 18 weeks post-inoculation. The blow up boxes are zoomed on a peptide (m/z =2275.4) that is present in infected hamsters. The peak at m/z 2257.4 is the dehydrated peptide. The panel on the right shows the distribution of this biomarker among infected and control animals. Each dash represents one of nine individual spectra collected in replicates from 21 infected or 22 uninfected hamsters. Colored pixels to the right indicate detection of a peak at the given mass. Color heat is indicative of the relative intensity of the peak within its spectrum. The multiplicity of pixels is indicative of the peptide's isotopic distribution. The peptide is present in 15% of uninfected animals and 60% of infected animals. Although this feature can be differentiated in these two spectra, it is not consistently present in the infected or absent in the uninfected and is therefore cannot be used alone as an indicative feature, thus requiring the consideration of multiple features as for accurate diagnosis of disease state.

Feature number versus accuracy. As the number of features incorporated into the model increases, the accuracy of the disease class predication improves up to ∼100 features. After that, additional features provide no increase in overall accuracy.

Confusion Matrix and performance equations. A confusion matrix has four measurements as follows: True Positive – Sample is actually positive and correctly predicted as positive. True Negative – Sample is actually negative and correctly predicted as negative. False Positive – Sample is actually negative and incorrectly predicted as positive. False Negative – Sample is actually positive and incorrectly predicted as negative. From this confusion matrix we can now describe a number of overall performance measures: Accuracy – Overall predictive accuracy of model, Recall – Ratio of positives correctly predicted, also sensitivity, Precision – Of positives predicted, how many were actually positive, F1-Score – 1^st moment between precision and recall, False Positive Rate – Ratio of false positives predicted (specificity is 1-FPR).

Receiver Operator Characteristic Curve. Plotting the true positive rate (sensitivity) as a function of the false positive rate (specificity) allows visualization of the trade off between the two. The light blue line is the line of no discrimination, a derived from a theoretically random classification. The classification models for discriminating disease state (linear SVM and Naïve Bayes) reside in the upper left quadrant indicating a signal of disease. The light green line is derived from a randomization of the data set and “dances” around the theoretical random line indicating a high level of noise and an inability to discriminate between the two classes.

Permutation test of SVM classifications. One thousand arbitrary classifications were performed on one thousand permuted data sets generated by randomization. The plot shows the predictive accuracy of each classification obtained by attempting machine learning on the randomized data sets and estimates the probability of obtaining a predictive accuracy of 72% (p<0.007), the accuracy at which prion disease was able to be distinguished from control.

Feature selection by information gain. A, B, C, heat maps displaying the top 30 bins ranked by information gain scores. A blue pixel indicates the presence of a peak in three out of the nine subsamples from each animal. A. Top 30 features distinguishing infected from uninfected animals. B, C, Samples are randomized at the animal (B) and spectra (C) level and the top 30 features are sorted by information gain. D. Plot of all 1,500 bins and their associated information gain score. Note that classification based upon disease state yields the best information gain scores.