A scatterplot of crystal mean resolution versus machine-learning score from the image, taken from all harvestable crystals in the set that had measured data.

A graph derived from the number of solved structures associated with encountered images as a function of the estimated total human annotation time of 88 h. The blue dashed line represents the cutoff chosen in §3.2.

An illustration of how machine-learning scores assigned to images taken over different time periods of the same well increase over time. (a) A score of 0.15 at 7 d; (b) a score of 6.21 at 14 d; (c) a score of 9.01 at 28 d.

The scoring pipeline. (a) The original image. (b) An image stack obtained from image processing. (i) Heatmaps of Gabor responses. White areas represent pixels of high response. (ii) Heatmaps of orientation histograms. White areas represent square centers with high ‘largest bin value’ statistic. (c) Scanning the image and scoring each square. (i) Each square is associated with a feature vector encoding the values of 466 features. Each colored arrow is intended to represent a feature vector from one square subregion of the image. (ii) Each feature vector propagates differently through the alternating decision tree. (iii) A real-valued score is thereby associated with each feature vector. (d) The maximum score marked in red over all squares is taken as the image score.

A summary of the main experimental results. (a) Scoring and performance evaluation of an image set: (i) images start out unscored with human annotations; (ii) the algorithm scores the set, inducing a rank ordering on it; (iii) an ROC curve is derived from scores and ground truth. (b) The ROC heatmap, a simultaneous view of all individual ROC curves. For the purposes of ROC analysis, we treat diffraction candidates as true-positive examples and discarded trials as false-positive examples. Rows delineate individual curves ordered from top to bottom in descending order of ROC-AUC score. The intensity values of the heatmap represent true positive rates, with an overlay marking the location of images containing the diffraction success in blue. (c) Diffraction successes and their images: representative rectangles are shaded in the same color in (b) and (c) to show position in the heatmap. (i) Crystals of an XisH-family protein from Nostoc punctiforme PCC 73102 at 1.60 Å resolution (PDB code 2inb). (ii) Crystals of Bacillus cereus ATCC 10987 at 2.10 Å resolution (PDB code 2p1a). (iii) Crystals of methyltransferase FkbM from Methylobacillus flagellatus KT at 2.20 Å resolution (PDB code 2py6). Much of the crystal contours are occluded by precipitant. (iv) Crystals of HD superfamily hydrolase from uncultured Thermotogales bacterium at 1.45 Å resolution (PDB code 2pq7). Aside from a telltale line, the rest of the crystal contours are barely visible. (d) An ‘average’ ROC curve (red line) with upper and lower standard deviation bands (green lines). (e) A ‘worst-case’ ROC curve for various confidences p (red for p = 0.05, green for p = 0.10, blue for p = 0.20).