• We are sorry, but NCBI web applications do not support your browser and may not function properly. More information
Logo of cdliPermissionsJournals.ASM.orgAEM ArticleJournal InfoAuthorsReviewers
Clin Diagn Lab Immunol. Nov 2002; 9(6): 1367–1371.
PMCID: PMC130105

Interpretation of a Commercial Bovine Paratuberculosis Enzyme-Linked Immunosorbent Assay by Using Likelihood Ratios

Abstract

Evidence-based medicine encourages the use of quantitative diagnostic test results to estimate the probability of a particular diagnosis. Likelihood ratios (LRs) are among the best tools for maximizing the diagnostic information gained from diagnostic assays that provide results on a continuous scale. They provide the odds that an animal with a particular test result actually has the disease in question based on the magnitude of the test result. A commercial enzyme-linked immunosorbent assay (ELISA) was used to test sera from 143 dairy cattle infected with Mycobacterium paratuberculosis and 2,974 cattle free of this infection. This assay transforms ELISA reader optical density values into sample-to-positive (S/P) ratios. The LR was calculated for S/P results from 0.00 to 1.00 at 0.05-S/P unit intervals. LRs were directly but not linearly correlated with ELISA S/P ratios (r2 = 0.94). The mathematical function describing the relationship between the ELISA S/P ratio and the LR was LR = 265 × (S/P value)2.03. LRs were also directly related to the frequency of animals testing positive for paratuberculosis by fecal culture and other serologic tests. Based on these LRs, guidelines for interpretation and application of this ELISA for the diagnosis and control of paratuberculosis in dairy cattle herds are recommended.

The principal goal of a diagnostic test is to help practitioners increase the probability of a correct diagnosis. Predictive values are useful in this regard but require the estimation of disease prevalence (i.e., pre-test probability of disease) in the population (27) and only utilize categorical test results, positive or negative. Food-producing animals exist in numerous discrete populations (herds or flocks), and the prevalence of disease can differ greatly among them. Infection prevalence significantly affects predictive values, i.e., the positive predictive value of tests is low when prevalence is low, and the negative predictive value is low when disease prevalence is high, thus the predictive values of tests can vary significantly among herds.

Evidence-based medicine (EBM) is the scrupulous, explicit, and judicious use of the best evidence available in making decisions about the care of individual patients. The practice of EBM means integrating clinical expertise with the best available clinical evidence from systematic diagnostic research (19). Additional information on EBM is available at the Learning and Information Services website on EBM (http://www.herts.ac.uk/lis/subjects/health/ebm.htm). The likelihood ratio (LR) is one example of external clinical evidence and a powerful tool in EBM. LRs give the same information as predictive values but can be used independent of pre-test disease prevalence (5).

The purpose of the study was to generate an algorithm for the calculation of LRs for subclinical Mycobacterium paratuberculosis infection in dairy cattle (Johne's disease) based on an enzyme-linked immunosorbent assay (ELISA) for serum antibodies and to show its utility in the control of this economically important and potentially zoonotic infectious disease (1, 6, 10, 17).

MATERIALS AND METHODS

Serum samples.

Samples originated from two well-defined dairy cattle populations. Sera from 143 subclinically M. paratuberculosis-infected cows were part of a previously described specimen repository (23). The case definition for M. paratuberculosis-infected cattle was the isolation of M. paratuberculosis by fecal culture and/or histopathologic evidence of infection. Sera from uninfected cows included 760 samples from U.S. dairy cattle and 2,214 samples from Dutch dairy cattle. These cattle were from herds free of M. paratuberculosis infection as defined by a minimum of three negative annual whole-herd (all cattle, ≥2 years old) fecal cultures.

ELISA.

An M. paratuberculosis antibody test kit (IDEXX Laboratories, Inc., Westbrook, Maine) was used to test all 3,117 sera according to the manufacturer's instructions. With this kit, optical density (OD) values were transformed to S/P ratios based on the OD for the serum sample together with those for the negative and positive controls provided with the kit by using the following equation: S/P ratio = (OD of sample − OD of negative control)/(OD of positive control − OD of negative control). All assays were run in duplicate. Any assay with a between-well coefficient of variation of >10% was repeated, and the second result was used for data analysis.

Other tests for paratuberculosis.

ELISA results were compared to those of other tests for paratuberculosis run on samples collected at the same time from the same cattle. Fecal culture was done both by the BACTEC system (2) and by using conventional solid medium (Herrold's egg yolk agar) (29). Serum antibody measurements were done by the complement fixation test (12), agar gel immunodiffusion assay (20), and another commercial ELISA kit (Paracheck; Biocor Animal Health, Omaha, Nebr.) (3). These tests were performed independently and simultaneously at the time the original samples were collected. Results of these other diagnostic tests for paratuberculosis were reported previously and not done specifically for the present study (22-24).

Data analysis.

Frequency distributions for S/P values on the sera from infected and noninfected populations were tabulated in intervals of 0.05 S/P units. At each interval, the sensitivity (Se), specificity (Sp), and LR [Se/(1−Sp)] of the ELISA were calculated. Linear and nonlinear regression analysis was used to determine the equation describing the line best fitting the plot of the S/P cutoff value versus LR (Lotus Freelance Graphics release 9.6 for Windows; Lotus Development Corp.). The regression model providing the highest r2 value was considered to best fit the data.

RESULTS

There was a small difference in ELISA false-positive rates between the U.S. and Dutch cattle free of M. paratuberculosis; maximum ELISA S/P values were 1.15 and 1.10 for the two groups, respectively, and the Sp values at the manufacturer's recommended S/P cutoff values of 0.25 were 98.7 and 96.3% for the U.S. and Dutch M. paratuberculosis-free cattle, respectively. Pooling the data for these two populations improved the precision of LR estimates and potentially made the findings more universally applicable across countries.

Estimated Se, Sp, and LR increased with increasing ELISA S/P cutoff values (Table (Table1).1). Regression analysis of ELISA S/P ratio category versus LR was attempted using linear, exponential, logarithmic, and power functions. The S/P-to-LR relationship that best fit the data was described by the power function LR = 265 × (S/P value)2.03. This function fitted the data with an r2 value of 0.94 (Fig. (Fig.1).1). Other regression functions had far lower r2 values.

FIG. 1.
Nonlinear regression of ELISA S/P values and LRs (bold line) with 95% confidence intervals (thin lines).
TABLE 1.
M. paratuberculosis serum antibody ELISA Se, Sp, and LR values by ELISA S/P value

When the 143 M. paratuberculosis-infected cattle were arbitrarily clustered into five groups according to ELISA S/P levels (levels found useful based on clinical experience), a relationship between the magnitude of ELISA S/P value and the rate at which cattle tested positive on other tests for paratuberculosis in historical data was apparent (Table (Table22).

TABLE 2.
Percentage of positive results by other diagnostic methods when M. paratuberculosis-infected cattle were grouped by ELISA S/P range

DISCUSSION

Control of paratuberculosis in dairy cattle herds requires hygienic measures to limit opportunities for transmission of the infection from cows to calves in combination with the management of cattle most likely to be infectious (7, 8, 28). Infected cattle should not provide colostrum or milk to calves, and their manure should not be allowed to contaminate feed, water, or the environment. This is particularly true for the pens in which calves are born and the location on the farm where calves are raised. In addition, as many of the infected cows as possible should be culled from the herd when economically feasible. Because the majority of M. paratuberculosis-infected cattle are infectious (shedding the organism in their manure, colostrum, and milk) but clinically normal, laboratory diagnostics are needed to identify them.

Culture of feces to diagnose M. paratuberculosis infection with conventional culture media, such as Herrold's egg yolk agar, requires 8 to 16 weeks (29). Laboratories typically charge $12 to 25 per sample. A liquid-culture-based detection system such as the Trek ESP system and the BACTEC system are able to shorten the detection time to 4 to 8 weeks but are not less costly than conventional culture when the costs of isolate identification are considered (2, 9). Genetic M. paratuberculosis detection technology coupled with PCR methods theoretically should enhance detection Se and considerably shorten the time to detection. However, commercial tests have yet to attain the analytical Se of culture methods, are roughly twice as expensive, and are difficult to scale up for handling large sample numbers (30).

Serology provides a cost-effective alternative to organism detection-based diagnostic methods for bovine paratuberculosis. ELISA-based methods have the highest Se of serologic tests for paratuberculosis (24), plus they offer the kind of low-cost and high-throughput process (>1,000/day) needed to serve the dairy industry. A disadvantage of ELISAs for paratuberculosis is that assay Sp is less than that for fecal culture (considered to be 100%) (13, 25, 31) and the economic consequences of mistakenly culling a cow due to false-positive test results are high (roughly $1,300/cow based on the average price of replacement cattle in the United States in 2001 and the average salvage value of culled dairy cow).

Traditional ELISA interpretation is dichotomous (positive or negative) based on a single-assay cutoff value designed to optimize assay Se and Sp. Use of multilevel LRs capitalizes on the assay's ability to report results on a continuous scale, thereby enhancing the amount of diagnostic information gained. LRs quantify the probability of an accurate diagnosis. Diagnostic probabilities generated from LRs, with or without use of pre-test probabilities, can be integrated with the economic impact of actions taken based on test results such as culling. Thurmond et al. nicely demonstrated this by an ELISA for Neospora caninum infection in dairy cattle (26).

LRs are derived from Se and Sp estimates. These traditional measures of test accuracy for infectious diseases are influenced by the gold standards for the definition of infection and absence of infection in the tested populations and the spectrum of disease in the infected population (14). The standard for diagnosis of infection used in the present study was in the isolation of M. paratuberculosis from a fecal or tissue sample, a widely accepted standard. The standard for freedom of infection was the residence of the animal in a herd certified free of infection based on multiple independent (nonserologic) tests spanning several years. It is inappropriate to use test-negative cattle resident in M. paratuberculosis-infected herds for the estimation of diagnostic Sp because the chance of erroneously considering the animal noninfected is too great, i.e., such a “gold standard” is imperfect and inappropriate.

The spectrum of disease in the M. paratuberculosis-infected animals used in the present study was typical of that found in clinically normal adult Holstein cows raised in infected herds in Wisconsin, the type of herds in which ELISAs are performed. Clinical samples from these same cattle have been used in the evaluation of other diagnostic tests for paratuberculosis (22-24). Se and Sp estimates found using the single manufacturer's S/P cutoff of 0.25 (45 and 97%, respectively) are similar to those reported by other investigators, suggesting that the spectrum of disease was at least comparable to that of animals typically used for paratuberculosis serologic test evaluations (11, 16, 25, 32). Arguably, these cases of paratuberculosis may not be typical of all truly infected cattle and do not include M. paratuberculosis-infected cattle that test negative by all available diagnostic methods; however, they were selected without bias for any single diagnostic test. Calculated LRs would be somewhat lower or higher if the spectrum of disease in the infected population was biased toward early- or very-late-stage infections, respectively.

Diagnostic laboratory medicine for animal agriculture is driven more by economics than is companion animal or human diagnostic laboratory medicine. Containment of testing costs and consideration of the economic consequences of the actions resulting from the diagnostic results are critical considerations in deciding which laboratory services to offer. These are factors that veterinary practitioners must consider in deciding what test to use in which circumstances.

The ELISA evaluated in this study produced quantitative results that were directly related to the likelihood that a dairy cow was infected with M. paratuberculosis (Table (Table1)1) and to the rate of positive results by using other diagnostic tests for paratuberculosis detection (Table (Table2).2). While the use of LRs in clinical epidemiology has long been advocated (18), few clinicians actually use them to estimate diagnostic probability (15). Additionally, as Feinstein nicely points out in his recent review, “the most important roles of technological tests today are in non-diagnostic clinical decisions,” e.g., estimating prognosis (4). Acknowledging this, application of LR data has been simplified for practitioners by the creation of five categories of ELISA interpretation coupled to a recommended action scheme for paratuberculosis control in dairy herds (Table (Table3).3). These categories were derived somewhat arbitrarily but take into consideration the magnitude of the LRs and clinical experience with infected dairy herds. This scheme effectively lowers the cutoff for the identification of high-risk cattle to an S/P value of ≥0.10 (the manufacturer's S/P cutoff for a positive test is 0.25) and couples this with recommendations for low-cost interventions involving cows with such results to limit the spread of infection (segregated calving pens and discarding of colostrum). It also functionally raises the cutoff for cattle that are recommended to be culled from the herd to an S/P value of ≥1.00, whereas the ELISA Sp is ≥99.9% (Table (Table1),1), thereby limiting the rate of false-positive results and thus the economic impact on the herd owner caused by the mistaken sale and replacement of a noninfected cow. In this way, veterinary practitioners and dairy producers have a simple scheme for ELISA use in herds that is based on EBM and LRs without need of calculations or nomograms. Whether such a system will effectively work to control paratuberculosis in dairy herds is the subject of on going studies.

TABLE 3.
Application of LR to the interpretation and use of ELISA results in M. paratuberculosis-infected dairy herdsa,b

Practitioners may wish to incorporate pre-test probabilities of infection (the estimated within-herd paratuberculosis prevalence) together with the magnitude of the M. paratuberculosis ELISA result, translated into an LR, for a more precise estimation of the post-test probability of M. paratuberculosis infection. Table Table44 illustrates the interaction of pre-test probability and ELISA S/P value on the post-test probability of M. paratuberculosis infection diagnosis. Other serologic tests for veterinary use could and should be interpreted based on LRs.

TABLE 4.
Effect of pre-test probability and ELISA S/P value on the post-test probabilitya of M. paratuberculosis infection

Acknowledgments

This work was funded by the Johne's Testing Center, School of Veterinary Medicine, University of Wisconsin.

The donation of serum samples from Dutch dairy cattle by Kees Kalis is gratefully acknowledged.

REFERENCES

1. Chamberlin, W., D. Y. Graham, K. Hulten, H. M. T. El Zimaity, M. R. Schwartz, S. Naser, I. Shafran, and F. A. K. El Zaatari. 2001. Mycobacterium avium subsp. paratuberculosis as one cause of Crohn's disease. Aliment. Pharmacol. Ther. 15:337-346. [PubMed]
2. Collins, M. T., K. B. Kenefick, D. C. Sockett, R. S. Lambrecht, J. McDonald, and J. B. Jørgensen. 1990. Enhanced radiometric detection of Mycobacterium paratuberculosis using filter concentrated fecal specimens. J. Clin. Microbiol. 28:2514-2519. [PMC free article] [PubMed]
3. Cox, J. C., D. P. Drane, S. L. Jones, S. Ridge, and A. R. Milner. 1991. Development and evaluation of a rapid absorbed enzyme immunoassay test for the diagnosis of Johne's disease in cattle. Aust. Vet. J. 68:157-160. [PubMed]
4. Feinstein, A. R. 2002. Misguided efforts and future challenges for research on “diagnostic tests.” J. Epidemiol. Community Health 56:330-336. [PMC free article] [PubMed]
5. Giocoli, G. 2000. Evidence-based clinical microbiology. J. Clin. Microbiol. 38:3520-3521. [PMC free article] [PubMed]
6. Hermon-Taylor, J., T. J. Bull, J. M. Sheridan, J. Cheng, M. L. Stellakis, and N. Sumar. 2000. The causation of Crohn's disease by Mycobacterium avium subspecies paratuberculosis. Can. J. Gastroenterol. 14:521-539. [PubMed]
7. Johnson-Ifearulundu, Y. J., and J. B. Kaneene. 1998. Management-related risk factors for M. paratuberculosis infection in Michigan, USA, dairy herds. Prev. Vet. Med. 37:41-54. [PubMed]
8. Kennedy, D. J., and G. Benedictus. 2001. Control of Mycobacterium avium subsp. paratuberculosis infection in agricultural species. Rev. Sci. Tech. O. I. E. (Off. Int. Epizoot.) 20:151-179. [PubMed]
9. Kim, S. G., S. J. Shin, R. H. Jacobson, L. J. Miller, P. R. Harpending, S. M. Stehman, C. A. Rossiter, and D. A. Lein. 2002. Development and application of quantitative polymerase chain reaction assay based on the ABI 7700 system (TaqMan) for detection and quantification of Mycobacterium avium subsp. paratuberculosis. J. Vet. Diagn. Investig. 14:126-131. [PubMed]
10. Manning, E. J. B., and M. T. Collins. 2001. Mycobacterium avium subsp. paratuberculosis: pathogen, pathogenesis and diagnosis. Rev. Sci. Tech. O. I. E. (Off. Int. Epizoot.) 20:133-150. [PubMed]
11. Milner, A. R., W. N. Mack, K. J. Coates, J. Hill, I. Gill, and P. Sheldrick. 1990. The sensitivity and specificity of a modified ELISA for the diagnosis of Johne's disease from a field trial in cattle. Vet. Microbiol. 25:193-198. [PubMed]
12. National Veterinary Services Laboratory Staff. 1986. Microtitration complement fixation techniques, p. 1-18. National Veterinary Services Laboratory, Ames, Iowa.
13. Nielsen, S. S., H. Houe, S. M. Thamsborg, and V. Bitsch. 2001. Comparison of two enzyme-linked immunosorbent assays for serologic diagnosis of paratuberculosis (Johne's disease) in cattle using different subspecies strains of Mycobacterium avium. J. Vet. Diagn. Investig. 13:164-166. [PubMed]
14. Ransohoff, D. F., and A. R. Feinstein. 1978. Problems of spectrum and bias in evaluating the efficacy of diagnostic tests. N. Engl. J. Med. 299:926-930. [PubMed]
15. Reid, M. C., D. A. Lane, and A. R. Feinstein. 1998. Academic calculations versus clinical judgments: practicing physicians' use of quantitative measures of test accuracy. Am. J. Med. 104:374-380. [PubMed]
16. Ridge, S. E., I. R. Morgan, D. C. Sockett, M. T. Collins, R. J. Condron, N. W. Skilbeck, and J. J. Webber. 1991. Comparison of the Johne's absorbed EIA and the complement fixation test for the diagnosis of Johne's disease in cattle. Aust. Vet. J. 68:253-257. [PubMed]
17. Rubery, E. 2001. A review of evidence for a link between exposure to Mycobacterium paratuberculosis (MAP) and Crohn's disease (CD) in humans, p. 1-55. Food Standards Agency, London, United Kingdom.
18. Sackett, D. L., R. B. Haynes, G. H. Guyatt, and P. Tugwell. 1991. The interpretation of diagnostic data, p. 69-152. In Clinical epidemiology. A basic science for clinical medicine. Little, Brown and Company, Boston, Mass.
19. Sackett, D. L., S. E. Strauss, W. S. Richardson, W. Rosenberg, and R. B. Haynes. 2000. Evidence-based medicine. How to practice and teach EBM, p. 1-261. Churchill Livingstone, Edinburgh, Scotland.
20. Sherman, D. M., B. Bray, J. M. Gay, and F. Bates. 1989. Evaluation of the agar gel immunodiffusion test for the diagnosis of subclinical paratuberculosis. Am. J. Vet. Res. 50:525-530. [PubMed]
21. Sherman, D. M., R. J. F. Markham, and F. Bates. 1984. Agar gel immunodiffusion test for diagnosis of clinical paratuberculosis in cattle. J. Am. Vet. Med. Assoc. 185:179-182. [PubMed]
22. Sockett, D. C., D. J. Carr, and M. T. Collins. 1992. Evaluation of conventional and radiometric fecal culture and a commercial DNA probe for diagnosis of Mycobacterium paratuberculosis infections in cattle. Can. J. Vet. Res. 56:148-153. [PMC free article] [PubMed]
23. Sockett, D. C., D. J. Carr, W. D. Richards, and M. T. Collins. 1992. A repository of specimens for comparison of the diagnostic testing procedures for bovine paratuberculosis. J. Vet. Diagn. Investig. 4:188-192. [PubMed]
24. Sockett, D. C., T. A. Conrad, C. B. Thomas, and M. T. Collins. 1992. Evaluation of four serological tests for bovine paratuberculosis. J. Clin. Microbiol. 30:1134-1139. [PMC free article] [PubMed]
25. Sweeney, R. W., R. H. Whitlock, C. L. Buckley, and P. A. Spencer. 1995. Evaluation of a commercial enzyme-linked immunosorbent assay for the diagnosis of paratuberculosis in dairy cattle. J. Vet. Diagn. Investig. 7:488-493. [PubMed]
26. Thurmond, M. C., W. O. Johnson, C. A. Munoz-Zanzi, C. L. Su, and S. K. Hietala. 2002. A method of probability diagnostic assignment that applies Bayes theorem for use in serologic diagnostics, using an example of Neospora caninum infection in cattle. Am. J. Vet. Res. 63:318-325. [PubMed]
27. Vecchio, T. J. 1966. Predictive value of a single diagnostic test in unselected populations. N. Engl. J. Med. 274:1171-1173. [PubMed]
28. Wells, S. J., and B. A. Wagner. 2000. Herd-level risk factors for infection with Mycobacterium paratuberculosis in US dairies and association between familiarity of the herd manager with the disease or prior diagnosis of the disease in that herd and use of preventive measures. J. Am. Vet. Med. Assoc. 216:1450-1457. [PubMed]
29. Whipple, D. L., D. R. Callihan, and J. L. Jarnagin. 1991. Cultivation of Mycobacterium paratuberculosis from bovine fecal specimens and a suggested standardized procedure. J. Vet. Diagn. Investig. 3:368-373. [PubMed]
30. Whipple, D. L., P. A. Kapke, and P. R. Andersen. 1992. Comparison of a commercial DNA probe test and three cultivation procedures for detection of Mycobacterium paratuberculosis in bovine feces. J. Vet. Diagn. Investig. 4:23-27. [PubMed]
31. Whitlock, R. H., S. J. Wells, R. W. Sweeney, and J. Van Tiem. 2000. ELISA and fecal culture for paratuberculosis (Johne's disease): sensitivity and specificity of each method. Vet. Microbiol. 77:387-398. [PubMed]
32. Yokomizo, Y. 1986. Evaluation on an enzyme-linked immunosorbent assay (ELISA) using Mycobacterium phlei-absorbed serum for the diagnosis of bovine paratuberculosis in a field study. Jpn. Ag. Res. Q. 20:60-67.

Articles from Clinical and Diagnostic Laboratory Immunology are provided here courtesy of American Society for Microbiology (ASM)
PubReader format: click here to try

Formats:

Related citations in PubMed

See reviews...See all...

Cited by other articles in PMC

Links

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...