Decisions marked with a double line indicate those that can be affected by DSSs and the broken lines mark processes that can be affected by ITs
 |
|
|
The Agency for Healthcare Research and Quality (AHRQ), through its Evidence-Based Practice Centers (EPCs), sponsors the development of evidence reports and technology assessments to assist public- and private-sector organizations in their efforts to improve the quality of health care in the United States. The reports and assessments provide organizations with comprehensive, science-based information on common, costly medical conditions and new health care technologies. The EPCs systematically review the relevant scientific literature on topics assigned to them by AHRQ and conduct additional analyses when appropriate prior to developing their reports and assessments.
To bring the broadest range of experts into the development of evidence reports and health technology assessments, AHRQ encourages the EPCs to form partnerships and enter into collaborations with other medical and research organizations. The EPCs work with these partner organizations to ensure that the evidence reports and technology assessments they produce will become building blocks for health care quality improvement projects throughout the Nation. The reports undergo peer review prior to their release.
AHRQ expects that the EPC evidence reports and technology assessments will inform individual health plans, providers, and purchasers as well as the health care system as a whole by providing important information to help improve health care quality.
We welcome written comments on this evidence report. They may be sent to: Director, Center for Practice and Technology Assessment, Agency for Healthcare Research and Quality, 6010 Executive Blvd., Suite 300, Rockville, MD 20852.
| Carolyn M. Clancy, M.D. | Robert Graham, M.D., Director |
| Acting Director | Center for Practice and |
| Agency for Healthcare Research | Technology Assessment |
| and Quality | Agency for Healthcare Research |
| and Quality |
| The authors of this report are responsible for its content. Statements in the report should not be construed as endorsement by the Agency for Healthcare Research and Quality or the U.S. Department of Health and Human Services of a particular drug, device, test, treatment, or other clinical service. |
During the preparation of this Report, Dr. Owens was supported by the Department of Veterans Affairs.
We are grateful to our expert advisors for their thoughtful comments and guidance. We would like to give special thanks to Sara H. Cody, M.D., Santa Clara County Communicable Disease Officer, for her many valuable suggestions informed by her front line experiences. We thank the Stanford University research librarians who helped us design our search strategies: Rikke Greenwald (Lane Medical Library), Ann Latta (Social Sciences Resource Center), Joan Loftus (U.S. Government Documents Bibliographer) and Michael Newman (Falconer Biology Library). Finally, we extend our thanks to Cynthia Yock for her help reviewing articles, and Amy J. Markowitz, Susan Nguyen, Lyn Dupr?, and Vandana Sundaram for their careful editorial review.
This project aimed to synthesize the evidence on information technologies and decision support systems (IT/DSSs) that may serve the information needs of clinicians and public health officials in the event of bioterrorism.
To direct literature searches, a conceptual model was developed that specifies the decisions and tasks of clinicians and public health officials in the event of bioterrorism. Searches of MEDLINE® and of other relevant databases for articles describing or evaluating potentially relevant IT/DSSs were performed. Additional references were found from Internet searches (including 16 government agency Web sites), and bibliographies of retrieved articles.
IT/DSSs were included that could potentially support the detection, diagnosis, management, prevention, treatment, guideline implementation, surveillance, reporting, and communication of information during a response to bioterrorism.
All peer-reviewed articles that met the inclusion criteria were blinded to the study authors, and 2 investigators independently abstracted study information. Information from Web sites was abstracted by a single investigator.
More than 20,000 citations and Web sites were reviewed. Of these, 251 articles, 36 government Web sites, and 54 non-government Web sites met selection criteria. From these, 217 IT/DSSs of potential use by clinicians and public health officials in the event of bioterrorism were described. They include 55 detection systems, 23 diagnostic systems, 18 management systems, 90 surveillance systems, 26 communication systems, and 7 systems that integrate surveillance, communication, and command and control functions. Most reports only described IT/DSSs; however, 79 studies evaluated 58 systems for at least 1 performance metric (e.g., timeliness). Few systems have been subjected to comprehensive evaluation. The sensitivity and specificity of rapid detection systems is not generally publicly available, complicating the interpretation of test results. None of the general diagnostic or management systems has been evaluated with respect to bioterrorism response. Syndromal surveillance systems collecting a variety of surveillance data have been deployed for both event-based and continuous bioterrorism surveillance, and evaluations are ongoing. Web-based communication systems are increasingly in use, but few have been formally evaluated. Current national efforts of particular promise include those to develop and evaluate systems that integrate the collection, analysis, and presentation of data from detectors, clinicians, laboratories, and hospitals to public health decision makers.
IT/DSSs have the potential to help clinicians and public health officials make better decisions regarding detection, diagnosis, management, prevention, surveillance, and communication during a bioterrorism event. However, few of these systems have been evaluated rigorously, and most were not specifically designed to address threats from bioterrorism. Furthermore, many of the systems have not been described in peer-reviewed literature. The lack of evaluative studies creates difficulties in assessing the usefulness of IT/DSSs. We note, however, that lack of evidence about effectiveness is not evidence for lack of effectiveness. Many of the systems we reviewed may be useful for response to bioterrorism and are reasonable candidates for further evaluation. Such evaluations would clarify their value both for response to bioterrorism and for the other purposes for which they were designed.
The Nation's capacity to respond to bioterrorism depends in part on the ability of clinicians and public health officials to detect, manage, and communicate during a bioterrorism event. Information technologies and decision support systems (IT/DSSs) have the potential to aid clinicians (e.g., physicians, nurses, nurse practitioners, and respiratory therapists) and public health officials respond effectively to a bioterrorist attack.
This Evidence Report details the methodology, results, and conclusions of a systematic and extensive search for published materials on the use of IT/DSSs to serve the information needs of clinicians and public health officials in the event of a bioterrorist attack. The information is intended to assist clinicians, public health officials, and policymakers improve preparedness for a bioterrorism event.
The University of California at San Francisco-Stanford Evidence-based Practice Center staff, in conjunction with a panel of expert advisors and the Agency for Healthcare Research and Quality, developed 4 Key Questions to be addressed in this Report. The Key Questions are:
Key Question 1: What are the information needs of clinicians and public health officials in the event of a bioterrorist attack?
Key Question 2: Based on the information needs identified for these decision makers, what are the criteria by which IT/DSSs should be evaluated with respect to usefulness during a bioterrorism event?
Key Question 3: When assessed by these criteria, in what ways could existing IT/DSSs be useful during a bioterrorism event? In what ways are they limited?
Key Question 4: In areas where existing IT/DSSs do not meet the information needs of clinicians or public health officials, what functional and technical considerations are important in the design of future IT/DSSs to support response to bioterrorism events?
We developed a conceptual model to specify the decisions and tasks involved in diagnosis, management, prevention, surveillance, and communication by clinicians and public health officials in the event of a bioterrorist attack. We used a process called task decomposition to specify the data requirements that need to be incorporated into an IT/DSS for it to assist clinicians and public health officials make these decisions. This list of tasks and data requirements served as the foundation of our evaluation system of the currently available IT/DSSs.
Based on input from our expert advisory panel, the conceptual model, the task decomposition, and practical considerations, we developed an inclusion-exclusion strategy to identify articles that described or evaluated IT/DSSs.
We used a scale developed at McMaster University to rate the quality of evidence from peer-reviewed evaluations of IT/DSSs for diagnosis, management, and communication. For reports of surveillance systems, we used an evaluation scale published by the Centers for Disease Control and Prevention (CDC).
In consultation with professional research librarians, we developed a search strategy for references from 3 sources: peer-reviewed articles, government reports, and Web-based information. For the peer-reviewed articles, we identified 5 databases of medical, scientific, and government references likely to contain reports of relevant IT/DSSs: MEDLINE® (January 1985 to April 2001), the Catalog of U.S. Government Publications, GrayLIT, the Library of Congress, and the National Technical Information Service. We identified the 16 government agencies most likely to fund, develop, or use IT/DSSs that could also be used by clinicians or public health officials. We also planned Internet searches to retrieve reports of potentially relevant IT/DSSs from sites other than those operated by government agencies (e.g., academic and commercial sites).
We developed separate search strategies, one for MEDLINE® and one for the government documents and Web-based information. Each included terms such as bioterrorism, biological warfare, information technology, decision support system, diagnosis, management, therapeutics, communication, surveillance, public health, and epidemiology. Additional articles were identified by members of our expert advisory panel, from conference proceedings, and by review of reference lists.
We reviewed titles, abstracts, and full-length articles as necessary to identify potentially relevant articles. All peer-reviewed articles that met our inclusion criteria were blinded to the investigators, 2 of whom independently abstracted study information onto a data-abstraction and quality-assessment form.
We abstracted the following data from all included articles: the purpose and description of the system (e.g., detection, diagnosis, management, surveillance, or communication), whether the system had been clinically evaluated and the results of these evaluations, what security measures the system uses, what kind of reasoning the system uses, and information about the quality of the report. A draft Evidence Report was critiqued by 16 expert advisors and 17 peer-reviewers who had expertise in nursing, clinical medicine, public health, hospital management, informatics, diagnostics, emergency management, epidemiology, national security, toxicology, and food safety.
We reviewed a total of 16,888 citations of peer-reviewed articles, 7,685 Web sites of government agencies, and 1,107 non-government Web sites. Of these, 251 articles, 36 government Web sites, and 54 non-government Web sites met our inclusion criteria. From these, we abstracted descriptions of 217 IT/DSSs of potential use by clinicians and public health officials in the event of a bioterrorist attack. They are comprised of 55 detection systems, 23 diagnostic systems, 18 management systems, 90 surveillance systems, 26 communication systems, and 7 systems that integrate surveillance, communication, and command and control functions (some systems have more than 1 function and are described in more than 1 section). Most reports only described IT/DSSs; however, 79 studies evaluated 58 systems for at least 1 performance metric. Some types of systems have been evaluated more than others. For example, 10 of the 18 management systems have been evaluated in at least 1 study; whereas, none of the 7 integrated surveillance, communication, and command and control systems has been evaluated. Most of the 217 included systems were not designed specifically for bioterrorism; instead, they were created for detecting and managing naturally occurring illness. The few systems that were designed for bioterrorism are principally for detection and integrated command and control purposes, and most were designed by the military and are being converted for civilian use. There is almost no publicly available evaluative data on these systems, although the military developers may have performed comprehensive evaluations.
Key Question 1: What are the information needs of clinicians and public health officials in the event of a bioterrorist attack?
Based on the conceptual model and task decomposition, the information required by clinicians and public health officials while preparing for and responding to bioterrorist events relate to the decisions they have to make and the tasks they have to perform.
Clinicians require the necessary information to make diagnostic, management, prevention, and reporting decisions. Diagnostic decisions require information to accurately estimate the pre-test probability of disease for a given patient. Clinicians' interpretations of test results require information about the sensitivity and specificity of the test. Management decisions require information about how to appropriately distinguish between those patients who need treatment and those who do not, how best to treat the acutely ill, whom to isolate and how, how to manage scarce resources, and how to maintain personal safety. Prevention decisions require information about prophylaxis and vaccination protocols. Reporting decisions rely on information about what constitutes a reportable case or cluster of cases and about the kinds of data that public health officials seek.
The information that public health officials require to prepare for and respond to a bioterrorism event can be considered in relation to the decisions they must make: the interpretation of surveillance data; the investigation of outbreaks; the institution of epidemiologic control measures; and the issuance of surveillance alerts. The decision to perform outbreak investigation requires information about the baseline characteristics of the surveillance data and threshold levels that suggest that an outbreak resulting from naturally occurring or bioterrorism-related illness may have occurred. Once a bioterrorism event has been identified, public health officials require information that will enable them to perform ongoing surveillance in the midst of the crisis to track the extent and spread of the epidemic. The decisions regarding the institution of epidemiologic control measures that prevent the spread of disease require information about the transmissibility of the suspected biothreat agent(s) and about the criteria for and effectiveness of prophylaxis and quarantine strategies. Decisions to issue a surveillance alert require information about the nature of the suspected bioterrorist attack and the characteristics and expected natural history of the suspected biothreat agent(s). Other communication decisions relate to the specific information that needs to be conveyed to other public health officials, clinicians, the media and other decision makers.
Key Question 2: Based on the information needs identified for these decision makers, what are the criteria by which IT/DSSs should be evaluated with respect to usefulness during a bioterrorism event?
The evaluation criteria vary depending on the purpose of the IT/DSS and the information needs of the users of the system as determined by task decomposition methodology.
All included systems: the purpose of the system; type of hardware required; methods for maintaining security of samples and data collected; timeliness; and measures of the accuracy of the system (e.g., sensitivity, specificity, collection efficiency, or concentration of organisms detected).
Detection systems: portability; number of samples that can be run simultaneously; number of biothreat agents that can be identified; and whether both toxins and organisms can be identified.
Diagnostic, management, and prevention DSSs: the type of information required by the DSS (e.g., a manually-entered list of signs and symptoms provided by the clinician or patient information from an electronic medical record); the type of information provided by the DSS (e.g., a list of differential diagnoses, antibiotic recommendation, or quarantine recommendation); whether the biothreat agents and their associated illnesses are included in the knowledge base; the method of reasoning used by the inference engine; and information regarding the ability to update the probability of biothreat-related illness as the epidemic progresses or from reports of a known attack.
Surveillance systems: the type of surveillance data collected; methods for determining when an outbreak has occurred; and information regarding the public health importance of the health event under surveillance, the system's usefulness, simplicity, flexibility, acceptability, representativeness, and the direct costs needed to operate the system.
Reporting and communication systems: the type of information the system is intended to communicate; the intended provider and recipient of the information; and whether the recipient has to actively seek the information from the provider (e.g., by visiting a Web site) or the information is transmitted by phone, fax, e-mail, or other means to the recipient (i.e., passive on the part of the recipient).
Key Question 3: When assessed by these criteria, in what ways could existing IT/DSSs be useful during a bioterrorism event? In what ways are they limited?
Our review identified 217 IT/DSSs, few of which were designed specifically for response to bioterrorism events. Most included systems had other intended purposes, but could potentially be useful to clinicians or public health officials in response to a bioterrorism event. The evidence by which to judge the usefulness of these systems is limited. Many of the systems were not evaluated even for their intended purpose. Of the studies that did evaluate systems for their intended purpose, few adhered to published criteria for high-quality evaluations. In addition, even if a system received a favorable evaluation for its intended purpose, it may not necessarily be feasible to evaluate its usefulness for response to bioterrorism.
We identified 55 detection systems that collect and identify potential biothreat agents within environmental and clinical samples. Many of these systems were developed for use by the military and some were adapted for civilian purposes. Few reports compare detection systems to a gold standard, and their sensitivity (i.e., the likelihood that the detection system will give a positive result when testing a sample containing a biothreat agent) and specificity (i.e., the likelihood that the detection system will give a negative result when testing a sample that does not contain a biothreat agent) remain poorly characterized in the publicly available literature. Most identification systems are limited in that each test cycle can evaluate a sample for only a single biothreat agent, often run only a limited number of samples at a time, and cannot test for many of the most worrisome agents (e.g., smallpox). We found no reports that directly compared 2 or more of the commercially available systems in any given category. The paucity of comprehensive evaluative information about these systems prevents conclusions about whether or not these systems are likely to serve the detection needs of first-responders, clinicians, and public health officials during a bioterrorist event.
We identified 23 diagnostic systems with potential utility for enhancing the likelihood that clinicians consider the possibility of bioterrorism-related illness. These systems are generally designed to assist clinicians in developing a differential diagnosis for a patient who has an unusual clinical presentation. We found 6 general diagnostic systems, 4 systems designed to improve radiologic diagnoses, 4 telemedicine systems, 4 systems for the diagnosis of infectious diseases, 1 system for the diagnosis of dermatologic lesions, 1 system for the diagnosis of community-acquired pneumonia, and 3 systems for other purposes. None of these DSSs has been evaluated formally with respect to bioterrorism response. In an evaluation of a DSS for infectious diseases that has more than 20 biothreat agents in its knowledge base, the system was able to list the actual diagnosis in an output of possible diagnoses for nearly 95 percent of 495 actual and hypothetical cases. However, this system is limited in that it is specific for infectious diseases; consequently, even those clinicians with access to this technology may not use it if the patient does not present with a fever or other signs or symptoms of infectious disease.
Three of the general diagnostic DSSs have been evaluated for their intended (non-bioterrorism related) purposes. In these evaluations, the general diagnostic DSSs typically performed better than physicians-in-training but not as well as experienced clinicians. However, the accuracy of the DSSs decreased for difficult cases. The need for clinicians to manually enter patients' signs and symptoms into diagnostic DSSs -- a laborious step that may be a barrier to the use of these systems and has been demonstrated to increase inter-user variability -- is eliminated by the few systems that automatically collect patient data from an electronic medical record. For example, there are diagnostic DSSs currently available in hospitals with electronic medical records that provide clinicians with an estimate of the likelihood of community-acquired pneumonia or active pulmonary tuberculosis based exclusively on data collected from the medical record. Many diagnostic DSSs use probabilistic information about the likelihood of disease. Because bioterrorism-related illness is relatively rare, in the event of bioterrorism these systems will have inappropriately low pretest probabilities for biothreat agents. None of the reports of diagnostic DSSs described the ability to change the probability of disease based on information about suspected bioterrorism events.
Management and prevention systems are designed to make recommendations to clinicians by abstracting clinical information from electronic medical records to make patient-specific recommendations. None of the 18 systems identified in this review has been specifically designed or evaluated for utility in providing management or prevention recommendations during a bioterrorism event; however, 10 of them have been evaluated for their intended purpose. These evaluations demonstrate that the expert systems that continuously search electronic medical records (including data from the laboratory, radiology reports, and clinician notes) for new evidence of infection and apply clinical practice guidelines to those data are able to affect clinicians' antibiotic selection decisions and increase compliance with clinical practice guidelines. We have no information as to whether the knowledge bases of these systems include comprehensive information about bioterrorism-related illnesses. The systems that are not linked to electronic medical records share many of the limitations of the general diagnostic systems -- including, that clinicians may not use the system to seek advice for patients presenting with common viral syndromes (i.e., the bioterrorism-related syndromes). Antibiotic recommendation programs are typically designed to provide recommendations for antibiotics with the narrowest possible spectra, thereby reducing the risk of developing resistant organisms. If clinicians make antibiotic selection decisions while unaware of the true bioterrorism-related diagnosis and select narrow-spectrum antibiotics, they may not be effective against biothreat agents. Therefore, whether the use of these systems would be helpful or detrimental is not known.
We identified 90 surveillance systems that collect a variety of surveillance reports: 7 for syndromal surveillance, 6 for reports from clinicians, 11 for influenza-related data, 23 for laboratory and antimicrobial resistance data, 16 for hospital-based infections data, 10 for food-borne illness data, 6 for zoonotic illness data, and 11 for other types of surveillance data. For a surveillance system to detect a covert bioterrorist event, it must collect data that are sensitive and specific for biothreat agents, analyze the data, and report results to public health decision makers in a timely manner. None of 90 included surveillance systems has been evaluated for its utility in detecting a bioterrorism event. Forty of 61 reports of evaluations or descriptions of surveillance systems described the timeliness, importance of the health event under surveillance, and usefulness of the system. However, less than one-third of the reports of evaluations of surveillance systems described the representativeness, simplicity, sensitivity, specificity, acceptability, or flexibility of the system. The quality of the evidence regarding the effectiveness of the systems reported by these articles is therefore limited. Most of the evaluations of surveillance systems demonstrated that the electronic collection and reporting of surveillance data improved detection over older, manual methods. When we consider the 90 surveillance systems described in this Report, there are relatively few systems collecting the earliest surveillance data -- such as school and work absenteeism, calls to telephone care nurses, over-the-counter pharmacy sales, or veterinary or zoonotic illness -- a potentially significant gap in available surveillance systems.
The earliest symptoms caused by most biothreat agents are flu-like illness, acute respiratory distress, gastrointestinal symptoms, febrile hemorrhagic syndromes, and febrile illnesses with either dermatologic or neurologic findings. Therefore, patients with these syndromes are the targets of bioterrorism-related syndromal surveillance programs. None of the 7 syndromal surveillance systems we identified has been clinically evaluated; however, several evaluations are ongoing. These systems are highly heterogeneous with respect to the syndromes under surveillance, the definition of the syndromes, and the type of data collected. Some systems use routinely collected diagnostic codes, others use syndromal reports collected from triage nurses for all patients presenting to an emergency department, and several use clinicians' reports of syndromal data collected on selected patients. We have no evidence to determine which of the methods of collecting syndromal data is the most sensitive, timely, acceptable, and cost-effective.
Syndromal surveillance systems have been used both for ongoing surveillance and for event-based surveillance. One syndromal surveillance tool, designed for ongoing collection of demographic and clinical data from remote regions of the developing world, downloads information daily to a national public health department. In event-based surveillance, the system is deployed for a limited period before, during, and after an event thought to be a potential target for bioterrorism, such as a major sporting or political event.
Because clinicians may be the first to recognize unusual or suspicious illnesses, reports from clinician networks are an important source of surveillance data for detection of bioterrorism-related diseases. Of the systems that have been evaluated for the collection of clinician reports, Eurosentinel provides the timeliest data (however, this is only true for influenza; data on other diseases and syndromes have a longer delay). The timeliness of the other systems varies from days to months. Systems that collect data on a weekly basis will be substantially less useful for bioterrorism surveillance than systems that can provide more rapid collection and analysis.
Although none of the 11 surveillance systems that collect influenza data has been evaluated specifically for the detection of bioterrorism-related illness, they are potentially useful for bioterrorism surveillance in 3 ways. First, sentinel clinicians who report on patients with suspected influenza are experienced at applying a case definition to a clinical population for the collection of public health data. Because many bioterrorism-related illnesses present with a flu-like illness, this network of trained sentinel clinicians could provide valuable surveillance data. (We note that the evaluation of these sentinel clinicians is derived from heterogeneous surveillance networks in North America, Europe, and Australia. It is difficult to know whether the cultures of medicine, the training that sentinel clinicians receive, and their commitment to public health surveillance efforts is sufficiently similar that we can assume that the results of an evaluation of a surveillance network in France will be generalizable to clinicians in the U.S.) Second, examples exist of effective influenza surveillance systems that integrate clinical and laboratory data for the detection of influenza outbreaks. Surveillance for bioterrorism may be aided by similar integration of multiple data sources. Finally, influenza surveillance, like bioterrorism surveillance, requires a coordinated global effort. New programs for the surveillance of bioterrorism-related illness could utilize the historical relationships that have been developed for influenza surveillance. Several of the influenza systems rely on weekly reporting by clinicians -- for bioterrorism surveillance, this time lag is likely to be problematic.
Laboratory surveillance systems are an essential component of any system for the detection of a covert bioterrorist event, both for the detection of uncommon organisms (e.g., smallpox, anthrax, and Ebola) and common organisms with unusual antimicrobial resistance patterns. Systems that facilitate the collection, analysis, and reporting of notifiable pathogens and antimicrobial resistance data could potentially facilitate the rapid detection of a biothreat agent. Our search identified 12 systems for the surveillance of laboratory data (4 of which were described in peer-reviewed evaluation reports) and 11 systems for the surveillance of antimicrobial data (1 of which was described in a peer-reviewed evaluation report). In general, the evaluative and descriptive reports of the systems collecting laboratory and antimicrobial resistance data suggest that the electronic systems improve the timeliness and sensitivity of conventional methods. Few reports specifically described how laboratory samples are handled, acceptability, or cost of implementation.
Laboratories that already report data in an electronic format to local public health officials could be incorporated into bioterrorism surveillance systems at local, state, national, and international levels -- creating a "network of networks." A principal challenge for laboratory networks is the timely communication of data from collection sites to central surveillance agencies. Efforts are ongoing to address these issues. Specifically, the Laboratory Response Network, which builds on existing laboratory capacity and is currently under active expansion, has been designed with the specific intention of being able to be integrated into surveillance networks (such as the CDC's National Electronic Disease Surveillance System) and communication networks (such as the California initiative to develop a Rapid Health Electronic Alert, Communication, and Training system (RHEACT)). These systems are under development and have not been evaluated.
The 16 hospital-based surveillance systems could play 2 roles in the early detection of a covert bioterrorist attack: the identification of a cluster of cases recently admitted suggestive of a community-based outbreak, and the identification of a cluster of cases within the hospital suggestive of in-patients with an unrecognized communicable disease. However, the reports of the surveillance systems for hospital-acquired infections suggest that, although these systems could be a valuable tool for hospital infection control officers, there is little evidence to demonstrate that they have sufficient sensitivity, specificity, or timeliness to detect a community-based bioterrorism event.
Terrorism attacks may be made against food and agriculture production facilities (domestically or abroad), transportation systems, water supplies (for either human consumption or to contaminate food production), farm workers, food handlers, and processing facilities. Similarly, concerns exist that a bioterrorist attack could involve the dissemination of a zoonotic illness among animal populations with the intention of infecting humans or livestock and causing economic and political chaos. We found 6 ITs designed to collect, process and disseminate information on zoonotic and animal diseases, none of which has been described in a peer-reviewed evaluation. Mosquito-borne viruses such as West Nile Virus, St. Louis encephalitis, and Western equine encephalomyelitis are all targets of ongoing zoonotic surveillance programs. Our search found reports of only 2 zoonotic surveillance systems -- a major gap in the literature of bioterrorism surveillance efforts. Most of the reports provided little or no information about the timeliness of these systems; those that did suggest lag times that would be too long for effective bioterrorism surveillance. None has been specifically evaluated for this purpose. In addition, the surveillance systems that collect data on food-borne illnesses and laboratory information about DNA strains of food-borne pathogens are limited in that they only collect routine surveillance data on a small number of pathogens (and do not typically include all of the most worrisome agroterrorism-related agents).
Eight of the 26 communication systems were designed for communication among public health officials at local, state and federal levels (e.g., Web-based discussion and reporting of surveillance data). In pilot evaluations directed by individual state health departments, these systems securely managed the disease reporting needs of local and state public health officials. However, these systems were limited to communication within a state. We found no single system that effectively links members of the public health community at national, state and local levels. However, there are ongoing efforts (such as the Urban Security Initiative project of Los Alamos National Laboratory, EpiX, Health Alert Network and RHEACT) designed to integrate communication of public health information vertically and horizontally within the U.S. public health system. Five systems were designed for the automated communication of information from hospital-based electronic medical records to clinicians (e.g., alerting systems to notify clinicians of abnormal laboratory tests). These systems have been subjected to the greatest evaluation of all the communication systems. Despite being limited to institutions with electronic medical records, they could potentially play an important role in improving the timely recognition of bioterrorism-related illness. Three systems facilitated communication between emergency departments and first-line emergency response personnel. ProMED© has demonstrated the capacity for rapid reporting and dissemination of information on a wide range of infectious diseases resulting from both naturally occurring and bioterrorism-related events. During a bioterrorism event, clinicians must be able to rapidly communicate with their patients. Systems exist that enable Web-based communications between these parties in a manner compliant with the Health Insurance Portability and Accountability Act of 1996 (HIPAA). Robust security measures that ensure patient confidentiality and resist cyberattack will be a necessary component of any bioterrorism-related communication system.
Key Question 4: In areas where existing IT/DSSs do not meet the information needs of clinicians or public health officials, what functional and technical considerations are important in the design of future IT/DSSs to support response to bioterrorism events?
We did not identify evaluations or studies that directly assess the functional and technical requirements that are important for future IT/DSSs. In this section, we provide our interpretation of factors that could be considered for the design of future IT/DSSs.
IT/DSSs for bioterrorism need to have documented sensitivity, specificity, and timeliness that are appropriate for their intended use. Because both false-positive and false-negative results can result in serious adverse outcomes, sensitivity and specificity should generally be high. Similarly, timeliness is of critical importance for IT/DSSs that aid with detection, diagnosis, management, communication, and surveillance. Systems should have measures to maintain security of samples and data collected.
Detection and diagnostic systems must be in use in the affected area. In the event of a covert attack collection systems will have to be in place in areas of likely attack. In the event of a known attack, these systems must be portable and sufficiently rapid that they can be used in a variety of field and clinical situations.
Clinicians would be helped by detection methods that include all of the most worrisome bio-threat agents, by systems that can test a individual sample for multiple biothreat agents simultaneously, and by systems that can run multiple samples simultaneously.
Because the individuals collecting and analyzing the environmental and clinical samples are often at considerable distance from public health decision makers, detection systems could benefit from the capacity for secure transmission of data to these decision makers.
Efforts to link diagnostic and management or prevention DSSs to other hospital information systems would reduce the data entry burden substantially.
The knowledge bases of diagnostic and management systems need to include current information and clinical practice guidelines about bioterrorism-related illness. The systems need to be able to appropriately adjust the probability of disease caused by biothreat agents if a known bioterrorism event has occurred.
Efforts to integrate surveillance data may benefit from definitions of the syndromes under surveillance; comprehensive analysis of the sensitivity, specificity, and timeliness of each source of surveillance data; improved spatial and temporal analysis methods; and systems that collect sources of data reflecting disease earlier in the course of illness (e.g., school and work absenteeism and over-the-counter pharmacy sales).
Communication systems that protect patient confidentiality and have adequate security measures would be useful for the rapid dissemination of outbreak-related information among all relevant decision makers, including public health officials, clinicians, and the public.
IT/DSSs have the potential to help clinicians and public health officials make better decisions when responding to a bioterrorism event. We identified IT/DSSs that could potentially aid with detection, diagnosis, management, prevention, surveillance, and communication. However, most of these systems were not designed specifically for bioterrorism. Many of these systems have not been described in peer-reviewed literature, and fewer still have been evaluated rigorously. The existing evaluations primarily assess the usefulness of systems for their intended purpose, and often do not provide direct evidence about the usefulness of the IT/DSSs for bioterrorism.
The lack of evaluative studies creates difficulties in assessing the usefulness of IT/DSSs. For detection systems, almost no information is available on sensitivity and specificity. Without this information, interpretation of test results is highly problematic. Diagnostic DSSs have not been used widely, and several of the available systems require time-consuming manual input of patient data, which is impractical in many clinical settings. Whether management DSSs could be useful for bioterrorism-related disease remains unanswered. Surveillance systems hold promise, and although many are undergoing evaluation, the systems designed for bioterrorism response have been fielded only recently. Web-based communication systems are increasingly available to link public health officials with clinicians and the public; however, their efficacy in crisis situations is untested.
Our review suggests important gaps in the available literature. We note, however, that lack of evidence about effectiveness is not evidence for lack of effectiveness. Many of the systems we reviewed may indeed be useful for response to bioterrorism and are reasonable candidates for further evaluation. Such evaluations would clarify their value both for response to bioterrorism and for the other purposes for which they were designed.
In addition to the development of systems described in the answer to Key Question 4, the following future research could provide additional insights into the information needs of clinicians and public health officials and the types of IT/DSSs that may best serve those needs:
Further research is needed for the development and evaluation of systems as outlined in the answer to Key Question 4.
Further research is needed that investigates the decisions and tasks of specific types of clinicians (e.g., primary care providers, Emergency Medicine specialists, and Infectious Disease specialists), different types of public health officials (e.g., those working in county public health departments, at the CDC, and in the Department of Health and Human Services), and other groups of relevant decision makers (e.g., laboratory personnel, paramedics, veterinarians, and hospital administrators).
Evaluations of current systems and the interaction of these systems during simulated bioterrorism events are currently under-reported, not available yet, or potentially classified. Detailed evaluations of IT/DSSs and situations where their use might enhance decision making would guide further system development and evaluation research.
Methodologies other than systematic review would provide additional valuable insight into the answers of the Key Questions addressed in this Report. For example, surveys of clinicians and public health officials could be used to better describe the information needs of these groups in preparing for and responding to bioterrorism events, the IT/DSSs currently in use, and the performance of these systems in routine use and times of crisis.
Further research is needed on how to provide effective training in the use of IT/DSSs and how to provide effective continual medical education to enhance the diagnostic capabilities of clinicians for bioterrorism-related illness through DSSs or other approaches.
Further research is needed on how to maintain the security and availability of systems in times of crisis.
Nonetheless, he knew that the tale he had to tell could not be one of final victory. It could be only the record of what had had to be done, and what assuredly would have to be done again in the never ending fight against terror and its relentless onslaughts, despite their personal afflictions, by all who, while unable to be saints but refusing to bow down to pestilences, strive their utmost to be healers.
-- A. Camus, 1948 1
The nation's capacity to respond to biothreat agents depends on the ability of first responders, clinicians, and public health officials to detect, manage, and communicate during a bioterrorist event.2-6 These first responders (e.g., emergency medical technicians, firemen, policemen, and hazard materials professionals), clinicians (e.g., physicians, nurses, nurse practitioners and respiratory therapists), and public health officials (local, state, national and international) will require substantial resources as well as appropriate information technologies and decision support systems (IT/DSSs) to perform their jobs effectively in response to a bioterrorist attack.2-6
The morbidity and mortality that may result from a poorly prepared health care system responding to a bioterrorist attack have been well described.7, 8 In 2 recent simulations,9 and perhaps most dramatically in the descriptions of 2 actual vulnerability tests performed during the U.S. offensive bioweapons program,10 the need for ITs for detection, diagnosis, management, prevention, surveillance, reporting, and communication during a bioterrorism event was deemed paramount.
An exercise called TOPOFF, designed to test the readiness of top officials of the government to respond to terrorist attacks, was conducted by the U.S. Department of Justice in response to a request by the U.S. Congress to "assess the Nation's crisis and consequence management capacity under extraordinarily stressful conditions."9 The exercise, which took place in May 2000 at a cost of $3 million, simulated a chemical weapons event in Portsmouth, New Hampshire, a radiological event in the Washington, D.C. area, and a release of an aerosol of Yersinia pestis in Denver, Colorado.9 The officials participating in this exercise included: county, state, and federal public health officials; emergency physicians; emergency management professionals; and infection control professionals.9 Significant difficulties were experienced during the exercise including clear identification of the crisis management leaders and decision makers, distribution of resources such as antibiotics from the National Pharmaceutical Stockpile, determination of best methods for the prevention of the spread of disease, and inadequate management of crisis situations that resulted from hospitals running out of beds, supplies, and personnel.9 The primary means of communication during the exercise was via conference calls.9 At times nearly 100 people, many of whom had no prior working relationships, were on the conference calls trying to participate in the decision-making process.9 ITs would have facilitated many of the decision-making processes during this exercise. In particular, IT/DSSs could have been used for the rapid diagnosis of people seeking medical attention; for the management of cases including isolation, treatment, and maintenance of personal safety among clinicians; and for communication among all participating organizations.
In the Dark Winter exercise, held at Andrews Air Force Base on June 22-23, 2001, former senior government officials played the roles of National Security Council members and representatives from the media portrayed journalists during a response to an evolving smallpox epidemic.11 The scenario presented in this exercise culminated in the infection of thousands of patients, death of hundreds, and generalized civil disorder.11 The Dark Winter exercise emphasized the lack of surge capability in U.S. hospitals, public health systems, and vaccine and pharmaceutical industries.11 Additionally, Dark Winter highlighted the lack of adequate communication systems among clinicians, public health organizations, and the media. The resulting poor communication contributed to the chaos and perception that public health officials had lost control of the situation.11 Ideally, public health officials would have predicted the information needs of clinicians, first responders, and the public, and utilized existing relationships with the media to communicate critical information in a timely manner. Instead, the media, with a 24-hour news cycle, provided ongoing coverage of the outbreak in a manner fairly disconnected from public health officials.11
In 1950, several dissemination experiments were carried out in the San Francisco Bay area.12 In one experiment, a naval vessel sprayed Bacillus globigii (a harmless simulant commonly used in bioterrorism experiments because of its morphologic similarity to Bacillus anthracis) in a 2-mile long line, approximately 2 miles offshore.12 Collection devices in downtown San Francisco demonstrated concentrations of more than 10,000 spores per liter, sufficient to have caused infections among more than 60 percent of the population.12 The Berkeley area was also contaminated, but at a much lower level.12
In another dissemination experiment in1965, light bulbs filled with B. globigii were dropped from the back of New York City subway trains onto the tracks.12 The trains ran over the organisms, creating aerosols that were carried throughout the subway system.12 Collection devices demonstrated B. globigii in "high concentration for 60 to 90 minutes in all trains tested."12 Given that the average subway user in 1965 spent 8 minutes on the trains during rush hour, it is estimated that 80 to 90 percent of the passengers would have become infected.12
Unlike the TOPOFF and Dark Winter exercises, in which the release of a biothreat agent was known, these latter 2 dissemination experiments simulated a covert bioterrorist attack. They emphasized the need for environmental detection systems in locations thought to be possible targets (e.g., subways, airports, government buildings, and large entertainment venues). They also highlighted the need for robust surveillance systems capable of timely detection of a bioterrorist attack. The bioterrorism exercises and the dissemination experiments demonstrated the potential vulnerabilities of the civilian population to a bioterrorist attack and emphasized the necessity for thoughtful preparedness and response planning for both covert and announced release of biothreat agents.
This Evidence Report details the methodology, results, and conclusions of a literature search on IT/DSSs that could serve the information needs of clinicians and public health officials in the event of a bioterrorism attack. We evaluated IT/DSSs that serve 1 or more of 4 main categories of information needs of clinicians and public health officers: detection and diagnosis, management and prevention, surveillance, and reporting and communication. The information presented is intended to assist clinicians and public health officials improve bioterrorism preparedness and response capabilities. We anticipate the report will be valuable to policymakers requiring evidence for informed decision making regarding the implementation of IT/DSSs for bioterrorism preparedness and response planning.
The focus of our analysis was on IT/DSSs required by clinicians and public health officials. Therefore, systems designed for other decision makers (e.g., hazardous materials personnel or incident commanders) that could not also be used by clinicians or public health officials were omitted. Additionally, our focus was preparation for and response to bioterrorism events. We included those IT/DSSs designed for other purposes (such as the management of naturally occurring outbreaks) if such systems are potentially useful for a bioterrorist response (i.e., "dual use" systems), or related public health functions (e.g., food safety and animal health). Similarly, we did not include those IT/DSSs for response to chemical or nuclear weapons unless they could also be of use against biothreat agents.
We considered first responders to be all personnel responsible for the direct management of a bioterrorism event in the field. These include, but are not limited to, emergency medical technicians, firemen, policemen, and hazard materials professionals. We considered clinicians to be all personnel who would be directly involved in the care of patients with bioterrorism-related illness in a clinic or hospital. These include physicians, nurses, nurse practitioners and respiratory therapists. We use the term public health official (unless otherwise specified) to refer to all professionals at the local, state, national and international levels responsible for preparing for and responding to acts of bioterrorism to ensure the public health.
We evaluated IT/DSSs that affect 1 or more of 4 main categories of information needs of clinicians and public health officers: detection and diagnosis, management and prevention, surveillance, and reporting and communication. We defined ITs and DSSs according to the definitions provided by Friedman and Wyatt: A DSS (also called decision-aid, decision-assistance system, decision-making system) is a "system that compares patient characteristics with a knowledge base and then guides a health provider by offering patient-specific and situation-specific advice. Such systems, by definition, offer more than a summary of the patient data."13 An IT (also called information resource) is a system "typically consisting of computer hardware and/or software that facilitate the collection, processing, and dissemination of information."13
IT/DSSs described by articles included in the Evidence Report include the use of 1 or more computers for the purpose of collecting, managing, analyzing, or communicating medical information. Because our search strategies were designed to capture all systems for a given purpose (such as detection), this Report does include some relevant technologies that are neither ITs nor DSSs. For example, we describe assays used in the field by first responders to make the rapid detection of B. anthracis spores. Because the information from these types of assays can be transferred via wireless or other connections to a data analysis system or to decision makers, we have included them.
We began the project by identifying advisors with wide-ranging technical expertise in bioterrorism preparedness and response, public health, and IT/DSSs. With the advice of these experts and input from the Agency for Healthcare Research and Quality (AHRQ), we refined the research questions and created a conceptual framework of the decisions likely to be faced by clinicians and public health officials during a bioterrorist attack. We used this framework to develop a list of tasks that IT/DSSs would have to perform to assist the decision-making process of clinicians and public health officials. We used this list and previously published reports to develop a framework for evaluating IT/DSSs. We conducted searches of peer-reviewed literature, government documents, and the Internet for reports of potentially relevant IT/DSSs, which we evaluated for their possible utility. Appendix A provides a complete listing of the acronyms and abbreviations used in this Report.
For advice on the scope of the project, we consulted technical experts (Appendix B) in the following fields: bioterrorism/biodefense, emerging infectious diseases, IT/medical informatics, public health, evidence synthesis/meta-analysis, and clinical medicine. These experts assisted us in refining the research questions, developing the conceptual model, and preparing the Report.
The targeted decision makers addressed in this Report are clinicians and public health officials. For the purpose of this Report, clinicians include all clinical health providers, such as physicians, nurses, and community health workers. Public health officials include those at the local, state, federal, and international levels.
The Key Questions that we developed in collaboration with AHRQ and the expert advisory panel were based on the premise that during a bioterrorism event, clinicians would have 4 major tasks and public health officials would have 3 major tasks. Clinicians would have to: (1) correctly diagnose the clinical manifestations of biothreat agents; (2) rapidly manage the care of potentially exposed patients; (3) take effective action to prevent the further spread of disease; and (4) report suspicious or confirmed cases to local, regional, and national public health officials. Public health officials would have to: (1) communicate with first responders (i.e., fire, police, and hazardous materials personnel), clinicians, and the public; (2) manage and interpret surveillance data to determine when to perform outbreak investigation; and (3) determine when to take epidemiologic control measures, such as quarantine, to prevent the spread of disease.
Therefore, the Key Questions to be addressed in this Report are:
Key Question 1: What are the information needs of clinicians and public health officials in the event of a bioterrorist attack?
Key Question 2: Based on the information needs identified for these decision makers, what are the criteria by which IT/DSSs should be evaluated with respect to usefulness during a bioterrorism event?
Key Question 3: When assessed by these criteria, in what ways could existing IT/DSSs be useful during a bioterrorism event? In what ways are they limited?
Key Question 4: In areas where existing IT/DSSs do not meet the information needs of clinicians or public health officials, what functional and technical considerations are important in the design of future IT/DSSs to support response to bioterrorism events?
We present detailed information about the Key Questions throughout the Results section and summarize our answers to these Key Questions at the end of Chapter 3.
To describe the information needs of clinicians and public health officials (Key Question 1), we evaluated the decisions they would have to make while preparing for and responding to a bioterrorism event. To determine these key decisions of clinicians and public health officials, we reviewed reports of their participation in naturally occurring outbreaks of infectious diseases (including infections due to Cryptosporidium parvum contamination of the water supply in Milwaukee during March and April 199314 and West Nile Virus in New York City in late August 199915), TOPOFF and Dark Winter bioterrorism preparedness exercises,9, 11, 16 emergency preparedness plans,17-20 standards for reporting public health surveillance data,21-23 and standards for maintaining the security of electronic data.24 We obtained additional information about the information needs of public health officials and the types of information systems that have been helpful to them in managing infectious disease outbreaks by soliciting input from public health officials working in the health departments of the 5 most populous states and 4 most populous counties in the U.S. We asked them to describe the surveillance and communication systems currently in use in their health departments and any new systems or initiatives instituted since the events of September 11, 2001. They reported that limitations in personnel and other resources prevented significant increases in preparation and response efforts; surveillance systems are largely dependent on voluntary clinician and laboratory reports; communication with clinicians and other members of the public is primarily by fax; syndromal surveillance systems are perceived to be costly and the data difficult to interpret; and few had instituted specific changes after September 11th but many had pilot projects ongoing.
We developed a conceptual model to specify the decisions made by clinicians and public health officials in the event of a bioterrorist attack. We then created an influence diagram to represent our conceptual model of the key decisions made by clinicians and public health officials during a bioterrorism event. Influence diagrams are graphical representations of formal mathematical models that facilitate the compact representation of the probabilistic structures of complex problems.25-27 A detailed description of the representation and analysis of medical decisions with influence diagrams is beyond the scope of this project. For a complete discussion of this topic, we direct interested readers to Owens et al.25 The influence diagram allowed us to assess the relationships between the decisions made by the 2 types of decision makers, to identify the uncertain events that affect these decisions, and to evaluate the information that is observable by the decision makers at the time they make their decisions.
We adopted standard influence diagram notation such that decisions are represented by rectangular nodes.25-27 Arrows between decision nodes indicate that at the time of the second decision, the decision maker has knowledge of the previous decision. Probabilistic events are represented by elliptical (chance) nodes. Arrows between chance nodes indicate that a probabilistic relationship may exist. That is, the outcome of the first chance event may change the probability of the outcome of the second. Arrows from a chance node to a decision node indicate that the outcome of the uncertain event is known at the time the decision is made.
The complexities of the processes involved in clinicians and public health officials responding to a bioterrorist attack created unique challenges to the standard method of using an influence diagram to represent the relevant decisions. Specifically, our conceptual model required the incorporation of different decision makers, different time horizons, and different value functions.
We used a process called task decomposition to describe the characteristics of IT/DSSs that would be required for these systems to assist clinicians and public officials as they make the decisions described in our conceptual model. Task decomposition provides a framework for specifying, documenting, and evaluating what types of data an IT/DSS should contain in order to serve its purpose.28-30 Task decomposition starts with the identification of the database's main purpose (or target task).28-30 This target task is then hierarchically decomposed into 3 components: (1) subtasks; (2) the methods to be used for accomplishing those subtasks; and (3) the necessary and sufficient information to complete those subtasks according to the specified methods.28-30 A database that models all the data required for completing a task is deemed competent for that task.28-30 For example, performing syndromal surveillance may be a target task of public health officials. Monitoring hospital-discharge diagnoses for the ICD9 codes associated with fever and rash (i.e., subtask) is one way to implement a syndromal surveillance system. For such a syndromal surveillance system to work, it is necessary to have a method for collecting ICD9 codes (i.e., method and data necessary to accomplish this subtask). Using this framework, the data requirements for other information needs can be similarly specified.
We decomposed the information needs of clinicians and public health officials into top-level tasks and subtasks. We then considered the key concepts driving each task and the data requirements for an IT/DSS to assist in that task. We used this task decomposition as well as our Key Questions to create a data abstraction form (Appendix C).
Based on input from our expert advisory panel, the conceptual model, task decomposition, and practical considerations, we developed an inclusion/exclusion strategy to identify articles that described or evaluated IT/DSSs that could be useful currently, or with potential adaptation, to clinicians and public health officials in the event of a bioterrorism attack. We included IT/DSSs designed for responding to naturally occurring outbreaks if such systems are potentially useful for a bioterrorist response. Similarly, potentially dual use diagnostic and management IT/DSSs are included. We also considered other technologies, such as detection devices, that are not IT/DSSs per se, but collect data for surveillance systems or otherwise enable the IT/DSSs to perform diagnosis, management, prevention, surveillance, reporting, and communication functions. Because the focus of our analysis was on systems for use by clinicians and public health officials, we did not include IT/DSSs designed for other decision makers unless clinicians or public health officials could also use them. Similarly, we did not include those IT/DSSs for response to chemical or nuclear weapons unless they could also be of use against biothreat agents.
Before developing our inclusion and exclusion criteria for information describing IT/DSSs for bioterrorism, we established the following guidelines:
Articles reporting descriptions of systems must at a minimum include a statement of the purpose of the system;
Articles reporting clinical evaluations of systems must at a minimum include a report of the results of an evaluation using either actual or simulated patient data. Outcomes of these evaluations may include, but are not limited to, those that relate to the system's effectiveness; sensitivity and specificity; implementation, usability and acceptability; cost; or timeliness. By sensitivity (also called true positive rate), we mean the likelihood that a system provides a positive response when a bioterrorism event has occurred. By specificity (also called true negative rate), we mean the likelihood that a system provides a negative response in the absence of a bioterrorism event. The false positive rate is equal to 1 minus the specificity and the false negative rate is equal to 1 minus the sensitivity.
Bioterrorism-relevant diseases are all conditions resulting from the biothreat agents as defined by the Department of Health and Human Services (HHS).31 Specifically, HHS has identified the following organisms as having the highest bioterrorist threat potential: Variola major (smallpox), B. anthracis (anthrax), Y. pestis(plague), Clostridium botulinum toxin (botulism), Francisella tularensis (tularemia), Filoviruses (Ebola hemorrhagic fever and Marburghemorrhagic fever), Arenaviruses (Lassafever, Junin/Argentine hemorrhagic fever) and related viruses.31 The next highest priority agents include: Coxiella burnetti (Q fever), Brucella species (brucellosis), Burkholderia mallei (glanders), Alphaviruses (Venezuelan encephalomyelitis, eastern and western equine encephalomyelitis), Ricin toxin from Ricinus communis (castor beans), Epsilon toxin of Clostridium perfringens, Staphylococcus enterotoxin B, and food or waterborne pathogens such as Salmonella species, Shigella dysenteriae, Escherichia coli O157:H7, Vibrio cholerae, and C. parvum. 31 The third highest priority agents are Nipah virus, Hanta virus, tickborne hemorrhagic feverviruses, tickborne encephalitis viruses, yellow fever, and multidrug-resistant tuberculosis.31 Because these agents primarily cause flu-like illness, acute respiratory distress, gastrointestinal symptoms, febrile hemorrhagic syndromes, and febrile illnesses with either dermatologic or neurologic findings, these syndromes will be considered bioterrorism-relevant syndromes.
We included systems specifically designed to support the diagnosis of bioterrorism-relevant diseases or syndromes (e.g., systems that make recommendations about ordering diagnostic tests for a bioterrorism-relevant organism). We also included general diagnostic systems (e.g., systems that provide differential diagnoses based on a patient's signs or symptoms), automated diagnostic test analysis systems, microbiologic test analysis systems for bioterrorism-specific agents, rapid detection technologies for use in the field or at the bedside, and automated radiologic diagnostic systems for the evaluation of bioterrorism-related illnesses or syndromes. We included radiologic diagnostic systems that automatically make the diagnosis of, or evaluate radiology reports for, the diagnosis of pulmonary infiltrate or widened mediastinum but excluded those systems that automatically read mammograms or pulmonary nodules.
We excluded systems designed for the diagnosis of non-bioterrorism-related diseases that could not be readily modified for diagnosis of bioterrorism-related diseases (e.g., DSSs for hypertension or automated laboratory systems for interpreting Pap smears). We also excluded those systems designed to improve general laboratory functions (e.g., BACTEC™ systems for positive blood culture detection).
We included systems designed for the management, treatment, prevention or guideline implementation of bioterrorism-related diseases or syndromes, as well as systems that facilitate outbreak management for bioterrorism-related diseases or syndromes.
We excluded systems designed for the management of non-bioterrorism-related diseases that could not be readily modified for applicability to a bioterrorism-related disease. Whereas we included systems designed to provide management recommendations about initiating or changing antibiotic therapy, we excluded general drug dosing or monitoring systems designed to reduce costs or side effects. We also excluded general reminder systems for routine medical care (e.g., systems that provide vaccination recommendations).
We included reports of systems designed for surveillance and reporting of bioterrorism-related diseases or syndromes. We also included systems for surveillance of the clinical epidemiology and antimicrobial resistance patterns of specific bioterrorism-related diseases or syndromes (e.g., systems that track Ebolacases or the resistance patterns of Salmonella, but not those that monitor measles or hospital-acquired intravenous catheter infections).
If an article described a surveillance system and presented only disease incidence or prevalence data, but did not evaluate the system's timeliness, sensitivity, specificity or any of the other outcomes of interest as described in the guidelines for articles reporting clinical evaluations, we abstracted only the information about the description of the surveillance system.
We included reporting and communication technologies that allow decision makers (e.g., clinicians and public health officials) to give or receive patient-specific information (e.g., systems that facilitate physician reports of communicable diseases, enable communication between ambulance personnel and emergency room staff, send hospital capacity information to command and control centers, and automatically send public health information from national authorities to local public health officials, clinicians, and the public). Additionally, we included IT/DSSs that facilitate communication between clinicians and their patients, public health officials and the public, laboratories and clinicians (e.g., systems that alert clinicians about abnormal laboratory tests), and laboratories and public health officials (e.g., systems that automatically send laboratory surveillance data to public health officials), and systems that disseminate verified public health information among interested parties.
We excluded general listserves, e-mail distribution lists, chat rooms, electronic versions of textbooks or Web sites that provide information on infectious diseases or bioterrorism-related diseases without a moderator or peer-review process.
Given the multidisciplinary nature of preparedness for and response to acts of bioterrorism, we believed that much of the information required to answer our Key Questions would fall outside the peer-reviewed medical literature. We therefore used literature sources that included medical, governmental, political science, informatics, and engineering references that were both peer-reviewed and non-peer-reviewed. We designed a search strategy for 3 sources of reports of IT/DSSs: (1) peer-reviewed articles and government documents from databases such as MEDLINE®; (2) government documents obtained from the Web sites of national and international agencies; and (3) descriptions of IT/DSSs available from other Internet sources.
In consultation with professional research librarians, we identified 5 databases of medical, scientific, and governmental references likely to contain reports of relevant IT/DSSs: MEDLINE®, The Catalog of U.S. Government Publications, GrayLIT, Library of Congress, and National Technical Information Service (NTIS). We considered articles peer-reviewed if they were published in a peer-reviewed journal.
| Agency | Organization within agency |
|---|---|
| Centers for Disease Control and Prevention (CDC) | |
| Department of Defense (DOD) | Defense Advanced Research Projects Agency (DARPA); Global Emerging Infections System (GEIS) a |
| Department of Energy (DOE) | Sandia National Laboratory; Lawrence Livermore National Laboratory; Argonne National Laboratory; Los Alamos National Laboratory a |
| Department of Veterans Affairs (VA) Environmental Protection Agency (EPA) Federal Bureau of Investigation (FBI) Federal Emergency Management Agency (FEMA) Public Health Service (PHS) National Technical Information Service (NTIS) World Health Organization (WHO) |
The Web sites of the Los Alamos National Laboratory and the DOD Global Emerging Infections System (GEIS) did not permit systematic searches using the same terms that we applied to the other government agencies. Therefore, we manually searched these sites for relevant information.
We conducted Internet searches to retrieve reports of potentially relevant IT/DSSs from sites other than those operated by government agencies (e.g., academic institutions and commercial enterprises).
We performed a series of initial literature searches and asked our expert advisory panel to provide us with relevant articles to develop a preliminary library of potentially relevant articles available in each of the 4 primary IT/DSS categories (detection and diagnosis, management and prevention, surveillance, and reporting and communication). In consultation with professional research librarians, we then developed 3 separate search strategies: one for MEDLINE®, one for all other databases of peer-reviewed articles and the government Web sites, and one for all other Web-based information.
We developed 4 subsearches that we combined into the final search strategy for articles referenced in MEDLINE® for the period January 1985-April 2001. An asterisk instructs the search engine to find all words that begin with a given text string. For example, bacter* will find all terms that begin with the root bacter such as bacteria, bacterium, and bacteriophage. The MeSH, or Medical Subject Heading, maps the search word to related concepts that MEDLINE® indexers use for categorizing articles.
Subsearch 1:
"artificial intelligence"[MESH] NOT "robotics"[MESH] OR
"decision making, computer-assisted"[MESH] OR "decision techniques"[MESH] OR
"clinical laboratory informatics systems"[MESH] OR "decision support
systems, clinical"[MESH] OR "hospital information systems"[MESH] OR
"integrated advanced information management systems"[MESH] OR "ambulatory
care information systems"[MESH] OR "clinical laboratory information
systems"[MESH] OR "clinical pharmacy information systems"[MESH] OR "decision
support systems, management"[MESH] OR "radiology information systems"[MESH]
OR "medical records systems, computerized"[MESH] OR "reminder systems"[MESH]
OR decision*[All Fields] OR expert*[All Fields] OR computer*[All Fields] OR
informatic*[All Fields] OR information system*[All Fields] OR reminder*[All
Fields] OR artificial intelligence*[All Fields]
Subsearch 2:
"diagnosis, computer-assisted"[MESH] OR "therapy,
computer-assisted"[MESH] OR "drug therapy, computer-assisted"[MESH] OR
"epidemiologic methods"[MESH] OR "disease outbreaks"[MESH] OR "disease
reservoirs"[MESH] OR "disease transmission"[MESH] OR "drug
contamination"[MESH] OR "endemic diseases"[MESH] OR "environmental
medicine"[MESH] OR "environmental microbiology"[MESH] OR "environmental
monitoring"[MESH] OR "inhalation exposure"[MESH] OR "food
contamination"[MESH] OR "communicable disease control"[MESH] OR "mandatory
reporting"[MESH] OR "mortuary practice"[MESH] OR "disease management"[MESH]
OR diagnos*[All Fields] OR therap*[All Fields] OR drug*[All Fields] OR
surveillance*[All Fields] OR manag*[All Fields] OR detect*[All Fields] OR
outbreak*[All Fields] OR transmission*[All Fields] OR monitor*[All Fields]
OR quarantine*[All Fields] OR isolat*[All Fields] OR exposure*[All Fields]
OR contaminat*[All Fields] OR alert*[All Fields] OR emergency*[All Fields]
OR report*[All Fields] OR decision making*[All Fields]
Subsearch 3:
"bioterrorism"[MESH] OR "biological warfare"[MESH] OR
"chemical warfare"[MESH] OR "bacterial infections and mycoses"[MESH] OR
"virus diseases"[MESH] OR "parasitic diseases"[MESH] OR veterinar*[All
Fields] OR coroner*[All Fields] OR military*[All Fields] OR bioterror*[All
Fields] OR biowar*[All Fields] OR bacter*[All Fields] OR viral*[All Fields]
OR virus*[All Fields] OR parasit*[All Fields] OR infectio*[All Fields] OR
communicable*[All Fields]
Subsearch 4:
"randomized controlled trial"[pt] OR "randomized
controlled trials"[MESH] OR "Random allocation"[MESH] OR "double blind
method"[MESH] OR "single blind method"[MESH] OR "comparative study"[MESH] OR
"evaluation studies"[MESH] OR "follow up studies"[MESH] OR "prospective
studies"[MESH] OR "longitudinal studies"[MESH] OR control*[Title/Abstract
Word] OR prospectiv*[Title/Abstract Word] OR volunteer*[Title/Abstract Word]
OR evaluation* [All Fields] OR stud* [All Fields] OR trial* [All Fields]
Final MEDLINE® Search Strategy: Subsearch 1 AND Subsearch 2 AND Subsearch 3 AND Subsearch 4.
bioterror* OR biological terror* OR biological warfare OR biodefense
database AND (1)
information technology AND (1)
decision support system
diagnosis AND (1)
public health AND (1)
surveillance AND (syndrome OR infectious disease)
communication AND (1)
If a search term returned more than 600 "hits," these Web sites were not screened because, on preliminary review, few "hits" were relevant in these cases.
In our preliminary Internet searches for potentially relevant IT/DSSs using individual general search engines such as Google and Alta Vista, we found a great deal of redundancy in the retrieved sites. Therefore, our final Web-based search strategy used a metasearch engine that simultaneously searched multiple search engines with the purpose of ensuring a comprehensive search while minimizing overlap in retrieved Web sites. We tested a variety of potential metasearch engines including MetaCrawler®, Dogpile®, C4 TotalSearchSM and Copernic® 2001. We found that Copernic® 2001 returned the greatest number of relevant Web sites for each search term. Copernic® 2001 is a publicly available, Web-based metasearch engine (Copernic Technologies Inc., Québec, Canada) that is easily downloaded onto the user's computer. Its database feature facilitates saving multiple searches retrieved from over 80 general search engines. We used the same terms in our Copernic® 2001 search as we did in our directed search of government agencies (see previous section).
One investigator reviewed each title to identify potentially relevant articles. If an article could not be excluded on the basis of its title, we then reviewed its abstract. We checked for agreement between reviewers after abstracting an initial set of titles and abstracts. We discussed all apparent systematic differences between the reviewers, and reached consensus on how to proceed with the kinds of articles that had caused the discrepancy. The reviewers then completed their independent reviews, meeting periodically to discuss articles about which they were uncertain.
All retrieved articles were reviewed by one or more investigators to determine if they met inclusion criteria. All peer-reviewed articles meeting inclusion criteria were independently abstracted by 2 investigators blinded to study authors. Data abstracted from each article included whether the article presented an evaluation of the IT/DSS or just a description, information about the study design, information about the purpose of the IT/DSS, whether it was in clinical use at the time of the publication of the article, what modifications might enhance the IT/DSS's usefulness for bioterrorism response, and technical aspects of the IT/DSS's design including what kind of hardware platform it uses, whether it uses standard vocabularies, what kind of reasoning it uses, what security measures it has, and whether access to the system is restricted by user type. Appendix C contains the complete abstraction form.
For each article, one abstracter entered data onto an abstraction form created in Microsoft® Access 2000 (form available upon request from authors) and the other abstracter entered the data onto a paper version of the abstraction form (Appendix C). The 2 abstracters then reviewed each datum entered and made corrections in the Microsoft® Access database. Bibliographies of obtained articles were screened for additional, potentially relevant references. Given the large number of Web sites screened, information abstracted from materials obtained via Web searches was abstracted by only a single investigator directly onto the Access abstraction form. We frequently conferred about the information abstracted from the Web sites to ensure consistency of data abstraction from the Web-based sources.
We sought published criteria by which to evaluate the evidence collected from our searches. We found 2 systems: one set of criteria for evaluations of IT/DSSs used in clinical practice, and one set of criteria specifically for surveillance systems.
The systematic reviews of the effects of IT/DSSs in clinical practice written by researchers at McMaster University used a scale that assessed 5 potential sources of bias in each of their included articles:33, 34
The method of allocation to study groups
Random
Quasi-random
Selected concurrent controls
The unit of allocation
Clinic
Physician
Patient
The presence of baseline differences between groups that were potentially linked to the study outcome
No baseline differences or appropriate statistical adjustments made for differences
Baseline difference present and no statistical adjustments made versus unable to assess
The type of outcome measure
Objective outcome or subjective outcome with blinded assessment
Subjective outcome with no blinding but clearly defined and explicit criteria for each outcome
Subjective outcome with no blinding of assessors and no explicit criteria for each outcome
Completeness of followup
Greater than 90 percent
80 to 90 percent
Less than 80 percent
This scale includes all the relevant measures for evaluating the use of IT/DSSs in clinical practice. We included all of the McMaster criteria in our abstraction form (Appendix C).
The Centers for Disease Control and Prevention (CDC) has published a guideline for evaluation of surveillance systems.35 It recommends that an evaluation of surveillance systems include the following:
A description of the public health importance of the health event
A description of the system under evaluation
An indication of the level of usefulness of the system
An evaluation of the system's simplicity
An evaluation of the system's flexibility
An evaluation of the system's acceptability
An evaluation of the system's sensitivity
An evaluation of the system's positive predictive value
An evaluation of the system's representativeness
An evaluation of the system's timeliness
A description of the direct costs needed to operate the system
This guideline is widely regarded as the standard by which surveillance systems should be evaluated. We included all of the CDC criteria in our abstraction form with the exception of cost because our preliminary reading of the literature suggested that almost none of the articles included in our systematic review included cost data (Appendix C).
In December 2001, we sent a draft of the Report to our expert advisory panel (Appendix B) and to 17 additional reviewers (Appendix D) with expertise in nursing, clinical medicine, public health, hospital management, informatics, epidemiology, national security, toxicology, and food safety. We requested comments on all aspects of the Report, using a structured format developed by the University of California at San Francisco (UCSF) Cochrane Center. The submitted Report reflects revisions made based on this input.
Decisions marked with a double line indicate those that can be affected by DSSs and the broken lines mark processes that can be affected by ITs
). The
structure of the diagram depicts 3 critical time periods. Time period 1 refers
to the interval in which decisions are made by clinicians regarding the events
associated with the initial cases, time period 2 refers to the interval in which
decisions are made by public health officials regarding the events associated
with the initial cases, and time period 3 refers to the interval in which
decisions are made by clinicians regarding the events associated with subsequent
cases. We recognize that time periods 1 and 2 are likely to occur concurrently
but have chosen to represent them as separate events in order to more clearly
delineate the decisions made by clinicians and public health officials.
) could be lives saved or morbidity prevented; it is affected primarily
by the patient's true infection status (which is unknown) and by management and
prevention decisions.
In time period 2, surveillance reports may suggest to public health officials that a potential bioterrorism event has occurred. In Figure 1
, we have only shown the surveillance reports
submitted by clinicians. However, public health officials could receive
surveillance data from a variety of sources, including environmental detection
systems, pharmacy sales, veterinarians, zoos, laboratories, first responders
(such as ambulance/911 calls), hospital discharges, school/work absenteeism and
coroners' reports (Figure 2
).
Additionally, they could receive alerts of suspected bioterrorist activity from
local law enforcement or the intelligence community. If these surveillance
reports suggest evidence of a possible bioterrorism event, public health
officials are faced with 3 primary decisions: (1) whether to perform outbreak
investigation; (2) whether to institute epidemiologic control measures; and (3)
whether to issue a surveillance alert to clinicians and first responders.
Decisions about performing outbreak investigation will be affected by the
methods used to calculate the expected values for each source of surveillance
data and to set the thresholds in the surveillance data analyses above which
outbreak investigation will be performed. Epidemiologic control measures include
actions intended to prevent the spread of disease, such as quarantine, mass
vaccination and/or antibiotic distribution, and requesting release of the
National Pharmaceutical Stockpile. Decisions about the institution of
epidemiologic control measures are based primarily on the results of the
outbreak investigation. Decisions about whether to issue a surveillance alert to
clinicians, first responders, other public health officials, the intelligence
community, the media, and/or interested groups will also be based primarily on
these results. The desired outcome of this decision-making process could be
lives saved or morbidity prevented; it is affected primarily by the population's
infection status and by epidemiologic control measures.
In time period 3, clinicians are faced with the identification of subsequent cases. Their estimation of the pre-test probability of disease may be increased secondary to alerts from public health officials, thereby affecting subsequent testing, management, prevention, and reporting decisions.
, a double line indicates those decisions that
could be affected by DSSs, and the broken lines mark processes in which ITs
could play a role.
The information needed by clinicians and public health officials is that which will enable them to make appropriate decisions during a bioterrorism event. We will describe this in greater detail in the next section. Briefly, these information needs include: understanding the clinical presentations of patients exposed to biothreat agents, the best strategies for treating patients thought to have been exposed and for preventing the spread of disease to the unexposed, the characteristics of systems used to detect biothreat agents in the environment and diagnose disease resulting from these agents, the expected values and the thresholds of surveillance data that determine when outbreak investigation should be initiated, and the circumstances under which information should be communicated between interested parties.
| Task | Subtask | Key concepts | Data requirement/Evaluation criteria |
|---|---|---|---|
| Detection | Collection of the specimen | Environmental samples, suspicious materials, and clinical specimens require different collection methods. |
|
| Determination of the presence or absence of biothreat agents | Samples collected must be tested to determine the presence or absence of a potential biothreat agent. |
| |
| Diagnosis | Increase likelihood of performing an appropriate diagnostic test | Clinicians are more likely to perform a diagnostic test if they consider the probability of disease to be high. | Information that may affect a clinician's pre-test probability
assessment of disease includes:
|
| IT/DSSs may augment clinicians' work-ups by adding to differential diagnoses, providing disease-specific information, and suggesting diagnostic tests. |
| ||
| Interpretation of diagnostic test results | The interpretation of test results will depend on clinicians' knowledge of test characteristics and pre-test probability. |
| |
| Management | Triage | Triage decisions will be affected by the patient's clinical syndrome, diagnostic test results, availability of resources, and information provided by public health officials. |
|
| Treatment of acute patients | Management decisions will be affected by the patient's clinical syndrome, diagnostic test results, and information provided by public health officials. |
| |
| Safety management | Personal safety |
| |
| Hospital security |
| ||
| National security information |
| ||
| Reassure patients and staff |
| ||
| Prevention of spread of disease | Prophylaxis, quarantine, and isolation of exposed patients | Prophylaxis, quarantine, and isolation decisions will be affected by the patient's clinical syndrome, diagnostic test results, and information provided by public health officials. |
|
| Surveillance | Collection of surveillance data | Continual, systematic, and timely collection of data from multiple sources for the early detection of potential bioterrorism events. | Data from a variety of surveillance sources including:
|
| Analysis of surveillance data | Timely analysis and presentation of surveillance data to public health officials. |
| |
| Reporting and Communication | Communication between clinicians and public health officials | Clinicians' estimations of pre-test probability of disease will be affected by information from public health officials about local outbreaks. |
|
| Clinicians need information from public health officials regarding typical presentations of suspected cases to know who is at risk. |
| ||
| Public health officials need syndromal and morbidity data from clinicians to detect potential outbreaks. |
| ||
| Communication among national, state, and local public health officials | National guidelines for the management and reporting of infectious diseases are typically created by the CDC and disseminated via local public health officials. |
| |
| Communication between public health officials and hospitals | Coordination and sharing of protocols for hospital bioterrorism preparedness and response plans. |
| |
| Communication between public health officials and first responders | Coordination and sharing of protocols for field detection, triage, and management by fire, police, and hazardous materials personnel. |
| |
| Communication between public health officials and the intelligence community |
| ||
| Communication between public health officials and the public (both directly and via the media) |
| ||
| Maintaining security of information |
| ||
| Role in averting panic in the hospital |
|
| Type of systems | Number of systems a | Peer-reviewed articles | Government reports | Web sites | Other b | ||
| Evaluation | Description | Government | Non-government | ||||
| Detection and diagnostic | |||||||
| Collection systems | 4 | 0 | 0 | 3 | 0 | 4 | 2 |
| Particulate counters and biomass indicators | 14 | 1 | 1 | 9 | 2 | 10 | 7 |
| Rapid identification systems | 27 | 4 | 5 | 6 | 3 | 11 | 13 |
| Integrated collection and identification systems | 10 | 0 | 1 | 5 | 4 | 4 | 5 |
| General diagnostic DSSs | 6 | 3 | 3 | 0 | 0 | 6 | 1 |
| Radiologic systems | 4 | 2 | 3 | 0 | 0 | 0 | 0 |
| Diagnostic systems using telemedicine | 4 | 2 | 3 | 0 | 0 | 0 | 0 |
| Other diagnostic systems | 9 | 7 | 3 | 0 | 0 | 0 | 0 |
| Management and prevention | 18 | 10 | 5 | 0 | 1 | 2 | 2 |
| Surveillance (by type of data under surveillance) | |||||||
| Syndromal data | 7 | 0 | 2 | 3 | 1 | 2 | 1 |
| Clinical reports | 6 | 2 | 5 | 0 | 0 | 3 | 1 |
| Influenza data | 11 | 3 | 5 | 1 | 4 | 1 | 0 |
| Laboratory data | 12 | 4 | 6 | 2 | 3 | 0 | 1 |
| Antimicrobial resistance patterns | 11 | 1 | 6 | 0 | 4 | 2 | 2 |
| Hospital-based infections data | 16 | 10 | 9 | 0 | 1 | 0 | 0 |
| Foodborne illnesses data | 10 | 2 | 9 | 1 | 0 | 0 | 0 |
| Zoonotic and animal disease data | 6 | 0 | 4 | 1 | 1 | 0 | 0 |
| Other data | 11 | 2 | 3 | 0 | 5 | 0 | 4 |
| Reporting and communication | 26 | 7 | 9 | 1 | 4 | 5 | 2 |
| Integrated surveillance, communication, and command and control | 7 | 0 | 0 | 0 | 3 | 4 | 2 |
| Total number of unique systems | 217 c | 58 d | 82 | 32 | 36 | 54 | 43 |
In this table we present the type of evidence for each category of systems. For example, of the 4 Collection systems, none of the included reports were peer-reviewed evaluations; however, 3 of these Collection systems were described in at least 1 government report.
Other sources of information include conference proceedings, books, newspaper articles, and personal communications.
The total number of systems represented in each of these sections is 219; however, LDS Data Mining Surveillance System (DMSS) (Hospital data surveillance) and SymText (Radiologic systems) are part of the Health Evaluation through Logical Processing (HELP) system and are therefore not included in the total.
The included evaluation articles present data for 60 systems; however, LDS DMSS (Hospital data surveillance) and SymText (Radiologic systems) are part of the HELP system and are therefore not included in the total.
We reviewed 5,173 titles and abstracts from our preliminary searches of MEDLINE® and 11,515 additional titles and abstracts from the final search strategy (Appendix E). Of these, 822 articles were retrieved and cataloged, and 251 met inclusion criteria. These articles report on 162 systems (multiple articles report on the same system).
Of the 5 databases of peer-reviewed articles, only MEDLINE® and NTIS provided references for articles that described potentially relevant IT/DSSs. None of the articles retrieved from the following databases described potentially relevant IT/DSSs: The Catalog of U.S. Government Publications (search resulted in 51 citations), GrayLIT (search resulted in 55 citations), or Library of Congress (search resulted in 7 citations). Because they were searched using the same search terms, we have included the result of the NTIS search with the result of our search of government agency Web sites (Appendix F). Our review of 7,685 Web sites from our search of the NTIS and government agencies provided us with information on 29 systems (Appendix F).
Our review of 1,107 Web sites identified through the Copernic® Internet search added 27 systems to our database from 54 non-government Web sites (Appendix G).
In the sections that follow, we present the results of the information retrieved on IT/DSSs for detection and diagnosis, management and prevention, surveillance, and reporting and communication. For each of these categories of IT/DSSs, we present general information about the category, including the criteria by which these systems should be evaluated with respect to usefulness for bioterrorism (answer to Key Question 2), information on each system, and summary comments about the potential usefulness of those systems for clinicians and public health officials in the event of an attack (answers to Key Question 3 and 4). We first describe those systems that are commercially available, followed by systems that remain under development or are not currently commercially available (i.e., available only to government agencies or the military). We recognize that there are omissions from the group of systems under development and that numerous systems recently made available in response to the events of September 11, 2001 have not been included. In Appendix H, we present detailed information on the type of evidence found on each system.
We attempted to classify each system according to its principal stated purpose. The considerable overlap in the functionality of some of the included systems made this classification difficult. For example, the National Molecular Subtyping Network for Foodborne Disease Surveillance (PulseNet) is a communication system that facilitates the reporting of laboratory data. Because its primary purpose is communication, we describe it in that section. However, it could be reasonably argued that PulseNet is a component of a laboratory-based surveillance system and therefore should be discussed in the context of other laboratory systems. Similarly, some bioterrorism experts consider detection and diagnostic systems to be a subcategory of surveillance systems since the data generated by them can be integrated into bioterrorism surveillance systems. We present the data on detection and diagnostic systems separately because most of the systems that we describe were not designed for the collection of bioterrorism surveillance data -- although these data could certainly be used for that purpose.
For each of the sections that follow, we present our results in the following order: first, a brief statement of background information on the category of system; second, the criteria by which we attempted to evaluate the systems; third, the information that we found about the systems presented according to the evaluation criteria; and, fourth, summary comments about the systems and the evidence describing them. More detailed information about the included systems is presented in a Table at the end of the section and in the Evidence Tables at the end of this Report. An index of the systems is provided in Appendix I.
Our systematic review identified 55 detection systems for the rapid identification of potential biothreat agents in environmental and clinical samples, 6 general diagnostic DSSs used by clinicians to generate a list of possible diagnoses for a given patient, 4 radiologic systems that detect or diagnose interstitial lung disease or pneumonia, 4 telemedicine/teleradiology systems, and 9 other diagnostic systems (Tables 3-12; Evidence Tables 1 and 2).
The ideal detection system can identify the release of a biothreat agent before a single person becomes infected. Clinicians and public health officials require detection systems than can identify a covert release of a biothreat agent early enough to take action that limits the spread and progression of disease, as well as systems that rapidly evaluate environmental and clinical samples. A comprehensive discussion of biologic detection systems is beyond the scope of this project. We refer interested readers to detailed reports on this topic.10, 36, 37 However, insofar as they provide critical data, which could be used in an IT/DSS to inform diagnostic testing decisions and patient management, we conducted a limited review of the topic. Our discussion of these detection systems is largely informed by the Institute of Medicine's report "Chemical and Biological Terrorism: Research and Development to Improve Civilian Medical Response"10 and the North American Technology and Industrial Base Organization's report "Biological Detection System Technologies: Technology and Industrial Base Report."36
Typically, detection systems have 3 parts: a sampler/collector to concentrate the aerosol and preserve samples for further analysis; a trigger component (often a particulate counter or a biomass indicator) that can identify the presence, but not identity, of a possibly harmful biologic material; and an identifier to provide specific identification of the biothreat agent.36 Additionally, systems often require an information management system to record and send data to a command and control center.10, 36, 37
The relative importance of the characteristics of a detection system varies by its intended use. Identification systems for use by emergency personnel evaluating a suspicious powder in the field require different characteristics than identification systems used by clinicians faced with patients reporting flu-like illnesses and potential biothreat exposures. Depending on the use, the characteristics most desirable in a rapid detection system vary but typically include high specificity (also called the true negative rate or the likelihood that a sample without a biothreat agent has a negative test result), high sensitivity (also called the true positive rate or the likelihood that a sample with a biothreat agent has a positive test result), minimal sample preparation, ability to detect numerous biothreat agents, ability to run numerous samples simultaneously, no expensive or specialized reagents, small/portable size, ability to detect agents in real time (i.e., within minutes), ability to provide an output that is clearly interpretable by the decision maker, and low cost. Often, there is a trade-off between these characteristics such that a more sensitive test may have a higher false positive rate or be less timely than a less sensitive test. For any given detection system, the design of each component -- collector, trigger, and identifier -- will affect its portability, sensitivity, specificity and other detection characteristics.
In the sections that follow, we first describe collection systems, followed by particulate counters and biomass indicators, then identification systems, and, finally, integrated collection and identification systems.36 For each category, we present general information (e.g., how collection systems generally work, where they are used, and what the important criteria are by which they should be evaluated), a table briefly describing the purpose and relevant test characteristics (such as sensitivity and specificity) of the system, and summary comments describing the usefulness of these systems for bioterrorism preparedness. We provide substantial additional detail on each system in Evidence Table 1.
Collection systems are used to take samples from either the environment (e.g., air, water, or particulate matter from suspicious surfaces) or from a patient (or an animal) for later identification.10, 36, 37 The principal design considerations for collection systems are that the system must preserve and not harm the collected sample.36 Many collection systems collect airborne particles onto filters. For those identification systems that require a liquid sample, the collection system must take an aerosol or particulate sample and put it into liquid.10, 36, 37
| System name | Purpose | Flow rate/collection efficiency | Availability |
|---|---|---|---|
| BioCapture™ 42 | To serve as a portable, collection system for use by first responders. | The performance of BioCapture™ was compared to an All Glass Impinger (AGI) that collects samples into liquid and a Slit Sampler that impacts bacteria directly onto growth media. The collection efficiency was 50-80% relative to the AGI and 60%-125% relative to the Slit Sampler. | Currently available through MesoSystems Technology, Inc. |
| Portable High-Throughput Liquid Aerosol Air Sampler System (PHTLAAS)43, 44 | For the portable detection of aerosolized and insect-carried biowarfare agents. | No information available. | Currently available through Zaromb Research Corp. |
| Smart Air Sampler System (SASS) 2000 Plus™ Chem-Bio Air Sampler10, 36, 39, 45, 46 | For collecting aerosolized samples. | The portable system has a flow rate of 260 L/min and is designed to collect particles ranging in size from 2-10 micrometers (µm). | Currently available through Research International, Inc. |
| SpinCon® Advanced Air Sampler10, 36, 39-41 | For sampling both soluble vapors and particulate matter in public buildings, workplace exposure cases, and clean-room monitoring. | The system is capable of sampling over 1000 L/min and can operate in batch or continuous monitoring mode with automatic or manual controls. It is portable. | Currently available through the Midwest Research Institute. |
Note: for additional information on these systems, see Evidence Table 1.
The purpose of most of these systems is to collect aerosol environmental samples for use by first responders and for monitoring workplace exposures. We found no collection systems specifically designed for clinicians to obtain a clinical sample from patients (symptomatic or asymptomatic) with suspected exposure to a biothreat agent. Instead, most identification assays can be used on microbiological samples from nasal swabs, sputum, urine, blood, and cerebrospinal fluid collected in standard culture tubes. The descriptions of the 4 collection systems all report that they are portable, and the weight and size dimensions provided seem to justify this claim. The collection efficiency of the devices ranged from 0.5 to 10 microns. (Note that the size of the causative agent of anthrax is 1 to 5 microns, of smallpox is 0.15 to 0.3 microns, of plague is 0.5 to 2 microns, and of tularemia is 0.125 to 0.7 microns.38) Only the SpinCon® reports provided a flow rate, which for this device was 1000 liters per minute (L/min).10, 36, 39-41 None of the reports described methods for maintaining the security of the sample.
The only evaluation information on any of these systems was provided by the manufacturer of the BioCapture™ device, which has been used by fire departments in Seattle, Los Angeles, and New York, among others, and was evaluated at Dugway Proving Ground.42 The collection efficiency of BioCapture™ was reported to be 50 to 80 percent relative to the All Glass Impinger standard and 60 to 125 percent relative to the Slit Sampler Standard.42 (We found no additional evaluation information about these standard devices.)
These portable systems are potentially useful for the collection of environmental samples either as part of ongoing surveillance for a covert release of a biothreat agent or for evaluation of environments suspected of being contaminated. However, there is insufficient evaluative information to determine the utility of these systems for either of these purposes.
Two conditions should be met for collection systems to be maximally useful to first responders, clinicians or public health officials in the event of a bioterrorist attack. First, the collection system must be in use in the affected area. In the event of a covert attack, this is only possible if the collection system is already in place in areas of likely attack (e.g., airports; subways; major sporting, political, or entertainment events). In the event of a known attack, these systems must be portable enough that they can be taken by first responders to the location of suspected release and collect testable environmental samples. The manufacturers of many of these systems claim that they are portable and therefore meet this last condition. Second, the collectors must have sufficiently high flow rate and collection efficiency to be able to collect aerosolized biothreat agents should they exist. According to the available information, only the BioCapture™ device has been subjected to an evaluation of these performance characteristics. (However, we have no additional information on the devices against which it was compared, the All Glass Impinger and Slit Sampler.) Other considerations for which we have no information include: how difficult it is to train first responders in the use of these collection devices, how difficult it is to use these devices to collect samples in a secure manner in the event that they are used as evidence in a criminal investigation of the bioterrorist attack, and how much it would cost to fully implement these systems.
A sample collected from one of the collection systems just described could be directly analyzed by an identification system (see following section) to determine the specific identity of biothreat agents contained within the sample. This could be a reasonable analytic strategy when the probability of bioterrorism is particularly high, as in an announced attack. Under other circumstances, it may be preferable to first analyze the sample with a particle counter or biomass indicator to determine if the size, number, or properties of the particles collected suggest the presence of biothreat agents.10, 36 This may be particularly true for environmental samples routinely collected as part of a surveillance system for a covert bioterrorist attack.
Particulate counters and biomass indicators use a variety of methods to determine the presence of potential biothreat agents. We briefly describe 2 methods used by some of the included systems: flow cytometry and biosensor technology. We refer interested readers to reviews on these topics.10, 36, 37, 47-55 Our discussion of these systems was largely informed by these references.
Flow cytometers are increasingly common in U.S. hospitals and public health laboratories.10 Samples introduced into a typical flow cytometer are separated into individual cells.53-55 As cell flow across a laser beam, they scatter light in a characteristic manner and dyes bound to different parts of the cell emit light, or fluoresce.53-55 By measuring the fluorescence and scattered light of the sample, flow cytometers assess a variety of cellular characteristics including: cell size, amount of DNA, presence of specific nucleotide sequences, and cellular proteins.10, 36, 53-55 In an experiment at Los Alamos National Laboratory comparing the ability of flow cytometry and gel electrophoresis to measure bacterial DNA, flow cytometry was about 200,000 times more sensitive than gel electrophoresis (able to detect picogram quantities of DNA) and did so in 10 minutes compared with 24 hours for gel electrophoresis.55
Biosensors use data from living organisms to evaluate environmental samples for potentially toxic substances.37, 47-52 For example, canaries have traditionally been used in coal mines to detect toxic levels of methane gas and, more recently, as nerve agent detectors in the 1995 police raid of the Aum Shinrikyo compound in Japan.37, 47-52 Similarly, fish have been used to monitor water quality. On a smaller scale, data from extracellular recordings of excitable cell types (such as neurons and cardiomyocytes) have been evaluated for their physiologic responses to toxins.37, 47-52 Currently, cytokine production from immune cells used as biosensors for antigens is increasingly a target of research.56 Technical problems associated with the nutrient media required to keep biosensors alive have prevented cell-based biosensors from becoming widely available; however, these sensors may become a valuable part of the future detection armamentarium -- particularly for the detection of toxins and chemical agents.37, 47-52
We present information on 14 particulate counters or biomass indicators, 1 of which was reported in a peer-reviewed evaluation (Tables 3 and 5; Evidence Table 1; Appendix H). We found reports of 1 commercially available particle counter (the Met One Aerocet 531 Mass/Particle Counter) and 3 commercially available biomass indicators (the Digital Smell/Electronic Nose, Ameba Biosensor, and Spreeta™) that could be used for biothreat detection but have not been specifically tested for this application. We also found 10 systems that are currently under development or limited to military use, but could be helpful to public health officials for biothreat agent detection. (We report on additional biosensors in the next section on Identification systems. The biosensors listed in the Identification section are designed to specifically identify the presence or absence of a particular organism. In contrast, the biosensors described in this section are designed for the more general purpose of indicating the presence or absence of a biologically active compound.)
The purpose of most of the particulate counters is to detect a statistically significant increase in the number of particles in aerosol samples over baseline, and they were generally designed specifically for bioterrorism detection. In contrast, some of the biomass indicators were designed for the general detection of microorganisms, not specifically for bioterrorism. The systems vary widely in terms of size: some, such as the Met One Aerocet 531 Mass/Particle Counter, are designed for use in monitoring clean-rooms and are the size of a large handheld calculator, while others weigh 10 kilograms (kg), which is considerably larger but can still be easily carried by a single person. In contrast, several of the military-designed units are designed for mounting on a ship or motor vehicle. The size limits of particles that can be counted by the included systems varies from 0.3 to 20 micrometers (µm). The Model 3312A Ultraviolet Aerodynamic Particle Sizer (UV-APS)/Fluorescence Aerodynamic Particle Sizer-2 (FLAPS-2) was the only system for which we found information on the concentration of organisms that can be detected (10 agent-containing particles per liter of air). None of the systems reported methods for maintaining the security of the sample or data about the sample.
Of the 3 systems in this section that have been specifically tested for the detection of biothreat agents, only FLAPS-2, about which we have evidence from government reports and the manufacturer, has been tested for both sensitivity and specificity.10, 36, 39, 57, 58 In field tests, FLAPS-2 was able to detect 39 of 40 blind releases of simulant aerosols at a distance of about 1 kilometer with no false alarms over a 3-week period. A description of the Portable Biofluorosensor (PBS), obtained from a government report, suggests that false positives occasionally occur.43 The meaning of "occasionally" was not explained further. Of the 14 systems, the only evidence available in a peer-reviewed evaluation article was for the Digital Smell/Electronic Nose, a diagnostic system based on the volatile gases given off as metabolites by microorganisms.59 Holmberg et al. demonstrated that an array of 15 sensors was able to correctly classify 68 of 90 colonies containing 0 or 1 of 5 test organisms and an uninoculated control (22 of 90 were false positives). The commercially available Electronic Nose is marketed for the detection of microorganisms causing bacterial pharyngitis, pneumonia in ventilated patients, urinary tract infections, and bacterial vaginosis. This device has not been evaluated for the detection of biothreat agents.
Of the systems identified, the effort by the Department of Energy (DOE) called the Program for Response Options and Technology Enhancements for Chemical/Biological Terrorism (PROTECT) is particularly geared towards the detection of a covert bioterrorist attack. PROTECT uses a network of particulate collectors and counters set up in public places (e.g., subway stations) to monitor the environment for patterns suggestive of abnormal aerosols.60, 61 This project also facilitates the testing of collection devices and particulate counters. However, there are no published reports of these results.
Many of the particulate counters and biomass indicators are specifically designed to assist in the detection of a covert release of a biothreat agent. The lack of publicly available evaluation data on these systems precludes any conclusions regarding how useful they are likely to be in assisting in the bioterrorism preparations or response planning of clinicians or public health officials.
In particular, the sensitivity and specificity of these systems needs to be carefully evaluated and reported. Because the costs associated with delay in diagnosing a bioterrorism event can be significant in terms of excess morbidity and mortality, these systems must have demonstrated high sensitivity (i.e., low false negative rate). Similarly, because the costs of responding to false alarms and the potential that users may disregard systems with known high false positive rates, these systems must have sufficient specificity to avoid these hazards.
In addition to the criteria by which we evaluated these systems, the following are important considerations for particulate counters and biomass indicators to be useful to clinicians and public health officials. First, they must be located in close proximity to collection systems so that the samples collected may be routinely and promptly tested. Second, they must have the ability to promptly report results to decision makers (often at a remote location) who can either have the sample tested further for specific identification of the biothreat organism or take action to limit additional exposures to the suspected aerosol/contaminated environment. We were not able to assess the capacity of these systems to securely deliver such information to remote decision makers.
| System name | Purpose | Biothreat agent(s) identified | Sensitivity, specificity, or related performance measures | Availability |
|---|---|---|---|---|
| Antibody-based tests | ||||
| BioThreat Alert (BTA™) Strips90 | To provide field detection of biothreat agents using an antigen-antibody test. | B. anthracis, ricin toxin, S. enterotoxin B, botulinum toxins, and Y. pestis. | Current specifications for BTA™ test strips are available to emergency response or law enforcement officials upon request by calling 847-419-1507. | Marketed by Alexeter Technologies. |
| DOD Biological Sampling Kit (BSK)91 | For screening of suspicious packages and munitions for biothreat agents. | 8 assays, not otherwise specified. | Should not be used with soil samples as they may cause false positives. | Currently the DOD BSK is available for military use from the Joint Program Office for Biological Detection. |
| Fiber Optic Wave Guide (FOWG); Rapid Automatic and Portable Fluorometer Assay System (RAPTOR™); and Analyte 2000™ Biological Detection36, 46, 77-81 | To provide a portable biothreat identification system that uses antibody probes. | B. anthracis, ricin toxin, S. enterotoxin B, F. tularensis, V. cholerae, Y. pestis, E. coli O157:H7, Listeria, Salmonella, and Crypto-sporidium | Estimated detection levels (in water): B. anthracis (30-100 CFU/mL), ricin (less than 10 ng/mL), S. enterotoxin (1 ng/mL), F. tularensis (105 CFU/mL), V. cholerae (10 ng/mL), Y. pestis (levels below 1 ppb from samples of a few hundred µL). | Developed by the Naval Research Laboratory. Commercialized under a license to Research International, marketing the portable device as RAPTOR™. |
| Handheld Immunochro-matographic Assays (HHA)10, 36 | For the rapid detection of biothreat agents through a handheld antigen-antibody test. | Designed to identify 1 agent per assay. Can currently identify 8 threat agents (Y. pestis, F. tularensis, B. anthracis, V. cholerae, S. enterotoxin B, ricin, botulinum toxins, Brucella species) and 4 simulant agents. | The sensitivity of these assays varies from an order of magnitude below a fatal dose (ricin) to more than an order of magnitude above an infectious dose (anthrax). Positive results need to be confirmed with standard assays. | Produced by the Navy Medical Research Institute. Similar devices have recently become commercially available through Environmental Technologies Corporation. |
| Luminometer Rapid Detector82 | For rapid, portable detection of live bacteria on animal carcasses. | No information available. | Sensitive to low levels of bacteria (1000 organisms). | Available through New Horizons. |
| Sensitive Membrane Antigen Rapid Test (SMART™) and the Antibody-based Lateral Flow Economical Recognition Ticket (ALERT)36, 74, 82-84 | To detect biothreat agents through a rapid, portable antigen-antibody test. | B. anthracis, S. enterotoxin B, Y. pestis, botulinum toxins, ricin, Venezuelan Equine Encephalitis, and Brucella species.84 | No quantitative estimates available. See Evidence Table 1. | Available through New Horizons. |
| Nucleic acid-based tests | ||||
| Advanced Nucleic Acid Analyzer (ANAA)76/Handheld Advanced Nucleic Acid Analyzer (HANAA)92 (also called mini-PCR) | For field detection of biothreat agents using a portable, rapid, rugged system. | Limited only by the available probes. | Able to detect 500 CFUs of E. herbicola in 7 minutes.76 | Developed by Lawrence Livermore National Laboratory. |
| DNA biochip93 | For the rapid identification of biothreat agents using microelectro-optical probes such as DNA. | No information available. | No information available. | Developed by Oak Ridge National Laboratory. |
| Field Kit for Rapid Detection of Anthrax94 | For the rapid detection of B. anthracis in environmental or clinical specimens. | B. anthracis. | Reported to have a low false positive rate even in specimens that contain closely related Bacillus species and other microorganisms.94 | Lawrence Berkeley National Laboratory is seeking an industrial partner to commercialize a field diagnostic kit. |
| GeneChip® (LifeChip High-Density Nucleic Acid Microarrays)10, 95 | For the rapid, simultaneous detection of numerous nucleic acids of biothreat agents or other pathogens. | The number of identifiable agents is limited by the development of probes. | No information available. | Available through a collaboration between Lawrence Livermore National Laboratory, the U.S. Army Medical Research Institute of Infectious Diseases (USAMRIID), and Affymetrix, Inc. |
| LightCycler™; Ruggedized Advanced Pathogen Identification Device (RAPID™); and Lightweight Epidemiology and Advanced Detection Emergency Response System (LEADERS)85 | LightCycler™ is an ultra rapid PCR cycler with a built-in detection system for real-time quantification of DNA samples. RAPID™ is a rugged, portable system that uses LightCycler™ technology for field detection of biothreat agents. LEADERS is a medical surveillance tool that provides real-time analysis of data coming from various sources to identify the presence of a biothreat agent. | Limited only by the available probes. Can assay for 10 unknown organisms per run. | A function of the probes used. Per the manufacturer, RAPID™ is reported to be 99.9% specific. For each assay, the sensitivity is set to half the infective dose (for example, the infectious dose of Foot and Mouth Disease is 10 virus particles; RAPID™'s sensitivity is set to detect 5 virus particles.)85 | Available through Idaho Technology and Roche Diagnostics. |
| MicroArray of Gel Immobilized Compounds on a Chip (MAGIChip™) 10, 36, 96 | For the rapid screening of drug-resistant mutations in Mycobacterium tuberculosis. | Limited by the probes used on the array. | No information available. | Developed by Argonne National Laboratory and the Russian Academy of Sciences. |
| SmartCycler® and GeneXpert™ 86, 87 | For real-time nucleic acid-based detection of organisms in laboratory and field locations. | Depends on available probes. | Reported to be specific to 12 B. anthracis strains tested and able to detect 5 genome copies. Up to 4 targets can be detected in 1 sample. | Commercialized by Cepheid, Inc. |
| Mass spectrometry-based test | ||||
| Pyrolysis-gas Chromatography-ion Mobility Spectrometer (PY-GC-IMS)43, 97 | For portable detection and identification of biological aerosols. | Protein toxins, bacteria, and sporulated bacteria. | No quantitative estimates available. See Evidence Table 1. | Recently developed in a joint partnership between Edgewood Chemical Biological Center and the University of Utah. |
| Flow cytometry-based test | ||||
| MiniFlo55, 98 | For rapid, portable detection of multiple biological agents using an innovative approach to flow cytometry. | Y. pestis and B. anthracis, as well as other viruses, bacteria and proteins. | Detected 87% of unknown biological agent simulants, including agents similar to anthrax and plague, with a false positive rate of 0.4% at the Dugway, Utah Field Trials in 1996.98 | Developed by Lawrence Livermore National Laboratory. |
| Tests based on other technologies | ||||
| AK (Adenylate kinase) Phage Biosensor99, 100 | To provide rapid, automated diagnosis of infectious diseases using the AK Phage technique. | Uses bacteriaphages (special viruses that infect particular bacteria) to identify bacterial threat agents. See Evidence Table 1. | No information available. | Available through a joint effort between the UK's Defense Evaluation and Research Agency (DERA) and Acolyte Biomedica Ltd. |
| Anthrax Sensor101 | To provide highly sensitive, portable detection of biological agents in seconds. | B. anthracis, other endotoxins. | Capable of detecting endotoxins at a level that is 20 times lower than previously achieved by similar devices.101 | Currently under development at Virginia Tech Pharmaceutical Engineering Institute. |
| Australian Membrane and Biotechnology Research Institute (AMBRI) Biosensor Technology102, 103 | For highly sensitive and specific detection of a variety of biothreat agents using a cell-based model. | Phage display antibody libraries are available for Y. pestis, in addition to monoclonal and polyclonal antibodies for Y. pestis, F1 antigen, B. anthracis, and C. burnetti. | Current sensitivity to bacteria is at 3000 CFU/mL, with further sensitivity enhancement strategies underway, and with a response time of 2 minutes.102, 103 | Currently under development at AMBRI. |
| Biolog88 | For detection of microorganisms, with possible uses for B. anthracis. | B. anthracis. | See Evidence Table 1. | Available through Biolog, Inc. |
| Biosensor for E. coli104, 105 | For the rapid detection of E. coli, using a simple change in color to denote the presence of the bacteria. | E. coli. | No information available. | Developed by Lawrence Berkeley National Laboratory. |
| CellChip™ 106, 107 | To perform a high-throughput and high-content analysis of intact cells. | B. anthracis. | No information available. | Developed by Cellomics, Inc. |
| Fluorescence-based array immunosensor89 | To provide simultaneous, antibody-based detection of bioactive analytes in clinical fluids such as whole blood or from a nasal swab. | Includes S. enterotoxin B and F1 antigen from Y. pestis. | Unable to detect physiologically relevant S. enterotoxin B levels (<125 ng/mL) in experimentally spiked urine, saliva, and blood products; sensitivity for F1 antigen from Y. pestis at 25 ng/mL. | Developed by Naval Research Laboratory and Geo-Centers, Inc. |
| Nitric Oxide (NO) Sensor108, 109 | For sensitive, rapid detection of biothreat agents. | No information available. | No information available. | Currently under development by DARPA. |
| Optical fluorescence biosensor technique110, 111 | To provide a reagent-free technique for the rapid detection of biological toxins and pathogens. | Capable of identifying specific protein toxins, such as the cholera toxin. | Sensitivities of less than 50 parts per trillion have been demonstrated. | Developed by Los Alamos National Laboratory. |
| RealTime BioSensor™ 112 | For automated, rapid detection of a wide variety of biological pathogens. | Includes airborne pathogens, E. coli 0157:H7, and Salmonella. | Capable of detecting as low as 100 particles of bio-contaminants in samples ranging from milliliters to liters. | Available through MesoSystems Technology, Inc. |
| Tissue-Based Biological Sensor (TBBS)113, 114 | For detection of biological pathogens using a technique that mimics the body's own immune response. Capable of detecting new organisms that have not been identified at the molecular level. | No information available. | No information available. | Currently under development at DARPA. |
| Upconverting Phosphor Technology115 | To provide rapid detection and identification of pathogens in the field while maintaining a high sensitivity and specificity. | Up to 9 agents can be identified simultaneously given available probes. See Evidence Table 1. | Detected picogram levels of small (e.g., virus or toxin) target antigens in samples of less than 1 mL.115 The goal for detection of spores and bacteria is below 100 organisms/mL. | Available through SRI International. |
Note: for additional information on these systems, see Evidence Table 1.
Antibody-based systems use antibodies developed to recognize specific targets on either antigens or cells of interest to detect potential pathogens.10, 36 An advantage of these systems is that the use of antibodies confers high specificity10, 36 The antigen-antibody binding can be monitored directly or indirectly. For example, sandwich assays use a second antibody, labeled with a fluorescent dye that binds to either the antigen itself or probe antibody to monitor antigen-antibody binding.10 The detection thresholds of these methods vary between 103 to 104 microbial cells per milliliter (mL).10 Technical problems with antibody-based sensors include nonspecific binding (which can lead to false positive results), cross reactivity, and degradation of the antibodies over time (which can lead to false negative results).10 Despite these technical problems, antibody-based systems can be both highly sensitive and specific.10, 36
In response to the recent cases of anthrax in the U.S., considerable interest has been generated in the use of handheld antibody-based detectors by first responders. The CDC recently issued a statement on its Web site stating that the analytical sensitivity of these assays is limited and that a minimum of 10,000 spores is required to generate a positive signal.75 Given concerns about the sensitivity and specificity of these kits, the CDC has undertaken an independent evaluation of these tests. Conclusions from this study are expected in the near future.75
The specificity of nucleic acid-based systems (sometimes called polymerase chain reaction- or PCR-based systems) is derived from the selective binding of nucleic acid probes to complementary nucleic acids from the pathogen of interest.10 Probes are designed to bind specifically to a nucleic acid sequence that is unique to the pathogen or to identify a nucleic acid sequence that is common to several pathogens.10 The sensitivity of nucleic acid-based systems for bacteria is between 1,000 and 10,000 colony forming units (CFU);10 however, recent reports suggest that they may be capable of greater sensitivity.76 Because the reaction occurs within minutes, the time-consuming parts of using nucleic acid systems are the sample preparation and the time required to detect the signal.10, 36 Significant limitations to the use of these methods for bioterrorism include the lack of highly specific probes for all biothreat agents (although the DOE and CDC have entered a collaboration to develop them) and the use of a single probe to test a single sample for the antigen of interest at a given time. Given security concerns, the distribution of highly specific probes will likely remain under strict federal control -- first responders are not likely to have access to these probes for testing samples in the field.
Mass spectrometry is an analytical technique in which materials under analysis are converted into gaseous ions or other characteristic fragments.10, 36 The fragments are separated on the basis of their mass-to-charge ratio.10, 36 The technique can reportedly detect concentrations of as low as 106 cells.10 When samples are tested in the field, they are likely to contain multiple constituents (contaminants), which must be separated before they can be reliably identified. This separation can be performed by a variety of techniques, including mass spectrometry.10, 36
We note that some technologies are better suited to particular agents. Nucleic acid-based systems, for example, cannot detect toxins (unlike bacteria and viruses, they do not contain nucleic acids). In contrast, mass spectrometry is more effective for the detection of toxins than bacteria.
In general, the purpose of these systems is to rapidly detect biothreat agents collected from environmental, human, animal, or agricultural samples. The available antibodies limit the antibody-based tests. Assays are commonly reported to be available for Y. pestis, F. tularensis, B. anthracis, V. cholerae, S. enterotoxin B, Brucella species, ricin, and botulinum toxins. Many of these systems are small (portable) enough for use in the field. They all test for a single biothreat agent per assay and run a single assay at a time (except for the Fiber Optic Wave Guide (FOWG)/Rapid Automatic and Portable Fluorometer Assay System (RAPTOR)/Analyte 2000™ Biological Detection system, which can run 4 assays simultaneously). Reports suggest that a result can be obtained from the handheld antibody tests in 5 to 45 minutes. However, several of the reports of these systems suggest that they are prone to false positives (typically attributed to soil contamination). The FOWG/RAPTOR/Analyte 2000 (described in a peer-reviewed article,77 government report,36 and by the manufacturer46, 78-81) has the following estimated detection levels (in water): B. anthracis (30-100 CFU/mL), Ricin (less than 10 nanograms per milliliter (ng/mL)), S. enterotoxin (1 ng/mL), F. tularensis (105 CFU/mL), V. cholerae (10 ng/mL), and Y. pestis at levels below 1 parts per billion (ppb) from samples of a few hundred microliters (µL). In their peer-reviewed evaluation of tests for agroterrorism, Von Bredow and colleagues reported that the Luminometer Rapid Detector, designed for portable quantification of the live bacteria on animal carcasses, could detect the presence of 1000 or more organisms (no additional information provided).82 This article also briefly reported on the Sensitive Membrane Antigen Rapid Test (SMART™) and the Antibody-based Lateral Flow Economical Recognition Ticket (ALERT) assays, for which we also had information from 2 government reports and the manufacturer. The government document reported that during battlefield tests, the SMART™ system had an "alarmingly" high false positive rate.74 However, the manufacturer reported that the Bengal SMART™ test for V. cholerae O139 has 99 percent sensitivity and 99 percent specificity, the Cholera SMART™ test for V. cholerae O1 can detect as few as 2x103 organisms, and the BengalScreen and CholeraScreen (coagglutination tests for V. cholerae O139 and O1 respectively) each have a sensitivity of 96 percent and a specificity of 94 percent.83, 84
The nucleic acid tests are similar to antibody-based tests in that they are limited by the availability of probes and only test for a single biothreat agent per assay. We have limited information on the availability of highly specific probes (which usually have to be obtained separately from the machinery itself); however, the descriptions of these systems suggest that probes can be obtained at least for B. anthracis and Y. pestis. Unlike antibody-based systems, many more of the nucleic acid-based systems are designed to run multiple assays at a time (as many as 16 in the currently available systems and tens to hundreds of thousands in some of the gene-chip/micro-array technologies that are currently under development) and can do so in 7 to 60 minutes. Many of these systems are small enough to be carried by a single person but, as a group, they tend to be larger than the antibody-based systems. In terms of their sensitivity and specificity, the Advanced Nucleic Acid Analyzer (ANAA) was described in a government report to be able to detect 500 CFUs of Erwinia herbicola.76 Per the manufacturer, the LightCycler™/Ruggedized Advanced Pathogen Identification Device (RAPID™) was reported to be 99.9 percent specific with the sensitivity set for each assay at half the infective dose (e.g., the infectious dose of Foot and Mouth Disease is 10 virus particles; therefore, RAPID™'s sensitivity is set to detect 5 virus particles.)85 The SmartCycler® and GeneXpert™, for which we have information from conference proceedings86 and the manufacturer,87 is reported to be "specific to 12 B. anthracis strains tested and able to detect 5 genome copies."
Of the other types of rapid identification systems, 2 were presented in peer-reviewed evaluation articles. Biolog is a general identification system for microorganisms, with potential utility for identification of B. anthracis. Using an in-house database (as opposed to the database that is currently commercially-available and does not include B. anthracis), Biolog correctly identified all samples of B. anthracis with "readable profiles" (19 out of 20). However, it falsely identified 5 out of 12 closely related Bacillus strains as B. anthracis (false positives). For both B. anthracis and related strains, roughly 20 percent of the samples gave uniformly false positive reactions, in which all reaction wells were positive.88 The Fluorescence-based array immunosensor is designed for the simultaneous, antibody-based detection of bioactive analytes in clinical fluids such as whole blood or from a nasal swab in less than 35 minutes. However, in an evaluation of this system, it was unable to detect physiologically relevant S. enterotoxin B levels (less than 125 ng/mL) in experimentally spiked urine, saliva, and blood products, and the detection limit for F1 antigen from Y. pestis was 25 ng/mL.89
Many of the rapid detection systems provide outputs that can be made available in electronic format. Several of these systems can be used with a laptop computer for storage and communication of test results to remote decision makers. None of the reports of these systems described security measures for handling the samples or the test results.
The rapid identification of biothreat agents in environmental samples is essential for a swift response to either covert or announced bioterrorist attacks. Without the availability of accurate rapid detection methods, first responders, and clinicians cannot make decisions about triage, management, or prevention of the spread of disease. However, adequate evaluative data about these critical systems have not been published. The available evaluation information suggests that, although the systems are convenient in terms of their portability and speed, they have high false positive rates. Without additional information about the sensitivity and specificity of these tests, their results are uninterpretable: users cannot readily determine the appropriate action to take given a sample producing either a positive or negative result. Reports of independent evaluations of currently available systems against gold standards represent a major gap in the literature.
Additional limitations to the usefulness of these systems for the rapid identification of biothreat agents are: the absence of tests for many of the most worrisome biothreat agents (e.g., smallpox); the lack of available probes for many nucleic acid tests; and the paucity of tests that facilitate the evaluation of a given sample for more than 1 biothreat agent. Similarly, since the identification systems described in this section tend to test for a single organism at a time, and tend to have the ability to detect only a limited number of biothreat agents, a negative result cannot be interpreted as being negative for a bioterrorism event, merely as being negative for the limited number of organisms detected by that assay.
The reports of the antibody-based technologies for the identification of anthrax did not typically specify whether the antibodies recognize the spore or vegetative form of the bacteria. This is an important determinant of the technology's utility. For example, antibodies against the vegetative form would not be useful for monitoring environmental samples (since anthrax is commonly found in the soil and could contaminate environmental samples) but would be useful for screening clinical samples.
In addition to the IT/DSSs that we described in the preceding sections, some systems combine collection and identification functions in a single unit. Often, these systems have a communication component that allows reporting of the results, typically to a remotely located command and control post. Most of these systems have been developed by the military and are likely to have been rigorously evaluated to meet Department of Defense (DOD) standards; however, much of the evaluative data are not publicly available.
| System name | Purpose | Biothreat agent identified | Sensitivity and specificity | Availability |
|---|---|---|---|---|
| Biological Aerosol Sentry and Information System (BASIS)117 | To serve as an early warning of airborne biological incidents for special events such as major sporting events and political meetings through a network of distributed sampling units deployed around the target area. Samples are regularly retrieved and brought to a field laboratory for analysis. | Agents identifiable via PCR techniques (therefore, limited by availability of reagents) | No information available. | In early deployment and testing. |
| Biological Agent Warning Sensor (BAWS) and Joint Biological Point Detection System (JBPDS)10, 36, 39, 91, 118, 119 | To detect biological agents in aerosol samples. | 10 agents (not otherwise specified). | During field-testing, the system experienced "many false positives," and there were "significant human factors deficiencies: operators in protective gear experienced difficulties, particularly in assembling and disassembling the system."36 | Lockheed Martin currently produces both BAWS and JBPDS. |
| Biological Integrated Detection System (BIDS)10, 36, 66, 120 | To serve as a vehicle-mounted continuous air sampler to determine the background distribution of aerosol particles. | B. anthracis, Y. pestis, botulinum toxin A, and S. enterotoxin B. | No information available. | Available through Battelle. |
| Canadian Integrated Biochemical Agent Detection System (CIBADS II) and 4WARN57, 116 | To provide a networked system for the detection of a broad spectrum of chemical and biological agents. | Determined by the identification system used. | No quantitative estimates available. Affected by weather. See Evidence Table 1. | Available through Computing Devices Canada. |
| Joint Biological Remote Early Warning System (JBREWS)121 | To provide a network of sensors with communication links to a command post. | Determined by the identification system used. | No information available. | Developed in collaboration among Lawrence Livermore Laboratory, Johns Hopkins Applied Physics Laboratory and Los Alamos National Laboratory. |
| Joint Service Warning And Reporting Network (JWARN)36, 66 | To serve as an automated nuclear, biological, and chemical information system that can integrate the data from detectors and sensors into the Joint Service command. | No information available. | No information available. | Available through the DOD. |
| Mobile Atmospheric And Sampling Identification Facility (MASIF)122 | For the collection and testing of aerosol samples for evidence of biothreat agents and communication of these findings to a central command location. | Determined by the assay used for identification. | No information available. | Available through the Canadian DRES. |
| Multi-Purpose Integrated Chemical Agent Alarm (MICAD)123 | To serve as a lightweight, automated nuclear-biologic-chemical detection, warning, and reporting system. | No information available. | No information available. | Developed by Lockheed Martin. |
| Nuclear-biologic-chemical (NBC) Field Laboratory10, 36, 39, 124, 125 | To detect and identify any kind of biological warfare agent or any other agent of biological origin representing a health risk to soldiers on the battlefield. | No information available. | No information available. | Available through Rheinmetall Landsysteme, Germany. |
| Portal Shield Air Base/Port Biological Detection System66 | To serve as a rapid, automated system that integrates data from multiple sites for outbreak detection. | 8 agents, not otherwise specified. | The system has a theoretical false positive rate of 0.25%. In practice, it has not had any false positives during over 10,000 assays.66 | Designed by the DOD. |
Note: for additional information on these systems, see Evidence Table 1.
These systems are generally intended to transmit test results electronically to decision makers at some distance from the collection and identification site(s). They have all been designed for military use but may be increasingly available to interested public health officials and national security professionals for ongoing environmental surveillance. These systems are the size of refrigerators or larger and therefore require trucks or similar vehicles for transportation. The Canadian Integrated Biochemical Agent Detection System (CIBADS II)/4WARN system, designed to collect and identify a variety of chemical and biological agents from a commercial sport utility vehicle, is radio-linked to a command and control unit. An evaluation of CIBADSII/4WARN reported in a government document57 and by the manufacturer116 suggested that the system was operated at speeds up to 50 miles per hour "without significant degradation of performance."57, 116 The impact of weather patterns on performance was also determined to be low. The exception was immediately after a thunderstorm, when the number of particles in the air rose dramatically and reduced the sensitivity of the system.57, 116 A government report provided evaluation data on the Portal Shield Air Base/Port Biological Detection System, which integrates data from multiple sensors linked to a centralized command post computer.66 This computer monitors the sensors and evaluates the data to determine if a bioterrorist attack has occurred. In the event a release is detected, the computer alerts the operator. The algorithm looks for a significant increase in at least 2 sensors before it will sound an alarm, giving the system a theoretical false positive rate of 0.25 percent. The report stated that, "after having gone through over 10,000 assays, the Portal Shield system has not had any false positives."66 The system can reportedly detect 8 agents, although they were not further specified.66 None of the reports of these integrated detection systems described the methods for maintaining the security of the sample or test results about the sample; very few details were provided about specific collection or identification components.
Systems that integrate collection and identification (often with communication) functions have potential utility for the detection of a covert release. The large size of several of these integrated systems prohibits their use by first responders and clinicians. However, public health officials and municipal leaders may be interested in using these systems for ongoing surveillance for bioterrorism events in public spaces considered to be likely targets. The military developers of these systems may have performed comprehensive assessments of their important test characteristics; however, no published peer-reviewed evaluative data are currently available to the general public. This lack of evaluative information prohibits drawing conclusions regarding the utility of these systems by groups outside the military.
The collection, particulate counters and biomass indicators, rapid identification, and integrated collection and identification systems described in the preceding sections have critical roles to play in the detection of a covert release of a biothreat agent. In addition, they are required by first responders and clinicians to test environmental and clinical samples in a known release. However, the paucity of comprehensive evaluative information about these systems prevents conclusions about whether or not one or more of these systems is likely to be useful for these purposes.
The evidence on detection systems was descriptive and predominantly collected from government sources and manufacturers' Web sites. We note that the definitions of what constitutes a "rapid" or "portable" test varied widely. We found no reports that directly compared 2 or more of the commercially available systems in any given category. Additionally, few of these systems have been compared to a gold standard, and their sensitivity and specificity remain poorly characterized. A significant gap in the literature is an analysis performed by an independent research group comparing the most promising technologies to each other and to the gold standard. For most systems, the available information does not describe if reagents are sold with the detector or if they are widely available. We conclude that potential users of these systems must carefully evaluate the data derived from them and consider strategies that include the use of these systems for rapid detection in conjunction with the slower but better-validated methods used in reference laboratories.
General diagnostic DSSs are designed to assist clinicians in generating a list of possible diagnoses for a given patient. For such systems to be useful in the event of a covert bioterrorist attack, they should prompt clinicians to consider biothreat agents as a potential cause of the patient's symptoms. In this way, these systems may increase the clinician's suspicion of bioterrorism, thereby increasing the probability that the clinician performs appropriate diagnostic testing. Most of these systems require that the clinician enter information about the patient's signs and symptoms. Typically, the diagnostic DSS then produces a differential diagnosis or list of possible diagnoses for the patient. These diagnoses are sometimes ranked according to the likelihood of disease. Alternatively, some DSSs provide a calculated probability score for each diagnosis, often based on a clinical prediction rule.
Our search found 6 currently available general diagnostic DSSs, 3 of which have been clinically evaluated and presented in peer-reviewed reports (Tables 3 and 8; Appendix H).
The purpose of each of these systems is to provide a differential diagnosis based on patient-specific signs and symptoms. Because general diagnostic DSSs typically provide a list of several possible diagnoses, in the event of unrecognized bioterrorism-related illness, even if the system fails to rank the correct diagnosis first, but ranks it among the top few diagnoses, this may prompt a clinician to order a diagnostic test for a biothreat agent.
All of the general DSSs require manual entry of patient information by clinicians. They then use either Bayesian (probabilistic) and/or rules-based methods to compare the patient's information with their knowledge base to generate a differential diagnosis that is typically ranked in descending order of likelihood. Some of the systems provide additional information about the suspected diseases and suggest appropriate tests if clinicians choose to pursue these diagnoses. Most of the general DSSs are available for use on personal computers, although a handheld version of DiagnosisPro® is also available. None of the reports described if it was possible to update the probability of biothreat-related illness as the epidemic progresses. No study of a general diagnostic DSS has specifically evaluated the performance of these systems for the diagnosis of biothreat-related illness.
DXplain™, Iliad, and Quick Medical Reference (QMR) were directly compared in a multi-center trial of 105 diagnostically challenging cases.126 The DXplain™ knowledge base contained the correct diagnosis for 96 cases (91 percent); the Iliad knowledge base contained the correct diagnosis for 80 cases (76 percent); and the QMR knowledge base contained the correct diagnosis for 77 cases (73 percent). DXplain™ correctly included the ultimate diagnosis in 72 cases (69 percent) with an average rank of 12.4, compared with 64 cases (61 percent) with an average rank of 10.4 for Iliad and 55 cases (52 percent), at an average rank of 6.6 for QMR. (The clinical significance of this difference in rank is not clear. The importance of rank depends on how this information is used. For example, if the clinician only scans the top 5 diagnoses or if the DSS only prints out the top 10 diagnoses, then the rank may well be important. If, however, the clinician reviews the entire list of possible diagnoses specifically seeking the unusual diseases that he or she had not previously considered as a means of enhancing their diagnostic capabilities, then rank is less important.) When considering only the 63 cases for which the correct diagnosis was present in all systems, Dxplain™ identified the correct diagnosis in 50 cases (79 percent) at an average rank of 11.7. Iliad was correct in 48 cases (76 percent) at an average rank of 10.2 and QMR correctly identified the final diagnosis in 45 cases (71 percent) with an average rank of 5.4.
The other evaluations of the general diagnostic DSSs differed with respect to their study designs. Some evaluated physician acceptance of the system. However, high acceptance does not necessarily mean that a clinician would use the system for routine cases (such as a patient presenting with a flu-like illness, a common early presentation of many biothreat-related illnesses). Other study designs addressed the observed phenomenon that different clinicians use a different set of diagnostic terms to describe the same patient.127-129 These differences may result in the DSS producing differing lists of diagnoses. Therefore, some studies compared the terms input into a system by different clinicians, given the same case, and the resulting differential diagnoses.
As of the publication of this Report, the manufacturer of Iliad has stopped selling and providing technical support for that system.130 We have nonetheless included Iliad in this section because it continues to be available through some retailers, and clinicians continue to use this product.
The role of general diagnostic DSSs in a bioterrorism response is to enhance the likelihood that clinicians consider the possibility of bioterrorism-related illness. Therefore, these systems could contribute to the detection of a previously unrecognized release of biothreat agents. However, the reports of general diagnostic DSSs have several important limitations that prevent conclusions regarding their ability to serve this role. First, none of the DSSs has been evaluated formally with respect to bioterrorism response. Second, all of these systems require laborious manual entry of patient findings, which may be a substantial barrier to use in clinical settings. Efforts to link general diagnostic DSSs to other hospital information systems, if successful, would reduce the data entry burden substantially. In addition, availability of the system on a handheld computer (as for DiagnosisPro®) might make the system more convenient for clinicians to use. Third, available evaluations do not indicate whether disease caused by biothreat agents are included in the databases for many systems. Thus, we were not able to assess the extent to which biothreat agents are included in any of the general diagnostic DSSs knowledge bases or whether the systems are updated with new information about the clinical presentations of these diseases (except that Iliad has not been updated since 1997). Fourth, general diagnostic systems that use probabilistic information about the likelihood of disease will have inappropriately low pretest probabilities for biothreat agents in the event of a bioterrorism event. To provide a ranking of differential diagnoses, the system relies on estimates of the prevalence or probability of diseases. If a biothreat outbreak was known or strongly suspected, the pretest probability for these agents would change dramatically from the probabilities appropriate during routine clinical use. It would be helpful if the knowledge base could be updated to reflect changes in the likelihood of diseases based on local public health data (i.e., if the system were automatically updated with local incidence and prevalence information) or could be modified in the context of a known bioterrorism event.
Because many biothreat agents cause pulmonary disease, chest X-rays would be a common diagnostic procedure performed on patients presenting after a bioterrorism event. Interstitial disease would be the most likely finding. In the case of inhalation anthrax, a widened mediastinum may be seen; however, this is not always present, even in some advanced cases.152
Radiology interpretation systems may increase the diagnostic accuracy of radiographic reports. For this Report, we limited our search to those technologies that could be used to automate the interpretation of radiologic images for the diagnosis of biothreat agents. For example, we excluded those systems that detect mammographic lesions or pulmonary nodules. In this section, we discuss 2 types of systems -- those that assist clinicians in the interpretation of radiographic images, and those that use natural language processing methods to abstract information from the reports of radiographic procedures for diagnostic purposes.
Our search found 2 radiologic interpretation systems, 1 of which has been clinically evaluated and described in a peer-reviewed article (Tables 3 and 9; Appendix H).
The first system, described in 3 evaluation articles, scans digitized radiographs for abnormal regions to assist clinicians in the identification of pulmonary infiltrates.153-155 These studies calculated receiver operating characteristic (ROC) curves for each of the systems under evaluation. ROC curves are a plot of the sensitivity of a diagnostic test (typically on the y-axis) against 1 minus its specificity (typically on the x-axis). Because the ideal diagnostic test is 100 percent sensitive and 100 percent specific, the area under an ideal ROC curve would be equal to 1. Minimal improvements in the area under the ROC curve were shown when computer-aided diagnosis was employed for a small set of radiographs. Since the vast majority of infiltrates will not be related to biothreat agents, it remains unclear whether this technology can be translated into improved detection of bioterrorism-related illness.
Researchers at the same institution also found that using an artificial neural network can improve the performance of radiologists in the differential diagnosis of interstitial lung disease.155 When chest radiographs were viewed in conjunction with network output, the average area under the ROC curve increased from 0.83 to 0.91. The clinical significance of such a change is not clear.
None of the reports described whether the biothreat agents and their associated illnesses are included in the knowledge base, whether the system uses a standard vocabulary, information regarding the ability to update the probability of biothreat-related illness as the epidemic progresses, the type of hardware required, or the system's security measures.
| System name | Purpose | Description | Diagnostic accuracy | Contact information |
|---|---|---|---|---|
| SymText156, 158, 159 | To identify patients with pneumonia through radiology reports. | SymText is a medical language processing system developed by LDS hospital in Utah. It can be used to analyze radiology reports for specific clinical concepts associated with specific disease processes. Preliminary studies show that SymText is similar to physicians in its ability to identify patients with pneumonia through radiology reports. SymText can also be integrated into a real-time DSS for implementation of automated guidelines for community-acquired pneumonia. | In selecting patients who are eligible for the pneumonia guideline, the area under the ROC curves was 89.7% for SymText and 93.3% for physicians. Average sensitivity, positive predictive value, and specificity for radiographic findings that assessed location and extension of pneumonia was 94%, 87%, 96% for physicians, and 34%, 90%, 95% for SymText, respectively. | Department of Medical Informatics University of Utah School of Medicine 30 North 1900 East - Room AB193 Salt Lake City, UT 84132-2913 Ph: 801-581-4080 |
| Columbia-Presbyterian Medical Center Natural Language Processor157 | To automate the identification of 6 disease processes through analysis of radiology reports. | The system looks for radiographic reports with appropriate findings that correlate with 6 conditions: congestive heart failure, pneumonia, pleural effusion, malignancy, pneumothorax, and chronic obstructive pulmonary disease. | There was no significant difference in the degree of disagreement between the physicians themselves and the natural language processor. For each of the radiographic reports, subjects were asked to note the presence or absence of each of the 6 conditions. One point would be assigned for every disagreement. The average intersubject disagreement among physicians was 0.24 out of a maximum of 6 while the average disagreement of the natural language processor from the physicians was 0.26. The system had a sensitivity of 81% (95% confidence interval (CI): 73%-87%) and a specificity of 98% (95% CI: 97%-99%); physicians had an average sensitivity of 85% and specificity of 98%.157 | Department of Medical Informatics Columbia-Presbyterian Medical Center 161 Fort Washington Avenue, AP-1310 New York, NY 10032 |
The purpose of these programs is to search electronic text for concepts related to pneumonia, and then either alert the clinician or incorporate this information with other data from the electronic medical record into diagnostic or management applications. Neither of the medical language processing systems that we found was specifically designed to diagnose bioterrorism-related illness. None of the reports of these systems described whether the biothreat agents and their associated illnesses were included in their knowledge bases or whether the systems used standard vocabularies, nor did they provide information regarding the ability to update the probability of biothreat-related illness as the epidemic progresses, specify the type of hardware required (minimally, each required an electronic medical record system), or describe the system's security measures.
Two studies have evaluated the ability of medical language processing systems to identify relevant concepts in radiology reports. SymText is a medical language processing system developed by the Latter Day Saints (LDS) Hospital at the University of Utah (an additional description of this system is provided in the Management section of this chapter).156, 157 In one study, researchers compared the ability of SymText to identify pneumonia-related concepts in 298 X-ray reports with those of 2-word search programs, a layperson, and a resident physician. SymText performed better than the word search programs and the layperson but similar to the resident physician.156 A similar system was evaluated at the Columbia-Presbyterian Medical Center in New York.157 This study compared differences in the interpretation of 200 radiology reports by groups of 2 physicians, groups of laypersons, and the natural language processor. The differences between the interpretations of the natural language processor and the physicians were similar to the differences among physicians.157 This suggests that the natural language processor's ability to identify these concepts was similar to that of the physicians.
Researchers at LDS Hospital have integrated SymText into a real-time DSS designed to implement guidelines for community-acquired pneumonia.158 The radiology department uses speech recognition technology that decreases the time necessary to transcribe radiographic reports.158 As soon as the radiologist completes his or her dictation, SymText identifies patients who may have pneumonia based on their radiology reports and assesses the severity of their pneumonia.158 These findings are combined with other clinical and laboratory data to generate management recommendations to clinicians in compliance with clinical practice guidelines. An evaluation of this automated guideline showed that SymText was similar to physicians in identifying patients eligible for the guideline, but worse than the physicians in extracting information about the location and extent of the infiltrates (patient outcomes were not assessed).158
Our search identified 4 IT systems designed to improve radiographic diagnoses or incorporate data from radiology reports into diagnostic or management DSSs. Their utility in recognizing illnesses caused by bioterrorism is unknown, as none has been formally evaluated for this purpose.
The system from the University of Chicago has established utility for the diagnosis of community-acquired pneumonia. However, because the radiologic findings for most bioterrorism-related illness will be identical to pulmonary diseases of other etiologies and because the presence of a specific radiologic finding associated with bioterrorism-related illness is the exception rather than the rule, it is not clear that these systems could help clinicians, beyond alerting them to the presence of a pulmonary infiltrate, pleural effusion, or widened mediastinum. For radiologic systems to have a significant effect on clinicians' diagnostic decisions in regards to a bioterrorism event, they would have to raise the clinician's index of suspicion that a biothreat agent may be causing the radiologic findings. Incorporating information from these systems with other information from patients' medical records and knowledge bases about the clinical presentations of bioterrorism-related illnesses could be a useful innovation. Specifically, radiologic systems could serve as a component of an integrated management system that incorporates radiologic as well as other clinical information with clinical practice guidelines for the management and reporting of suspected bioterrorism-related illness.
Telemedicine is the use of telecommunications technology for medical diagnostic, monitoring, and therapeutic purposes when distance separates the users.160 We direct readers interested in this topic to a recent Evidence Report from AHRQ entitled "Telemedicine for the Medicare Population."160 Briefly, this Report describes 3 types of telemedicine systems: "(1) store-and-forward services that collect clinical data, store them, and then forward them to be interpreted later; (2) self-monitoring/testing services that enable clinicians to monitor physiologic measurements, test results, images, and sounds, usually collected in a patient's residence or care facility; and (3) clinician-interactive services that are real-time distance clinician-patient interactions."160 They found that telemedicine consults increased steadily throughout the 1990s with most programs designed to serve rural populations, veterans, and the elderly.160 Additionally, they report that teledermatology is the most-studied clinical specialty in store-and-forward telemedicine; its diagnostic accuracy and patient management decisions are comparable to those of in-person clinical encounters.160
| System name | Purpose | Description | Diagnostic accuracy | Contact information |
|---|---|---|---|---|
| Deployable Radiology System (DEPRAD)163 | To provide teleradiology consultation for military personnel at remote sites. | DEPRAD is a deployable radiology-imaging network used for teleradiology by the U.S. military. | No information available. | ISIS Center Department of Radiology Georgetown University Medical Center Washington, D.C. |
| Walter Reed Army Medical Center (WRAMC) Telemedicine Service161, 162 | To provide telemedicine consultation for military personnel at remote sites. | The WRAMC has implemented a number of telemedicine projects over the past decade. During Operation Desert Shield, U.S. Army medical units in Saudi Arabia connected a deployable computer tomography scanner to a maritime satellite, linking first responders in the field with specialists at WRAMC.162 In Somalia, U.S. military physicians sent digitalized photographs of dermatologic lesions for consultation with dermatologists at WRAMC. The objective of the WRAMC Telemedicine service is "the establishment of a global, comprehensive system of digital communication, enabling forward projection of medical expertise from any fixed medical facility, on demand, to physicians in any deployment area."162 In a retrospective case review evaluating the experience of the Army in conducting telemedicine consultation between February 1993 and March 1995, WRAMC received 171 telemedicine consultations.161 Of these, 114 consults were reviewed: 39% were for dermatology, 16% for surgical subspecialties and 15% for orthopedics. Telemedicine was felt to affect the diagnosis in 30%, the treatment in 32%, and the overall patient status in 70% of cases. | No information available. | Walter Reed Army Medical Center 6900 Georgia Ave., NW Washington, DC 20307-5001 Ph: 202-782-3501 |
| Mobile Operational Support System (MOSS)164 | To provide telemedicine consultation for military personnel at remote sites. | This project was the first telemedicine link for British Forces in the field. A preliminary report from 1988 described the use of the system for the management of the first 56 patients for whom MOSS enabled consultations with the following types of specialists: radiology (32 cases), dermatology (10 cases), and plastic surgery and burns (7 cases).164 | No information available. | Telemedicine Unit Royal Hospital Hasslar Gosport, Hants PO 12 2AA |
| MERMAID165-167 | To serve as a telemedicine system for the merchant marine. | MERMAID is a European Union-financed telemedicine project. It is designed for use in a maritime setting. It is unclear whether this system is fully operational. We found no evaluations of this system. | No information available. | BIOTRAST 111 Mitropoleos Str., GR-54622 Thessaloniki Greece |
Three of the 4 telemedicine systems were designed by the military to provide telemedicine consultation for military personnel at sites distant from military hospitals. Similarly, the other system, MERMAID, was designed for the European Union to provide telemedicine consultations to members of the merchant marine. None was designed or evaluated specifically for providing telemedicine consultations for disease resulting from bioterrorism. The sensitivity and specificity of the diagnoses provided by consultants using the system, the type of hardware required, and the system's security measures were not described in any report.
The Walter Reed Army Medical Center (WRAMC) Telemedicine Service system was evaluated in a retrospective case review of 171 telemedicine consultations.161 Of these, 114 consults were reviewed: 39 percent were for dermatology, 16 percent for surgical subspecialties and 15 percent for orthopedics. Telemedicine was felt to affect the diagnosis in 30 percent, the treatment in 32 percent, and the overall patient status in 70 percent of cases.161, 162
No telemedicine system has been evaluated specifically for bioterrorism. Telemedicine systems are most useful in areas with limited direct access to medical specialists. Since acts of bioterrorism against civilian populations may be less likely to occur in remote areas than in population centers, these systems may be of limited value against bioterrorism. However, since few practicing primary care or emergency physicians have ever seen the rashes associated with smallpox or other bioterrorism-related illness, the use of teledermatology technologies may increase the likelihood of a timely diagnosis by facilitating access to dermatologic experts. In the event of a widespread epidemic reaching geographically isolated areas, existing telemedicine infrastructures could be used by public health officials to relate public health information and alerts to clinicians.
In this section, we present a variety of other types of diagnostic systems. Unlike the general diagnostic DSSs discussed earlier, most of these systems are specifically for the diagnosis of infectious diseases (thereby limiting their use to only those circumstances in which the clinician suspects an infectious etiology).
| System name | Purpose | Accuracy |
|---|---|---|
| DSSs for the diagnosis of infectious diseases | ||
| Computer Program for Diagnosing and Teaching Geographic Medicine168 | To provide a differential diagnosis of infectious diseases matched to 22 clinical parameters for a patient; also to provide general information about infectious diseases, anti-infective agents, and vaccines. | The computer program correctly identified 75% (222 of 295) of the actual cases and 64% (128 of 200) of the hypothetical cases of patients with infectious diseases.168 The clinical diagnosis was included in the computer differential diagnosis list in 94.7% of cases. Among the cases included in this evaluation, several were for biothreat diseases such as: anthrax, brucellosis, cholera, Cryptosporidiosis, Hantavirus respiratory distress syndrome, Lassa fever, plague, Q fever, Rocky Mountain spotted fever, Shigellosis, and tularemia.168 |
| Fuzzy logic program to predict source of bacterial infection171 | To use age, blood type, gender, and race to predict the etiology of bacterial infections. | The system generates 4 classifications of infections: "staphylococci" (S. aureus and S. epidermidis), "streptococci" (S. pneumoniae, groups B and D streptococci), "E. coli," and "non-E. coli gram negative rods" (Klebsiella, Serratia, Proteus, Morganella, Prevotella, Pseudomonas, and Bacteroides species). The program was able to correctly classify 27 of 32 patients into 1 of these 4 groups based on demographic data alone.171 |
| Global Infectious Disease and Epidemiology Network (GIDEON)169 | To provide differential diagnoses for patients with diseases of infectious etiology. | The diagnostic accuracy of GIDEON was compared with that of medical house officers admitting 86 febrile adults to the Boston Medical Center. The house officers listed the correct diagnosis first in their admission note 87% of the time (for 75 of 86 patients) compared with 33% (28 of 86 patients) for GIDEON. All potential biothreat agents as specified by the CDC are included in the GIDEON knowledge base.169 To limit the differential diagnosis provided by the system, users enter the geographical area where the outbreak occurred. This is compared with the known areas of natural occurrence. For the purposes of detection of bioterrorism, adding this geographic information could falsely decrease the probability of disease if a biothreat agent was used in a region that had little naturally occurring disease from that organism.172 |
| Texas Infectious Disease Diagnostic DSS173 | To provide a weighted differential diagnosis based on manually entered patient information. | Records of 342 cases of brucellosis were obtained from the Texas Department of Health. Ninety-eight patients had been diagnosed more than 11 days after presentation and were considered missed diagnoses. In 86 of the 98 patients defined as missed diagnoses, the DSS listed brucellosis in the top 5 diagnoses on the differential diagnosis list, and in 69 of these 98 patients, brucellosis was the only disease suggested. The DSS missed the diagnosis in 12 of 98 patients. The mean number of days to suspect the correct diagnosis without the DSS was 17.9 days and with the DSS was 4.5 days (an improvement of 12.9 days; p = 0.0001).173 |
| DSSs for the diagnosis of active pulmonary tuberculosis | ||
| Clinical DSS for detection and respiratory isolation of tuberculosis patients174 | To automate the detection and respiratory isolation of patients with positive cultures and chest X-rays suspicious for tuberculosis (TB). | In a retrospective analysis, 171 adult culture positive TB inpatients were used to assess the accuracy of the system: without the DSS 51% (45 of 88) were appropriately isolated compared with 75% (62 of 83) with the DSS.174 The system would have erroneously recommended isolation of 27 of 171 patients (false positives). In a prospective analysis, clinicians adhering to the hospital's isolation policy correctly and promptly isolated 70% (30 of 43) of patients with TB. The DSS did not identify 21 of these patients (false negatives). However, the DSS identified 4 patients not identified by the clinicians.174 |
| Neural Network for Diagnosing Tuberculosis 175 | To use an artificial neural network that incorporates clinical and radiographic information to predict active pulmonary TB so that patients may be appropriately isolated at the time of presentation. | In a retrospective analysis of 119 patients, the neural network correctly identified 11 of 11 patients with active TB (sensitivity of 100% and specificity of 69%). Clinicians correctly diagnosed 7 of 11 patients with active TB (sensitivity of 64% and specificity of 79%).175 |
| DSSs for other diagnostic purposes | ||
| BloodLink176 | To decrease diagnostic test ordering by clinicians. | No information available. |
| DERMIS177, 178 | To provide a differential diagnosis of skin lesions. | In a 1992 evaluation of DERMIS using descriptions of lesions by a dermatologist, the system correctly diagnosed a lesion 76% of the time and included the correct diagnosis among its top 3 choices 95% of the time (out of a total of 5203 cases).177 In a subsequent evaluation, DERMIS gave the correct diagnosis 51% of the time when given a description of a skin lesion by general practitioners, and 80% of the time when given a description by dermatologists (out of 100 cases). It listed the correct diagnosis in the top 3 of its differential list 70% of the time when given a description by general practitioners, and 93% of the time for dermatologists.178 |
| PNEUMON-IA179 | To diagnose community-acquired pneumonia from clinical, radiologic and laboratory data. | Reports of 76 cases of adult community-acquired pneumonia were analyzed by PNEUMON-IA and by 5 clinician experts. Ten of these 76 cases had confirmed diagnoses from microbiology data. The DSS correctly identified the diagnosis in 4 of these 10 cases, compared with between 3 and 6 cases for the clinician experts.179 |
Note: for additional information on these systems, see Evidence Table 2.
Of all the diagnostic DSSs, we could only verify that GIDEON and The Computer Program for Diagnosing and Teaching Geographic Medicine specifically include most of the worrisome bioterrorism-related organisms in their knowledge bases.168, 169 Both of these systems provide differential diagnoses of infectious diseases based on clinical parameters regarding a patient that are entered into the program. The Computer Program for Diagnosing and Teaching Geographic Medicine also provides general information about infectious diseases, anti-infective agents, and vaccines.
The evaluation of GIDEON compared the diagnostic accuracy of the DSS to that of medical house officers admitting 86 febrile adults to the Boston Medical Center. The house officers listed the correct diagnosis first in their admission note 87 percent (75/86) of the time compared with 33 percent (28/86) for GIDEON.169 In a study to evaluate the diagnostic accuracy of The Computer Program for Diagnosing and Teaching Geographic Medicine, 6 infectious disease specialists (blinded to the patients' actual diagnoses) were asked to record all positive and negative clinical data for 295 consecutive patients with established diagnoses and 200 hypothetical cases. The computer program correctly identified 75 percent (222 of 295) of actual cases and 64 percent (128 of 200) of hypothetical cases. The clinical diagnosis was included in the computer differential diagnosis list in 94.7 percent of cases. Among the cases included in this evaluation, several were for the causative agents of: anthrax, brucellosis, cholera, cryptosporidiosis, Hantavirus respiratory distress syndrome, Lassa fever, plague, Q fever, Rocky Mountain spotted fever, shigellosis, and tularemia. However, this system was only tested on cases for which the diagnosis was known; therefore, there is no information on how it would perform for cases with unknown outcomes.168
If a system produces a single diagnosis for a given case as its output, the sensitivity and specificity of the system can be readily determined if the case's actual diagnosis is known. Frequently, systems are designed to provide a list of many possible diagnoses, often ranked according to their probability of being the actual diagnosis. Under these circumstances the clinician will have to determine the lower threshold of probability for which they will make a diagnostic or therapeutic decision (i.e., if a system generates a list of possible diagnoses for a case and suggests that smallpox is on the differential but highly unlikely, he or she may not choose to send a viral culture or notify the local public health official). When diagnostic systems provide a list of possible diagnoses, it may be more appropriate to calculate receiver operator characteristic (ROC) curves to evaluate the performance of the system over a range of probability thresholds. We direct interested readers to an article by Fraser and colleagues measuring the performance of systems that generate differential diagnoses using ROC curves and other methods.170 Only the neural network for the diagnosis of active pulmonary tuberculosis was evaluated with ROC curves.
For those programs that require a user to input case-specific information, we again found that the differential diagnoses provided by the systems were highly dependent upon the information input about the cases. This was particularly true for DERMIS, where generalists' inputs were less likely than those of specialists to result in a correct diagnosis. This is an unfortunate finding because patients with bioterrorism-related skin lesions are more likely to present to general clinicians than dermatologic specialists and because the early recognition of skin lesions associated with smallpox, Glanders, bubonic plague, and tularemia could significantly reduce bioterrorism-related morbidity and mortality.
None of the reports of these general DSSs discussed barriers to the use of the systems, whether the system uses a standard vocabulary, information regarding the ability to update the probability of biothreat-related illness as the epidemic progresses, the type of hardware required, or the system's security measures.
The role of this heterogeneous group of diagnostic DSSs in a bioterrorism response is to enhance the likelihood that clinicians consider the possibility of bioterrorism-related illness. Therefore, these systems could contribute to the detection of a previously unrecognized release of biothreat agents. However, the reports of general diagnostic DSSs have several important limitations that prevent conclusions regarding their ability to serve this role.
In this section, we discuss the systems designed to assist clinicians and public health officials in making management and prevention decisions. Most of the systems included in this section are designed to make recommendations to clinicians, not to public health officials. Typically, they abstract data from an electronic medical record, apply a set of rules, and generate patient-specific management and prevention recommendations. In general, these systems are limited to institutions with electronic medical records and robust medical informatics infrastructures.
In this section, we describe 18 systems designed to make management or prevention recommendations; 10 of these have been described in at least 1 clinical evaluation report (Tables 3 and 13; Evidence Table 3; Appendix H). We found no systems specifically designed to provide recommendations to clinicians or public health officials for management of a bioterrorist attack. None of the reports of the management or prevention programs stated that bioterrorism-related diseases were included in their clinical practice guidelines, prediction rules, or knowledge bases. Most of the management systems described in this section provided recommendations at the point of care -- typically, when the clinician entered the electronic medical record of the patient in question. These systems are therefore relatively timely. The reasoning used by these systems varies, including both probabilistic and rules-based methods. Few reports specified whether the system uses a standard vocabulary, whether it would be possible to update management recommendations as the epidemic progresses, the type of hardware required, or the system's security measures (although most are associated with hospital-based electronic medical records that require a user login).
The management DSS that has been the topic of the most numerous descriptive articles and clinical evaluations is the Health Evaluation through Logical Processing (HELP) system at LDS Hospital in Salt Lake City.156, 158, 180-215 The HELP System is a complete computer-based hospital information system designed to support applications including order entry/charge capture, pharmacy, radiology, nursing documentation, and intensive care unit (ICU) monitoring as well as to maintain robust decision support functions. Decision support has been used to provide alerts and reminders, to make patient diagnosis and management recommendations, and to implement clinical protocols. Specifically, it alerts clinicians about infections in normally sterile body sites (e.g., from urine and blood cultures), makes antibiotic recommendations, suggests appropriate timing and duration of prophylactic antibiotics, and identifies adverse drug reactions (as such, the system has considerable similarities to some of the communication systems discussed later in this Report). HELP also generates alerts for infections that are required by law to be reported to state or national public health officials. Earlier in this Report, we briefly described the elements of the HELP program that search radiology reports for pneumonia-related concepts and its decision support module for the diagnosis of community-acquired pneumonia.156 We also described SymText, a medical language processor developed at the LDS Hospital to analyze free text reports. In this section, we describe 3 additional HELP functions that could serve the information needs of clinicians or public health officials in the event of a bioterrorist attack: its antibiotic management program, its Data Mining Surveillance System (DMSS) for the detection of hospital-acquired infections, and its community-acquired pneumonia diagnosis and management program.
The HELP system has multiple antibiotic protocols to provide comprehensive management recommendations for all antibiotic agents used in the LDS hospital system, including those for prophylactic, empiric, and therapeutic purposes. The HELP investigators convert local physician-derived antibiotic prescribing guidelines regarding drug choice, dosage, and timing into rules, algorithms, and predictive models. These guidelines are combined with 4 knowledge bases to provide timely, patient-specific management recommendations. The knowledge bases include the following information: (1) the probabilities of infectious diseases based on signs, symptoms, risk factors, and diagnostic test results; (2) the expected courses of infectious diseases in terms of morbidity and mortality if left untreated; (3) the expected courses of infectious diseases if treated optimally; and (4) the fraction of patients with each infectious disease expected to respond to each intervention under consideration.215 During a 7-year study period of its antibiotic management system, HELP improved the timeliness of administration of prophylactic antibiotics.203 The proportion of patients receiving their first dose within 2 hours before surgical incision increased from 40 percent in 1985 to 99 percent in 1994. During the same period, the rate of antibiotic-associated adverse events decreased from 27 percent to 19 percent, the adjusted antibiotic cost per treated patient decreased from $123 to $52, and antibiotic resistance patterns remained stable.203 Another study reported reductions in costs of antibiotics, total hospital costs, and length of stay for those patients who always received the antibiotics recommended by HELP compared with the pre-intervention cohort and those who did not always receive the HELP-recommended regimen.196 Interpretation of this analysis is complicated by the fact that the post-intervention group was divided into those patients who always received recommended antibiotics and those who did not. Patients for whom clinicians chose to override the recommendations of HELP may have been more medically complicated and would, therefore, be expected to have longer, more costly hospitalizations.
For the purpose of surveillance, HELP uses the DMSS, a novel approach for identification of unusual patterns in data.186, 191 The program reviews data from a variety of sources including the microbiology laboratory, nurses' charts, chemistry laboratory, surgical records, and pharmacy to identify association rules over time. For example, a decision rule could be developed to describe the conditional probability of multi-drug resistant Pseudomonas given all Pseudomonas isolates in the ICU. This decision rule can then be used to identify unexpected patterns by statistically comparing the rates of multi-drug resistant Pseudomonas in each time interval with those that preceded it. A daily report detailing any suspicious outbreaks is sent to the hospital epidemiologist.186, 191
HELP has a rapid decision support module designed to identify and manage patients with community-acquired pneumonia on presentation to the emergency department.182 The HELP system notifies the DSS whenever a new patient registers in the emergency department. The DSS then queries HELP's databases every 5 to 10 minutes for any new clinical information on that patient. It can retrieve up to 42 data elements from which it calculates the probability of community-acquired pneumonia and a severity index score and presents these to clinicians in the emergency department caring for that patient. In an evaluation of this system, the authors compared the diagnostic accuracy of a Bayesian network and an artificial neural network using data from the HELP databases.182 They set the diagnostic threshold to achieve a sensitivity of 95 percent and calculated the specificity (SP), positive predictive value (PPV), and negative predictive value (NPV) of the Bayesian network and the artificial neural network. For the Bayesian network: SP = 92.3 percent, PPV=15.1 percent, NPV=99.9 percent. For the artificial neural network: SP = 94.0 percent, PPV=18.6 percent, NPV=99.9 percent. These values suggest that the neural network performed somewhat better than the Bayesian network; however, it remains unclear if this difference implies a significant difference in clinical outcomes for pneumonia patients.
| System name | Purpose | Evaluation data |
|---|---|---|
| Health Evaluation through Logical Processing (HELP)156, 158, 180-215 | Multiple purposes; please see text above. | Multiple evaluations; please see text above. |
| Antibiotic recommendation programs | ||
| ABIX217 | To advise non-specialist physicians on suggested management plans for infectious diseases. | A survey of 50 doctors from a pilot evaluation of ABIX reported that the system is easy to understand and use. In addition, 85% indicated that the information included and the way it was classified was satisfactory.217 |
| Antibiotic Assistant™218 | To provide physicians with patient- and disease-specific decision support on antibiotic treatments. Uses an inference engine and syndrome-specific evidence-based knowledge bases. | None available. |
| Pneumonia Therapy Advisor (PTA)219, 220 | To advise intensive-care unit (ICU) physicians on the diagnosis and initial treatment of ventilator-associated pneumonia. Cost, side effects, and expected efficacy are taken into consideration for therapy recommendations. A total of 32 different therapy regimes are included. | A comparison of PTA's treatment recommendations for 12 ICU patients with those of an infectious disease specialist demonstrated that 100% (12 out of 12) of the model's choices were considered "acceptable" or "second-best choice." However, in 66% (8 out of 12) of the cases, PTA recommended a therapy that covered more pathogens than the expert-recommended therapy, due to the model's lack of knowledge regarding the broadness of the antimicrobial spectrum.220 Preliminary evaluations suggest that this issue may be partially alleviated with the addition of a function that only includes pathogens with a posterior marginal probability of 31% or greater, as well as a utility function that discourages the prescription of broad-spectrum antibiotics.219 |
| QID221 | To calculate a differential diagnosis from an infectious disease knowledge base that runs on Iliad's inference engine. Using local antibiograms, QID generates a list of antibiotics and its toxicity and cost for each of the possible diseases/organisms. | Forty physicians evaluated randomly selected cases of urinary tract infections, bacteremia, pneumonia, and meningitis selected from the University of Utah medical center with and without QID. (The cases were limited to these conditions since the knowledge base used for QID only includes hospital-acquired infections.) After reviewing the case, physicians were asked to note their suggested antibiotic regimen. They were then given QID's antibiotic recommendation and could make any changes to their selected antibiotic regimen. When compared to the antibiotic susceptibility pattern isolated for each case, the physicians' choices of an antibiotic with appropriate coverage for the most likely organism increased from 66% to 75% with the use of QID (p <0.001).221 |
| Rabin Medical Center Antibiotic DSS222 | To assist in the selection of empiric antibiotics in suspected moderate to severe bacterial infections. Combines site-specific information regarding pathogen prevalence and susceptibility to antibiotics with prediction models derived from large pools of data and validated in other sites. | A study comparing the recommendations of the system with those of a physician for 219 patients with positive cultures or serological tests demonstrated that the system recommended treatment to which the pathogen was shown to be susceptible in 77% of patients, compared to 58% for physicians. The DSS made inappropriate drug recommendations for 23% of the patients (compared with 42% for the physicians) and recommendations for an antibiotic that was either unneeded or too broad-spectrum in 11% of the patients (compared with 15% for the physicians). Use of the system would have reduced the rate of inappropriate treatments in patients with a known pathogen by 19%.222 |
| Intensive care management systems | ||
| Eindhoven Automated Knowledge Acquisition Tool223 | To provide decision support to health care workers in clinical and emergency care environments. Using the knowledge acquisition tool, physicians enter guidelines that are exported to the reminder system used in daily practice after being checked for accuracy. | ICU physicians used the knowledge acquisition tool to enter 58 guidelines into the reminder system's knowledge base. These guidelines were tested on a database of 803 previously admitted patients. During this test, 27 of the 58 guidelines generated at least 1 reminder; a total of 406 reminders were generated. Of the 406 reminders, 356 (88%) were issued correctly and 50 (12%) were considered false alarms.223 The false alarms were attributed to lack of specificity in the underlying guideline. This realization led to improvements in the guidelines. |
| ICONS224, 225 | To provide rapid antibiotic recommendations for ICU patients with hospital-acquired infections using case-based reasoning. | None available. |
| Intelligent Decision Aid System (IDEAS) for ICU and IDEAS for NICU226 | To make treatment recommendations for new patients admitted to the ICU or neonatal ICU (NICU) based on similarity to former ICU/NICU patients. | After a 3-week study involving a prototype version of IDEAS for ICUs and 27 patients, 5 evaluation forms submitted by physicians indicated that the system would be beneficial to clinicians, while 22 said that no benefit was currently foreseeable. Comments regarding improvements were incorporated into a newer version of the system. In a separate preliminary study, IDEAS for NICUs was rated highly in terms of usability by 5 neonatologists but not considered very clinically useful in its current form.226 |
| Pneumonia management system | ||
| Severity Scores for Community-acquired Pneumonia216 | To classify patients using a published prediction rule by calculating a severity score based on laboratory, radiographic, and historical data (from the electronic medical record) to make severity-based management recommendations for patients with community-acquired pneumonia. | In comparison to a reference standard obtained manually by an independent expert for 79 community-acquired pneumonia cases, the system achieved an accuracy of 93%, a sensitivity of 92%, and a specificity of 93% for processing discharge summaries. For chest X-rays, it demonstrated an accuracy of 96%, a sensitivity of 87%, and a specificity of 98%. It was 80% accurate in assigning the exact risk class, with the remaining 20% differing by only 1 class, and 85% accurate with vital sign values.216 |
| Other management DSSs | ||
| ABDX227 | To aid the Naval Independent Duty Corpsman in the diagnosis and treatment of shipboard patients with acute abdominal pain. | None available. |
| Columbia-Presbyterian Clinical Event Monitor157 | To monitor a patient database in order to generate alerts, interpretations, and screening messages for clinicians throughout the Columbia-Presbyterian Medical Center. | Formal studies are in progress. No information is currently available. |
| Déjà Vu228 | To automatically recognize and monitor time-dependent medical scenarios using temporal reasoning. | None available. |
| Emergency Medical Alert Network (EMAN)229, 230 | To provide early detection of adverse health events and dissemination of health information regarding disease treatment and personal protection. | None available. |
| MEDTRAK231 | To monitor patients as they flow through a medical facility from the emergency department through the radiology and surgical departments and onto the wards. | None available. |
| NexProfiler232 | To provide easily accessible disease- and patient-specific treatment information to patients and physicians. | None available. |
| TEsting COMpetency (TECOM)233 | To teach medical decision making to medical students using computer-based cases that include real patient data and optimal treatment plans validated by a hospital antibiotic center. | None available. |
| Utrecht Emergency Hospital Patient Barcode Registration (PBR) System234, 235 | To track medical, nursing and logistic information for patients admitted to the Utrecht Emergency Hospital during mass casualty incidents. | For 4 experimental exercises performed between 1993 and 1994, each of which involved 30 patients, both the amount and accuracy of data recorded increased with the PBR system (p<0.05 and p<0.01, respectively), as compared to medical charts completed by an experienced administrative assistant. Specifically, inaccuracy decreased by 25% as compared to handwritten medical charts.235 During 12 mass casualty admissions at the Utrecht Emergency Hospital, personal data for the patients were entered into the system within 30 minutes of admission for 58.1% (161 out of 277) of the cases and 60 minutes for 80.5% (223 out of 277) of the cases.234 |
Note: for additional details, see Evidence Table 3.
In general, the antibiotic recommendation programs provide differential diagnoses from infectious disease databases and make patient-specific antibiotic recommendations often taking into consideration cost, pathogen prevalence, and susceptibility patterns. The evaluation evidence for 4 of these systems is mixed: half of them recommended antibiotics with narrower spectra than the clinicians would have otherwise used. This may be the intention of the developers of these systems since it could reduce the general problem of escalating antibiotic resistance. However, if clinicians make antibiotic selection decisions while unaware of the true bioterrorism-related diagnosis and select narrow-spectrum antibiotics, they may not adequately treat the pathogens.
The 2 evaluations of the intensive care management programs suggest inadequate acceptance by users and high false positive rates, limiting their potential utility at this time. However, the results of the system from Queens College that calculates Severity Scores for Community-acquired Pneumonia demonstrated greater diagnostic accuracy.216 The system generates severity-based pneumonia management recommendations from a clinical prediction rule. The evaluation of this system compared its diagnostic accuracy with an independent expert for 79 patients with community-acquired pneumonia cases. Depending on the information it used to calculate the severity score, the system achieved sensitivities of 87 to 92 percent and specificities of 93 to 98 percent. It was 80 percent accurate in assigning the exact risk class, with the remaining 20 percent differing by only 1 class.216
The systems included in this section are designed to make recommendations to clinicians by abstracting clinical information from electronic medical records to make patient-specific recommendations. None of the 18 systems described in this section has been specifically designed or evaluated for utility in providing management or prevention recommendations during a bioterrorism event. We have no information as to whether the knowledge bases and inference engines of these systems include comprehensive information about bioterrorism-related illness. Moreover, none of the evaluations describes effects on patient outcomes other than length of stay in the hospital.
The systems that are not linked to electronic medical records share many of the limitations of the general diagnostic systems -- including, that clinicians may not use the system to seek advice for patients presenting with common viral syndromes (i.e., the bioterrorism-related syndromes). Expert systems that continuously search electronic medical records (including data from the laboratory, radiology reports, and physician notes) for new evidence of an infection and apply clinical practice guidelines to those data have potential utility in bioterrorism management. However, this requires relatively robust hospital IT infrastructures and the incorporation of clinical practice guidelines for biothreat-related illnesses.
Surveillance is the collection, consolidation, and evaluation of morbidity, mortality and other relevant data...and its regular dissemination to all who need to know.
-- A. Langmuir, 1963236
Surveillance is the routine collection and analysis of relevant data, and their distribution to clinicians, public health officials, and others in the community who use them to take action to prevent further morbidity or mortality.237 Surveillance data serve the following purposes: to discover the natural incidence of the events under surveillance, to detect abnormal situations that require epidemiologic control measures, to direct preventive actions, to guide resource allocation, and to assess interventions.237 The most important aspect of a surveillance system is not the nature of the events under observation; rather, it is the ability to detect an outbreak at a stage when intervention may affect the expected course of events. Therefore, the ideal surveillance system for the detection of a covert bioterrorist attack would collect data that are both sensitive and specific for biothreat agents and provide reports of these data to public health officials as soon after the dispersal of the agent as possible.
In the event of a covert aerosol release of a biothreat agent, exposed people will initially present with minor symptoms. Exposed individuals may stay home from work or school, buy over-the-counter medications, or perhaps telephone a triage nurse. During subsequent days their symptoms may worsen, prompting them to seek care from their physicians. Physician visits may in turn result in the use of prescription drugs. If patients become acutely ill, they may call 911, or present directly to an emergency department. A visit to the emergency department may result in laboratory tests or admission to the hospital. The decisions patients make and the data their behaviors generate form the basis of surveillance systems for the early detection of a covert bioterrorist attack.
). Data regarding school and work absenteeism, records of phone calls to
triage nurses, and over-the-counter pharmacy sales could provide additional
early warnings. Surveillance systems that rely on sentinel clinicians reporting
suspicious patients to their local public health officials and pharmacies
reporting prescription sales will detect an outbreak somewhat later. Less timely
surveillance systems rely on data from emergency departments, 911 calls,
laboratories, and hospitals. Because the window period during which antibiotics
and antiviral agents are effective against several biothreat agents ends before
the patient becomes sick enough to be admitted to the hospital, the outbreak has
to be detected early so that effective therapies can be started and excess
morbidity and mortality prevented.
In addition to timeliness, these data sources differ in 2 additional ways. First, they differ with respect to their sensitivity and specificity for the detection of biothreat agents. Whereas a large increase in the sales of over-the-counter cold-and-flu preparations may be sensitive for an aerosol biothreat-agent release, it is not likely to be specific. Conversely, viral cultures may be highly specific but not sensitive, particularly if the organism is difficult to collect properly or to grow in culture. Second, these data sources differ with respect to the ease with which they can be collected. For example, some data (such as ICD9 codes) are already routinely collected in electronic format and may be amenable for use by a surveillance system. In contrast, clinicians do not typically report the chief complaint, signs, and symptoms of patients presenting to their clinics; therefore, the routine collection of syndromal surveillance data could significantly add to clinicians' workloads. Additionally, if the data are collected electronically in a systematic manner, they are less likely to be incomplete or to contain errors.
The CDC, the primary agency responsible for the collection of disease surveillance data in the U.S., is working to create an integrated surveillance system -- a "system of systems" that combines many of the existing laboratory and clinical surveillance systems. Through its collaborations with its public health partners (e.g., Association of Public Health Laboratories (APHL), Association of State and Territorial Health Officials, Council of State and Territorial Epidemiologists (CSTE), and National Association of County and City Health Officials), the CDC maintains over 100 separate surveillance and health information systems (not all for infectious diseases).238 Most of these systems are specific for a particular category of diseases (e.g., sexually transmitted diseases) and are not linked even to related diseases. One example of the integrative efforts of the CDC is the Laboratory Response Network, which now consists of 120 state and local public health laboratories linked to advanced capacity laboratories including clinical, military, veterinary, agricultural, water, and food-testing laboratories.239 The Laboratory Response Network is described in greater detail in the Laboratory section of this chapter.
To improve data collection and sharing for surveillance purposes at the state and local levels, and to enhance the ability of the public health system to respond to public health threats, the CDC is implementing the National Electronic Disease Surveillance System (NEDSS).238 This system has not yet been fully deployed or clinically evaluated. When it is completed, NEDSS will include data and information system standards, an Internet-based communications infrastructure, and policy-level agreements on data access, sharing, and security. The NEDSS architecture is designed to enhance the ability to electronically link individual surveillance activities, to improve the integration of information reporting into the provider's workflow through automatic transfer of data from clinical information systems, and to facilitate accuracy and timeliness of surveillance reports.238 It is intended to integrate and replace several current CDC surveillance systems, including the National Electronic Telecommunications System for Surveillance (NETSS), the HIV/AIDS reporting system, the vaccine preventable disease system, and systems for tuberculosis and infectious diseases.238
Global infectious disease surveillance is carried out through a loose framework of formal and informal arrangements that the World Health Organization (WHO) characterizes as a "network of networks."240 The formal partners in this effort include: WHO regional and country offices; national public health authorities; United Nations Children's Fund (UNICEF) and United Nations High Commissioner for Refugees (UNHCR) country offices; WHO collaborating centers/laboratories; epidemiology training networks; and military laboratory networks.241 The informal contributions come from non-governmental organizations, the media, Internet discussion sites, and the Global Public Health Intelligence Network (GPHIN).241 Surveillance systems in most nations are limited by a lack of adequately staffed and equipped laboratories; surveillance data already collected in electronic form that can be readily imported into analysis algorithms; trained personnel to collect and analyze additional surveillance data; and information infrastructures to support the communication of surveillance data from remote collection sites to a central analysis site.241 These limitations are most significant in developing countries.241 Weaknesses in the detection and response capabilities in poorer countries affect the ability of international efforts to detect and control infectious disease outbreaks.241
In 1947, an American businessman returned to New York from a sightseeing trip to Mexico during which he had become infected with smallpox.242 Nine days after arrival, he died, having infected at least 12 others, 2 of whom also died.242 As a result of concerns that the disease would spread further, more than 6 million people in New York were vaccinated within a month.242 Given the current state of international travel, importation of food products, and trade in pharmaceuticals and blood products, it is in the best interest of the U.S. to support global surveillance efforts. Infectious diseases from natural sources (such as the as yet unidentified animal reservoirs of Ebola) or from acts of bioterrorism can readily travel from the remotest parts of the world to the U.S.
Since most surveillance systems are identified by the type of data that they collect, we have organized this section accordingly. We found 90 surveillance systems: 7 collecting syndromal data; 6 collecting clinician reports; 11 collecting influenza-related data; 23 collecting laboratory or antimicrobial resistance data; 16 collecting hospital-based infections data; 10 collecting foodborne illness data; 6 collecting zoonotic and animal illness data; and 11 collecting other kinds of surveillance data. We conclude this Surveillance section with a brief description of the standard methods for analysis and presentation of surveillance data.
The earliest signs and symptoms caused by most biothreat agents are flu-like illness, acute respiratory distress, gastrointestinal symptoms, febrile hemorrhagic syndromes, and febrile illnesses with either dermatologic or neurologic findings. Therefore, patients with these syndromes are the targets of bioterrorism-related syndromal surveillance programs. There is no widely accepted definition for any of these syndromes. As a result, syndromal surveillance systems are widely heterogeneous with respect to the syndromes under surveillance and the definitions of each syndrome.
With written permission from: RSVP ® Project, Sandia National LaboratoryThis screen enables clinicians to report additional clinical data about the patient (not required). When the clinicians have finished they touch the yellow "Done" button, sending the data directly to a server in the State Epidemiologist's office.
to 10; Appendix H). We
have no specific information about the usefulness of these systems in
terms of detecting known infectious disease outbreaks (e.g., no
information about whether the syndromal surveillance system detected
last season's influenza outbreak). Additionally, we have no specific
information on any of these systems' flexibility, acceptability,
representativeness, or the direct costs of implementation.
| System name | Purpose | Geographic location | Syndromes under surveillance | Data used in surveillance system | Method of data collection and analysis | Sponsoring agency |
|---|---|---|---|---|---|---|
| Border Infectious Disease Surveillance Project (BIDS)249, 250 | To detect infectious disease along the U.S.-Mexican border. | U.S.-Mexican border. | Hepatitis and febrile-rash illnesses in border populations. | No information available. | Data collection is conducted at 4 sites on both sides of the border. No other specific information available. | CDC; Mexican Secretariat of Health; Pan American Health Organization; multiple U.S. and Mexican state health departments. |
| Early Warning Outbreak Recognition System (EWORS)245, 246 | For early outbreak detection. | Indonesia. | Fever; watery diarrhea; bloody diarrhea; dehydration; difficulty breathing; seizures; jaundice; vomiting; cough; paralysis; unconsciousness; bleeding; intradermal hemorrhage. | Clinician reports, including data such as patients' basic demographic information and symptoms. | Data are entered into a simple computer program designed for use by personnel without significant training at a remote location. Each day, data are downloaded from remote sites around Indonesia to the Indonesian Ministry of Health. | Naval Medical Research Unit-2; DOD-GEIS; Indonesian Ministry of Health. |
| Electronic Surveillance System for the Early Notification of Community-Based Epidemics (ESSENCE)248 | To provide daily analysis of trends in diagnosis codes grouped into "syndromal clusters." | 104 DOD primary care and emergency clinics within 50 miles of Washington, D.C. and 121 Army, 110 Navy, 80 Air Force, and 2 Coast Guard installations worldwide. | Respiratory illness; gastrointestinal illness; fever; neurologic (suggestive of botulism and meningitis); dermatologic-infectious; dermatologic-hemorrhagic; coma/sudden death. | Ambulatory diagnosis codes grouped into "syndromal clusters." | The data are downloaded daily onto a server and automatically analyzed with both traditional epidemiologic methods and time-space analyses. Over 2,700 syndrome- and location-specific graphs are prepared each day and automatically analyzed for patterns that suggest a need for further investigation. The graphs are available daily to approved DOD public health professionals on a secure Web site. | DOD-GEIS |
Health Buddy® and the Biothreat Active
Surveillance Integrated Information and Communi-cation
System (BASIICS)244 (See Figure
7 ) | To display syndromal surveillance questions to users in a variety of clinical areas. The device can also present an alert from public health officials to clinical staff. | Piloted in the Stanford University Medical Center Emergency Department. | The system is flexible in that the screens presented to nursing staff can be changed remotely in order to collect different surveillance questions. | Answers from triage nurses to questions of the users' choosing. | Nurses use 3 out of the 4 buttons on the device to answer whether the patient has none, 1, or more than 1 of the syndromes of interest. When the survey is completed, the data are automatically sent to a Data Center via a telephone line for analysis. Because of its telephone connection, public health officers can remotely change questions asked by the nurses and send them alerts. | HealthHero 650-559-1023 http://www.healthhero.com/ |
Lightweight Epidemiology Advanced Detection and
Emergency Response System (LEADERS), also called
Enhanced Surveillance Project (ESP), formerly called the
Enhanced Consequence Management Planning and Support
System (ENCOM-PASS) (See Figure 4 -5 )85 | To integrate data collection, analysis, and management system for syndromal and other surveillance data for the early detection of a covert bioterrorism event. Typically used for event-based surveillance. Can also track casualties, bed occupancy, and emergency department diversion status. | 1999 World Trade Organization Summit; 2000 Democratic and Republican National Conventions; 2001 Presidential Inauguration; 2001 Super Bowl; and 2001 World Series. Employed by U.S. Air Force in Cameroon, Germany, and El Salvador. | The forms presented on the Web-based collection tool are flexible and can ask any questions of interest. | Clinicians at participating hospitals fill out a brief form during the first encounter with patients. | This system can integrate multiple sources of
surveillance data (e.g., detection data -- see
Table 6 for
description of RAPID), analyze these data, and present
analyses in multiple formats to decision makers. LEADERS
includes a Web-based syndromal surveillance collection
tool (Figures 4
and 5) that
automatically reports to an Oracle database. The Patient
Encounter Module is a handheld software program that
interfaces with the system to facilitate data collection
in the field. Medview is a medical surveillance
visualization tool that assists medical planners,
epidemiologists, and responders to identify the
geographic origin of covert biological warfare agent
releases. The progression of syndromal data over time
can be viewed in a playback/play forward format. | CDC; Oracle; Idaho Technology Inc.; DARPA |
Rapid Syndrome Validation Project (RSVP®)247 (See Figure 8 -10) | For syndromal surveillance by clinicians in a variety of clinical areas. | Two emergency departments in New Mexico. | Flu-like illness; fever with skin findings; fever with altered mental status; acute bloody diarrhea; acute Hepatitis; acute respiratory distress syndrome. | Clinician reports of patients with 1 of the syndromes of interest. | RSVP® is set up on touch screens in the 2 participating emergency departments so that clinicians can enter clinical and demographic data on patients with 1 of 6 syndromes during or after the clinical evaluation. A screensaver on the State Epidemiologist's computer contains a continuously updating graph of the 6 syndromes. Additionally, when a patient with a predetermined set of highly worrisome clinical findings is entered into the system, the State Epidemiologist is notified automatically by e-mail and pager and can post alerts notifying emergency room staff of other surveillance data and suspected outbreaks. | Sandia and Los Alamos National Laboratories; University of New Mexico; New Mexico Office of Epidemiology. http://www.cmc.sandia.gov/~sacaske/bio/rsvp/1_AMIA-RSVP2001a. pdf |
Syndromal Surveillance Tally Sheet243 (See Figure
6 ) | For the surveillance of 6 syndromes by triage nurses in emergency departments. | Currently in use in the emergency departments of Santa Clara County, California. (Similar systems may be in use in other U.S. counties; however, we did not survey other public health departments.) | Flu-like symptoms; fever with mental status changes; fever with skin rash; diarrhea with dehydration; visual or swallowing difficulties, drooping eyelids, slurred speech or dry mouth; acute respiratory distress; exposure to 'suspicious' item/substance. | Triage nurses' counts of numbers of patients presenting with the syndromes of interest. | For each patient they evaluate, triage nurses are asked to record on a paper tally sheet whether the patient has none, 1, or more than 1 of the syndromes of interest. At the end of their shift, they are asked to add up the total number of patients in each syndromal category and to fax the sheet to the County Health Department. The faxes are collected several times a day by staff who manually enter the data into the surveillance database. Graphical displays of the prior days' counts are manually generated. | Santa Clara County, California, Department of Public Health http://www.sccphd.org |
With written permission from: LEADERS ConsortiumUsers select from among the LEADERS tools in the upper left section of this screen. For example, The Critical Care Tracking System allows emergency departments and 911 dispatchers to share information in a secure environment, allowing geographic displays of those emergency departments that are open to accept transport of patients. Hospital status may be tracked at the department level or by bed category.
.
With written permission from: LEADERS ConsortiumThis is an example of a syndromal surveillance form. The user can customize the list of syndromes or use one of the preprogrammed lists (i.e., U.S. Air Force 70 Reportable Events, CDC 52 Nationally Notifiable Infectious Diseases, GEIS-ESSENCE list of syndrome categories, or the Public Health Office Medical Surveillance application list of syndromes).
) can send an alert to a public health
official when user-defined syndromal thresholds are exceeded. It
also allows public health officials and incident commanders to
remotely map surveillance data and visually monitor and track
test-results or cases of interest.
and 5). LEADERS has been used for event-based surveillance for the
1999 World Trade Summit, the 2000 Democratic and Republican National
Conventions, and the 2001 Presidential Inauguration. (LEADERS can also
be used for ongoing surveillance.)
).243 Triage nurses in urgent care centers and emergency departments
enter data on each patient whom they evaluate and fax the sheet to the
local health department at the end of each shift (typically 3 times a
day). Health Buddy® is a device that can be used to display
syndromal surveillance questions to users in a variety of clinical areas
(Tables 14 and
15; Figure 7).244 It was recently piloted in the Stanford University Medical Center
Emergency Department. It is likely that similar tools have been
developed and implemented by county health departments throughout the
U.S.; however, a comprehensive survey of the syndromal surveillance
methods currently in use was outside the scope of our project.
The Early Warning Outbreak Recognition System (EWORS),245, 246 developed collaboratively by the Naval Medical Research Unit-2, Department of Defense Global Emerging Infections System (DOD-GEIS), and the Indonesian Ministry of Health, is a global syndromal surveillance system (Tables 14 and 15). EWORS uses a simple computer program designed to enable untrained personnel to collect basic demographic and symptom data that are downloaded daily from remote sites to the Indonesian Ministry of Health.
.
to 10). Data are entered on only
those patients whom clinicians believe fit into 1 or more of 6 syndromal categories.247 RSVP® facilitates (but does not require) the
collection of more detailed information about a particular patient than
the other systems (although several of the other systems have flexible
interfaces that allow customization and the design of more detailed
collection forms). In addition, RSVP® enables public health
officials to send alerts and public health information to clinicians.
The Electronic Surveillance System for the Early Notification of Community-Based Epidemics (ESSENCE) was developed by the DOD-GEIS (Tables 14 and 15).248 This system downloads ambulatory diagnosis codes grouped into "syndromal clusters." Initially, ESSENCE collected data from the primary care and emergency clinics serving the DOD health care beneficiaries living in and around Washington, DC. The data were downloaded daily onto a server and automatically analyzed with both traditional epidemiologic methods and time-space analyses. Following the events of September 11, 2001, ESSENCE was expanded to include virtually the entire Military Health System. Currently, ESSENCE downloads outpatient data on a daily basis from 121 Army, 110 Navy, 80 Air Force, and 2 Coast Guard installations around the world. Over 2,700 syndrome- and location-specific graphs are prepared each day and automatically analyzed for patterns that suggest a need for further investigation.248 Beyond these centralized assessments, the graphs are available daily to approved DOD public health professionals on a secure Web site.248 Two evaluations of ESSENCE are currently ongoing: one to address the issue of data quality, and the other to test the system's sensitivity and specificity over a range of outbreak scenarios. DARPA has awarded a $12-million 4-year grant to the ESSENCE consortium to construct a more powerful system for the Washington, DC area that includes both military and civilian data.
| Tally Sheet | Health Buddy® | RSVP® | EWORS | ESSENCE | LEADERS | |
| Hardware Requirements | Fax | Phone line and the device | Personal computer with Web connection | Personal computer with modem | Electronic recording of ICD9 codes | Personal computer with Web connection |
| Data Entry Personnel | Triage nurse | Triage nurse | Clinicians | Clinicians | None | Clinicians or trained clerks a |
| Timeliness | > 1 day | Immediate | Immediate | Immediate | > 1 day | Immediate |
During at least 1 implementation of LEADERS, it was necessary to deploy clerks in order to obtain data, as busy clinicians were unable to provide satisfactory data input shortly after program initiation.251
Syndromal surveillance systems could provide an early indication of cases resulting from a bioterrorism event. However, we have no evidence to determine which of the methods of collecting syndromal data provides the most sensitive, timely, acceptable, and low cost data. Local public health officials are the primary users of these data. Therefore, the ability of the system to present syndromal data in a timely manner, and minimize the necessity for additional analyses, will enhance the system's usefulness and acceptability.
We found no published standard definitions for the most common syndromes under surveillance. Additionally, none of the systems that we found that rely on clinicians to enter syndromal data provided definitions of the syndromes used by the collection tool (i.e., on the data entry screen, there was no definition of "flu-like illness"). A well-accepted list of the key syndromes for surveillance and detailed definitions of these syndromes will facilitate the integration of numerous sources of surveillance data. For example, the definition of "flu-like illness" should include its clinical characteristics so that triage nurses and clinicians can clearly identify patients with the syndrome, the specific ICD9 codes and other administrative data likely to be associated with it, the pharmaceuticals likely to be used to treat it, and the laboratory tests likely to be ordered to diagnose it. Then, each source of syndromal surveillance data can be systematically mapped to each of the syndromes, enabling the integration of all of these data into a single surveillance system.
As public health officials consider which of these systems to implement for ongoing or event-based syndromal surveillance, they are likely to find that a hospital's IT infrastructure may affect the acceptability of a syndromal collection tool. Hospitals and clinics with nascent IT infrastructures and an associated culture of paper may be better suited to the Tally Sheet or Health Buddy®. Within a given county, some hospitals may want to report ICD9 codes and others may prefer reporting triage nurse counts on the Tally Sheet. If different hospitals within a county are reporting syndromal surveillance data from different collection tools, public health officials will need to have the capacity to collect, analyze, and present these data in single system. The Oracle database associated with LEADERS has this capacity but is limited by the lack of standard definitions for the syndromes.
Although most industrialized nations mandate the reporting of selected infectious diseases, compliance is typically poor.252 A study of 176 clinicians in the United Kingdom (U.K.) found that 123 (70 percent) did not know where to obtain notification forms and most were unaware of reporting requirements.253 For example, 79 clinicians (45 percent) were unaware that cases of pneumococcal meningitis should be reported.253 Notable exceptions to this trend are the "astute clinicians" who, by reporting suspicious cases or clusters of cases to their local public health officials, have been largely responsible for the timely detection of infectious disease outbreaks (as in the case of West Nile Virus outbreak in New York City in late August 199915 and the recent cases of mail-associated anthrax254, 255). Efforts to enhance the awareness of clinicians, educate them about bioterrorism-related illness, and remind them of their reporting obligations are outside the scope of this Report. We refer interested readers to a comprehensive Evidence Report entitled "Training of Clinicians for Public Health Events Relevant to Bioterrorism Preparedness."256
Increasingly, public health officials have realized that the establishment of networks of practitioners with training in disease reporting could improve the quality of data obtained from voluntary communicable disease notification systems. A sentinel network is a disease surveillance program that involves the collection of health data on a routine basis by clinicians with some training (albeit minimal) in reporting communicable disease. The growth of such sentinel systems has generated demand for information systems capable of automating data collection, analysis, reporting, and communication. In this section, we describe systems for the electronic collection and analysis of clinical reports from individual clinicians and sentinel networks.
Our search identified 6 IT/DSSs used for the reporting of clinical cases for diseases other than influenza (which we present in a separate section of this Report); 2 of these have been described in peer-reviewed evaluation reports (Tables 3 and 16; Appendix H).
| System name | Purpose | Geographic location | Population or process under surveillance | Data used in surveillance system | Method of data collection | Sponsoring agency |
|---|---|---|---|---|---|---|
| EMERGEncy ID NET271, 272 | To detect new or unusual clinical events and gain knowledge about outcomes, as well as diagnostic and therapeutic approaches related to emerging infections. | Eleven U.S. urban, university-affiliated hospital emergency departments | Current projects include investigation of bloody diarrhea and the prevalence of Shiga toxin-producing E. coli, animal exposures and rabies post-exposure prophylaxis practices, and nosocomial emergency department Mycobacterium tuberculosis transmission. | Reports from clinicians (including housestaff) working in the affiliated emergency departments. | Sentinel physicians collect data on patients with certain clinical syndromes through multiple-choice forms. The information is entered into a desktop computer and electronically transferred to a central database at the Olive View-UCLA Medical Center for later analysis. The system does not provide real time feedback. | National Center for Infectious Diseases, CDC |
| Eurosentinel263 | To monitor influenza and other syndromes and diseases. | Europe | A wide variety of both infectious and non-infectious diseases, including influenza-like illness, meningitis, and pneumonia. | General practitioner reports. | Volunteer physicians submit weekly reports to a coordinating center in Belgium. Registration commitment is 1 year. Outputs for influenza in real time; outputs for other syndromes and diseases quarterly (released in a newsletter). | Institute of Hygiene and Epidemiology, Brussels |
| EUROTB268-270 | To collect, analyze and publish data on tuberculosis to improve control of the disease in Europe. | Europe | Tuberculosis and related drug resistance in Europe. | Reports include information on patient demographics, laboratory cultures and drug susceptibility testing. | Data are collected jointly with WHO Euro. As of 2001, yearly data are collected from each country using a standardized form. | European Centre for the Epidemiological Monitoring of AIDS (CESES); Royal Netherlands Tuberculosis Association (KNCV) |
| French Communicable Diseases Computer Network (FCDN)257, 259, 261, 262, 275-280/The French Sentinel System281/SentiWeb282, 283 | To collect, analyze, and present data on communicable diseases. | France | Communicable diseases, including influenza-like illness, urethritis, measles, mumps, chicken pox, acute diarrhea, viral hepatitis and prescription of HIV serological tests. (An epidemic is defined as rate in excess of the 95% confidence limit for 2 consecutive weeks.) | General practitioner reports. | A volunteer sample of about 1% of French general practitioners enters weekly reports on PCs with either modem or videotext terminals. Data are available online through a Web site called SentiWeb. | French Department of Health (Direction Generale de la Sante); National Institute of Health and Medical Research (INSERM) |
| Global Emerging Infections Sentinel Network (GeoSentinel)273, 274 | To monitor geographic and temporal trends in morbidity among travelers and other mobile populations. Can also be used to send alerts and surveys to a widespread network of providers. | Global | Communicable disease-related morbidity in mobile populations. | Clinician reports from travel/tropical medicine clinics around the world. | Reports from 25 sentinel clinics are either faxed or electronically submitted to a central database in Atlanta, GA. In addition, a rapid worldwide query and response function electronically links 1,500 ISTM providers worldwide. | CDC; International Society of Travel Medicine (ISTM) |
| National Electronic Telecommuni-cations System for Surveillance (NETSS)264, 266, 267 formerly known as the Epidemiologic Surveillance Project (ESP)265 | For rapid communication of notifiable disease and injury reports between participating health agencies and CDC. | U.S., Puerto Rico, the Virgin Islands, Guam, American Samoa, Common-wealth of the Northern Mariana Islands | Over 40 notifiable diseases, including Salmonella and E. coli. | Reports from participating agencies that contain 40 variables, including demographic characteristics, type of disease, and date of disease onset. | Notifiable disease reports from local health providers and central agencies are forwarded to state health departments. Each week, the state database sends data directly to a NETSS mainframe computer at the CDC via an electronic link. The data are processed for publication as a table in Morbidity and Mortality Weekly Report (MMWR). | CDC; CSTE |
In 1983, France took a technological lead in electronic disease surveillance with a national telecommunications program that provided videotext home terminals free of charge to French citizens.252 In 1984, the Institut National de la Santé et de la Recherche Médicale (INSERM), in collaboration with the French Ministry of Health, initiated a program for electronic monitoring of communicable diseases called the French Communicable Disease Network (FCDN).257-259 Today, a volunteer sample of about 1 percent of French general practitioners enters weekly reports on personal computers with either modem or videotext terminals.257-259 These data are available online through a Web site called SentiWeb. A major strength of the FCDN system for surveillance of infectious diseases is the timeliness of the collection, analysis and distribution of data. The reports of influenza-like illness from sentinel general practitioners, combined with information on viral isolates from the French Reference Centers, provide timely information for developers and evaluators of the influenza vaccine.260 FCDN's early detection system is based on a regression model, which has been demonstrated in a retrospective study to forecast epidemics with a delay of only 1 week.261 A 1998 evaluation of 500 sentinel practices of the FCDN found that although the system quickly offered estimates of the effectiveness of the influenza vaccine, and all results were available on the Internet within 1 week of data collection, delay could be shortened by updating data from the clinicians daily instead of weekly.262
The Eurosentinel project depends on an international network of sentinel general practitioners to monitor influenza and other syndromes and diseases in Europe. Volunteer physicians submit weekly reports to a coordinating center in Belgium. Outputs for influenza are within minutes of reporting, while data for other syndromes and diseases are released in a quarterly newsletter. A report describing the experiences of the first 3 years of the project found that discrepancies in disease reporting practices, particularly the use of different denominators, between the sentinel networks of different countries made it difficult to compare the data from each network.263
In the U.S., each state health department uses a standardized weekly form submitted by e-mail to CDC through the National Electronic Telecommunications System for Surveillance (NETSS).264-267 In the past, CDC returned analyses of this information back to the state health departments via an electronic text message system of the Morbidity and Mortality Weekly Report (MMWR), which these departments received earlier than other subscribers. CDC then established the Public Health Network (PHNET) to provide a tool featuring up-to-date maps and graphs to return information alerts to state health departments.252 Data on selected notifiable infectious diseases continue to be published weekly in the MMWR and at year-end in the annual Summary of Notifiable Diseases of the United States. An observational study monitoring the delay between the date of disease onset and date of report to CDC found that for states providing the date of disease onset for at least 70 percent of reported cases, the median reporting delay was 23 days for shigellosis, 22 days for salmonellosis, 33 days for hepatitis A, and 20 days for bacterial meningitis.264 Within 8 weeks of disease onset, more than 75 percent of reports were made for each of these diseases. Reporting delays varied widely between states, particularly for salmonellosis and shigellosis.264
In addition to the major surveillance efforts just described, we report on 3 other surveillance projects. EUROTB collects data on tuberculosis from multiple European countries and reports these data to the WHO.268-270 EMERGEncy ID NET is a collection of sentinel physicians from 11 U.S. emergency departments who collect data on patients with selected clinical syndromes through multiple-choice forms. The information is entered into a desktop computer and electronically transferred to a central database at the Olive View-UCLA Medical Center for later analysis.271, 272 The Global Emerging Infections Sentinel Network (GeoSentinel)273, 274 collects reports from 25 sentinel clinics to monitor geographic and temporal trends in morbidity among travelers and other mobile populations. Reports are either faxed or electronically submitted to a central database in Georgia. GeoSentinel can also be used to send alerts and surveys to a widespread network of providers. The only additional information available about these systems describe the incidence and prevalence of the diseases under surveillance. It is difficult to determine their potential utility during a bioterrorism event from these data.
Because clinicians may be the first to recognize unusual or suspicious illnesses, reports from clinician networks are an essential source of surveillance data for detection of bioterrorism-related diseases. However, the usefulness of the systems described in this section is difficult to assess. First, none of these systems has been evaluated for its ability to detect illness caused by bioterrorism events. Thus, the sensitivity, specificity, and timeliness of the systems have not been documented. Second, the descriptions of many of the systems were published in the early 1990s. It is possible that some have been upgraded to rely on electronic reporting, but current descriptions have not been published.
Systems for detection of bioterrorism require more rapid response times than do systems designed for some other purposes. The timeliness of systems that collect clinical reports depends on the time required to report data, analyze the data, and communicate the results of such analyses to decision makers who can respond appropriately. Systems that collect data weekly will be substantially less useful than systems that can provide more rapid collection and analysis. Of the systems in this section, Eurosentinel provides the timeliest data (but only for influenza, data on other diseases and syndromes has a longer delay). The timeliness of the other systems varies from days to months. Investigation of whether the systems can be modified to increase timeliness should be a high priority.
On June 26, 1957, 1,688 delegates from 43 states and foreign countries arrived at a church conference at Grinnell College.236 Among them were 100 Californian delegates who had made the trip to Iowa in a single chartered railroad coach.236 Cases of influenza had developed en route and when the 100 Californians interacted with the other delegates, an epidemic exploded.236 By July 1, more than 200 clinical cases of influenza were documented, the delegates ended their conference early, and returned home -- taking the virus with them.236
Often perceived as a comparatively low-level threat, the viruses that cause influenza are continually evolving and occasionally undergo sufficient genetic drift that they cause significant morbidity and mortality.241 For example, the 1918-1919 pandemic killed more than 20 million people around the world.241 The less severe pandemics in 1957 and 1968 killed a total of 1.5 million people and caused an estimated $32 billion in economic losses worldwide.241 Established in 1948, the WHO's global network for influenza surveillance currently includes 110 collaborating laboratories in 82 countries, continuously monitoring locally isolated influenza strains.240, 284 These data are used to make recommendations on the 3 virus strains to be included in the next season's influenza vaccine.240, 284 The WHO has also created FluNet, an Internet site for reporting and monitoring clinical cases of influenza.
The U.S. Air Force has actively contributed to the global influenza surveillance effort since 1976 through the efforts of Project Gargle, based at Brooks Air Force Base in San Antonio, Texas.245, 285, 286 In June 1996, President Clinton issued Presidential Decision Directive NSTC-7, which formally expanded the mission of the DOD to support global surveillance, training, research, and response to emerging infectious disease threats.245, 285, 286 The DOD-GEIS was formed in 1997 in response to this Presidential Directive. During the 1999-2000 influenza surveillance season, Project Gargle processed 3,825 throat swabs from 19 sentinel U.S. military bases, 49 nonsentinel bases, and 3 DOD overseas laboratories.245, 285, 286 The Air Force also correlates the immunization status of Air Force personnel with influenza morbidity, providing a marker for vaccine efficacy.245, 285, 286
The lessons of the past 50 years of global surveillance for influenza are applicable to the problem of surveillance for biothreat agents. Effective surveillance for bioterrorism-related diseases requires similar global "networks of networks" integrating clinical and laboratory data collected by governmental, charitable, military, private and professional organizations and reported to local, national, and international public health organizations.
Our search identified 11 IT/DSSs for influenza surveillance, 3 of which have been described in peer-reviewed evaluation reports (Tables 3 and 17; Appendix H).
The AAH Meditel United Kingdom General Practitioner Reporting System has been used for disease surveillance across the U.K.287 Each night, data are downloaded from the computers of hundreds of sentinel general practitioner offices to a mainframe computer at AAH Meditel.287 The system was originally intended to collect prescribing data but has been extended to collect other patient data. A 10-week study compared the AAH Meditel database of clinical reports of influenza cases to the manual influenza surveillance system of the Royal College of General Practitioners' and found the slope of the influenza epidemic curve to be nearly identical for both systems.287 This suggests that the system was able to detect an outbreak of influenza at least as well as the sentinel General Practitioners against whom it was compared (we have no additional sensitivity or timeliness information on this group). However, the incidence of disease as detected by the General Practitioners was 3 to 4 times higher than that derived from AAH Meditel.287 The authors suggest that the underreporting of the AAH Meditel system might be due to clinicians inexperienced in surveillance methods, especially in comparison with the trained sentinel Practitioners of the Royal College, and overestimation of the surveillance population.287
The French Regional Influenza Surveillance Group (GROG) system consists of almost 1500 voluntary general practitioners, pediatricians, pharmacists, and military medical officers who provide a weekly activity summary via telephone.288, 289 A regression-based method for early recognition of influenza epidemics based on time series analysis was used to evaluate surveillance data from various indicators. Sick-leave as prescribed by physicians and recorded by the "Assistance Publique" (Health and Social Security Services), emergency house calls, and numbers of patients with influenza-like illness seen by general practitioners and pediatricians were the most sensitive indicators for the early recognition of influenza epidemics. The detection of influenza epidemics using drug consumption data was 1 week delayed relative to house call and sick-leave indicators.288, 289 The annual epidemic threshold criteria are derived from a combination of these indicators (with virologic data) designed to minimize the false positive rate.288, 289 During the 1992-1993 season, using the threshold criteria developed from the 1984-1992 historical data, GROG detected an influenza B epidemic on week 52 -- 3 weeks after the virus began circulating in the population.288, 289
In 1991, the National Influenza Centre of the Netherlands set up an electronic influenza surveillance project using the existing Medimatica computer network that connected several health care institutions via a central server. The 3000 clinician and 1300 pharmacist subscribers to the Medimatica service receive low cost e-mail service and access to medical database services (including access to patient medical records and current information on a variety of diseases).290 A new database was added to the Medimatica system providing current information on Dutch influenza outbreaks and served as a means for clinicians and pharmacists to report suspected influenza cases. A prospective evaluation of this influenza reporting system during the 1991-2 influenza season found that no clinicians or pharmacists reported any influenza cases.290 The investigators suggested that this disappointing result might have been caused by a clinical culture in the Netherlands in which electronic reporting has not yet been embraced.290
None of the other influenza surveillance systems has been reported in a peer-reviewed evaluation. Many of them collect data on a daily basis with weekly reporting of data to public health officials. The systems vary with respect to their methods of data collection, including telephone, fax, mail, and e-mail. Most incorporate several sources of surveillance data (e.g., virologic data, clinician reports, hospital admissions data, pharmacy data, and work absenteeism data). No reports of these systems discussed information on methods to maintain the security of data or direct costs associated with the program.
Surveillance systems that collect influenza data are relevant to bioterrorism surveillance in 3 ways. First, sentinel clinicians who report on patients with suspected influenza are experienced at applying a case definition to a clinical population for the collection of public health data. Because many bioterrorism-related illnesses present with a "flu-like illness," this network of trained sentinel clinicians could provide valuable surveillance data. Second, most influenza surveillance systems integrate clinical and laboratory data for the detection of influenza outbreaks. Surveillance for bioterrorism may be aided by similar integration of multiple data sources. Finally, influenza surveillance is a coordinated global effort. Given the current state of international travel, importation of food products, and trade in pharmaceuticals and blood products, infectious diseases from natural sources or from acts of bioterrorism can readily travel to the U.S from remote areas of the world. New programs for the surveillance of bioterrorism-related illness could be derived from the existing IT infrastructures and the historical relationships that have been developed for influenza surveillance. Several of the influenza systems rely on weekly reporting by clinicians -- for bioterrorism surveillance, every effort should be made to reduce this lag.
Laboratories are critical to the detection of naturally occurring emerging infectious diseases, identification of biothreat agents, and monitoring for unusual patterns of antimicrobial resistance. The laboratory that encounters the first case of a covert bioterrorist attack may be located in a community hospital.302 Many of the biothreat agents are morphologically similar to the organisms that normal inhabit the human respiratory tract.302 For example, B. anthracis is morphologically identical to other Bacillus species -- most of which are contaminants of clinical samples.302 Therefore, if the organism is not speciated, the laboratory technician may consider the isolate a contaminant and discard the sample.302
APHL and CDC are developing the Laboratory Response Network, a network of civilian public health and private laboratories in the U.S. for routine disease surveillance and detection of biothreat agents.302 In the Laboratory Response Network, laboratories in local hospitals (Level A) can rule out biothreat agents or refer organisms to Level B laboratories.302 Level B laboratories can identify certain pathogens, such as anthrax, but must refer other organisms to Level C laboratories for further evaluation.302 Level C laboratories can definitively identify a broader range of pathogens, but must also refer some samples (such as smallpox and hemorrhagic fever viruses) to Level D laboratories.302 Only 2 Level D laboratories exist in the U.S., at the CDC and at the U.S. Army Medical Research Institute of Infectious Diseases (USAMRIID).302 An information system is currently being established for the Laboratory Response Network that includes: a password-restricted site (using NEDSS standards and systems) for ordering reagents, distributing procedural information, and standard messaging for results.302 The Laboratory Response Network has recently received additional federal funds enabling its expansion to 120 laboratories, each equipped with advanced laboratory technologies and trained staff.303
| System name | Purpose | Geographic location | Population or process under surveillance | Data used in surveillance system | Method of data collection | Sponsoring agency |
|---|---|---|---|---|---|---|
| Active Bacterial Core Surveillance (ABCs)309-311 | For surveillance of invasive bacterial diseases. | U.S. | Invasive bacterial diseases from organisms such as H. influenzae, S. pneumoniae and Neisseria meningitides. | Laboratory results from 9 Emerging Infections Program sites. | For each case of invasive disease, the laboratory files a case report that includes basic demographic information and the sample is sent to the CDC for evaluation. | CDC http://www.cdc.gov/ncidod/dbmd/abcs/default.htm |
| California Electronic Laboratory Disease Alert and Reporting (CELDAR) system312 | For laboratory-based surveillance of reportable diseases in California. | California | 28 electronically reportable diseases including anthrax, brucellosis, botulism, plague (animal or human), and tularemia, among others. | Laboratory reports from public health laboratories and animal and food safety laboratories. Also monitors number of requests for certain tests. | The system monitors for unusually high numbers of requests for certain tests, and also initial positive lab findings for unusual or atypical diseases for a geographic area. The system simultaneously sends reports to the state and local health departments. | California Department of Health Services |
| Detection Algorithm using Syndromal Classification 313, 314 | To detect and diagnose unknown or rare syndromes and/or changes in frequencies of more common syndromes. | Not applicable. | Changes in frequencies of syndromes, whether emerging or more common. | Laboratory records. | The system automatically extracts desired structural data from the laboratory records. A complex algorithm is applied to detect disease. | Department of Biology, University of New Mexico, Albuquerque, New Mexico |
| Electronic Communicable Disease Reporting System (ECDRS)304 | To automate laboratory reporting of communicable diseases to public health officials. | U.S. | Communicable diseases. | Laboratory results. | At a predetermined time each day, a computer dedicated to reporting purposes in each of the participating laboratories automatically connects to the laboratory information system in routine use at that facility. This connection launches a data extraction program that extracts and transmits data to a public health database. | Hawaii Department of Health |
| Emerging Pathogens Initiative (EPI)315-317 | For surveillance of emerging pathogens in 172 VA health care facilities worldwide. | Global | Fourteen pathogens and diseases, including vancomycin-resistant enterococcus, penicillin-resistant pneumococcus, E. coli O157:H7, and certain diseases of military importance, such as malaria. | Laboratory results, patient demographics, co-morbidities, antimicrobial susceptibility, and number of patients by facility. | The EPI program automatically extracts data from the existing VA medical information system and transfers it to a central repository for statistical analysis. Data are transferred on a monthly basis. | Program for Infectious Diseases, VA http://clinton5.nara.gov/textonly/WH/EOP/OSTP/Security/html/eidann_rpt.html#Federal |
| Laboratory Response Network302 | To improve response capabilities in the event of a bioterrorism attack with a nationwide network of medical laboratories. | U.S. | Infectious agents, both natural and those due to a bioterrorism attack. | Laboratory results. | The network consists of 4 levels of laboratories, each with differing biohazard capabilities. Please see text for additional details. | CDC; APHL |
| National Respiratory and Enteric Virus Surveillance System (NREVSS)318 | To monitor temporal and geographic patterns associated with the detection of respiratory and enteric viruses. | U.S. | Respiratory syncytial virus, human parainfluenza viruses, respiratory and enteric adenoviruses, and rotavirus. Influenza specimen information, also reported to NREVSS, is integrated with CDC Influenza Surveillance data. | Laboratory results from collaborating university, community hospital, commercial, and state and county public health laboratories. | Participating laboratories report virus detections, isolations, and electron microscopy results on a weekly basis. Annual summaries from NREVSS are published in MMWR. | CDC http://www.cdc.gov/ncidod/dvrd/nrevss/ |
| National Tuberculosis Genotyping and Surveillance Network319 | To combine data on DNA fingerprinting with other disease surveillance information into 1 research and surveillance tool. | U.S. | Tuberculosis strains. | DNA fingerprint images and epidemiologic information. | The members of the network input data into a centralized database at the CDC, where matching of DNA fingerprints of TB strains is automated. | CDC http://www.cdc.gov/ncidod/dastlr/tb/tb_tgsn.htm |
| Netherlands National Institute of Public Health and the Environment (RIVM) Surveillance System320 | To catalog and track resistance patterns of clinically isolated bacteria. | Netherlands | Antibiotic resistance, including that of S. aureus and coagulase-negative staphylococci. | Laboratory reports from 7 public health labs; encompasses roughly 25% of all susceptibility tests done in Dutch labs. | Participating public health labs submit data according to the criteria of the Dutch Committee on Antibiotic Susceptibility. | Netherlands National Institute of Public Health and the Environment (RIVM) |
| Public Health Laboratory Information System (PHLIS)307, 321 | For the investigation and surveillance of outbreaks. | U.S. | Cases/isolates of specific notifiable diseases. | Data from epidemiologic, laboratory, survey, and case control studies. | Data Entry Screens (modules) are created and distributed to all reporting sites electronically. Data input and reporting occurs within hours. In addition, there is relational database software on individual PCs with connections to labs and public health computers and databases. | CDC http://www.cdc.gov/ncidod/dbmd/phlisdata/default.htm |
| Public Health Laboratory Service (PHLS) Communicable Disease Surveillance Centre (CDSC)322 | To compile microbiology reports from around the U.K. | U.K. | Laboratory reports. | Microbiology reports from PHLS and non-PHLS laboratories from around the U.K. | Microbiology labs send either paper-based reports or electronic reports to CDSC. Analysis programs run over the weekend and reports are generated for Monday morning review by CDSC staff. | PHLS |
| Salmonella Potential Outbreak Targeting System (SPOT)305/National Enteric Pathogens Surveillance Scheme (NEPSS), 306 formerly known as the National Salmonella Surveillance Scheme | For the early detection of potential Salmonella outbreaks. | Australia | Cases of gastroenteritis caused by Salmonella enterica and Shigella species. | Laboratory reports of Salmonella isolated from human sources. | Isolates are serotyped and, where appropriate, phage typed before being entered into the central database at the University of Melbourne. An electronic system automatically flags geographically and/or temporally abnormal clusters of a single serovar (and phage type, if appropriate), using a hybrid statistic/heuristic algorithm. Sensitivity of this system is over 90%.306 | Microbial Diagnostic Unit, The University of Melbourne, Australia |
| System name | Purpose | Geographic location | Population or process under surveillance | Data used in surveillance system | Method of data collection | Sponsoring agency |
|---|---|---|---|---|---|---|
| Enter-Net, formerly known as Salm-Net323, 324 | To monitor infections and antibiotic resistance related to enteric pathogens, and to investigate outbreaks. | All European Union countries, plus Switzerland and Norway. | Enteric pathogens such as Salmonella and E. coli. | Laboratory results, including susceptibility testing, from each country's national reference lab. | Laboratory results from each country are entered into a central database, which can be used to monitor outbreaks. | European Commission |
| Global Salm-Surv (GSS)325, 326 | To facilitate communication and data exchange between labs that isolate, identify, and test specimens for Salmonella in order to improve the quality and capacity of testing. | Global | Salmonella. Eventually, it is anticipated that GSS will be extended to other major foodborne pathogens. | Laboratory results from national and regional salmonellosis labs, including annual summaries of serotypes and antimicrobial resistance patterns. | Laboratory data are collected annually from each participating lab. An online database is currently being developed, into which these data will be deposited. Web users will be able to search the database for serotype frequency at different geographical levels (i.e. nationally, globally). | WHO, Department of Communicable Disease Surveillance and Response; WHO Collaborating Center for Foodborne Disease Surveillance; Danish Veterinary Laboratory |
| Intensive Care Antimicrobial Resistance Epidemiology (ICARE)327 | To track antimicrobial resistance among pathogens responsible for nosocomial infections in ICUs. | U.S. | Antimicrobial resistance at a subset of hospitals involved in the National Nosocomial Infection Surveillance System (NNIS). | Usage of antimicrobials, resistance data, and hospital characteristics. | Voluntary reports from participating hospitals along with isolates for confirmatory testing. | Hospital Infections Program (CDC) |
| International Network for the Study and Prevention of Emerging Antimicrobial Resistance (INSPEAR)328, 329 | To serve as an early warning system that can rapidly distribute information and provide microbiologic and epidemiologic support to INSPEAR members. | Global | Drug-resistant organisms are monitored at 160 health care agencies in 33 countries. | Laboratory results. | Antimicrobial resistance testing is performed at local INSPEAR facilities. In the case of an emerging event, confirmation testing is performed at a regional or national INSPEAR location, public health officials are notified, and additional epidemiologic testing is performed at the local or regional level. | CDC |
| Laboratory Information Tracking System (LITS)321 | To track laboratory specimens across different laboratories. | U.S. | Emerging diseases. | Laboratory data, including results from all laboratories that performed tests on a particular specimen. | Specimen information is deposited at a central receiving site from PCs integrated with a local area-network. Any laboratories performing tests on a particular specimen can update the information. | CDC |
| Military Public Health Laboratories330 | To develop regional surveillance networks. | Military laboratories and partner laboratories worldwide. | Drug-resistant organisms including enteric organisms, malaria, Neisseria gonorrhoeae. | Laboratory specimens. | Isolates are obtained and tested at sentinel and non-sentinel sites. Reports include geographic analysis and are transmitted electronically. | DOD-GEIS |
| National Antimicrobial Resistance Monitoring System (NARMS)331 | To monitor antimicrobial resistance in human enteric pathogens. | U.S. | Approximately 103 million people (38% of the US population) are located within NARMS sites. Resistance in pathogens such as Shigella, E. Coli O157:H7, Campylobacter, and Salmonella are monitored. | A fixed proportion of all isolates received at 17 state and local public health laboratories (e.g., every 10thShigella, every 5thE. Coli O157). | Enteric pathogen isolates are sent to the CDC for susceptibility testing. | CDC; Food and Drug Administration (FDA); U.S. Department of Agriculture (USDA) |
| SENTRY332, 333 | To monitor prevalent bacterial and fungal pathogens and their patterns of antimicrobial resistance. | Global | Bacterial and fungal pathogens, such as S. aureus and S. epidermidis, involved in community and nosocomial-acquired infections. | Laboratory isolates (roughly 540 per year for each participating lab). | Isolates are forwarded to regional monitors for pathogen susceptibility tests and for confirmation of identity. | Not available |
| Surveillance for Emerging Antimicrobial Resistance Connected to Healthcare (SEARCH)334 | To monitor vancomycin-resistant S. aureus. | U.S. | Network of voluntary participants (i.e., hospitals, representatives of private industry, professional organizations, and state health departments). | Susceptibility results of vancomycin-resistant S. aureus isolates. | Initial results sent by e-mail. Confirmatory testing provided by the CDC. | Division of Healthcare Quality Promotion (CDC) |
| The Surveillance Network™ (TSN™) Database-USA308, 335 | To collect antimicrobial resistance data for use in the surveillance of resistance trends. | U.S. | Antimicrobial sensitivity in bacterial organisms. | Laboratory data from 229 laboratories "chosen for participation on the basis of their geographic and demographic characteristics" and ability to perform antimicrobial susceptibility testing. | No additional information provided. | MRL Pharmaceutical Services, Herndon, VA |
| WHONET336-339 | To track antimicrobial resistance. | Not applicable. | WHONET is a software program that facilitates local collection, analysis, and dissemination of resistance data into a universal file format. | Antimicrobial resistance data. | Data collected and entered locally (manually). | WHO; Brigham and Women's Hospital, Boston, MA |
A clinical evaluation of the Electronic Communicable Disease Reporting System (ECDRS) operated by the Hawaii Department of Health demonstrated that this automated laboratory reporting system improved communicable disease reporting to public health officials relative to conventional reporting methods.304 At a predetermined time each day, a computer dedicated to reporting purposes in each of the participating laboratories automatically connected to the laboratory information system in routine use at that facility. This connection launched a data extraction program that then transmitted the laboratory data to a public health database. During a 6-month evaluation period, 325 (91 percent) of 357 laboratory reports were reported through ECDRS, compared with 156 (44 percent) reported through conventional methods.304 For the combined 124 reports received from both reporting mechanisms, electronic reports were received an average of 3.8 (95 percent confidence interval: 2.6-5.0) days earlier than were corresponding reports sent by mail or fax.304 Of the 21 data fields per report, 12 were significantly more likely to be complete in the electronically reported data; however, the field describing the type of specimen (e.g., from blood or stool) was more likely to be reported by conventional methods.304
The National Enteric Pathogen Surveillance Scheme (NEPSS)/Salmonella Potential Outbreak Targeting System (SPOT) is designed for the early detection of potential Salmonella outbreaks in Australia.305 A preliminary retrospective analysis of this system demonstrated a sensitivity of 100 percent (15 out of 15 Salmonella outbreaks), compared with 50 percent for another surveillance method in which epidemiologists "eye-balled" the data.305 During a separate 3-year retrospective study, NEPSS/SPOT detected 124 out of 134 potential Salmonella outbreaks, thereby demonstrating a sensitivity level greater than 90 percent and a positive predictive value in the range of 53 to 68 percent.306
The Public Health Laboratory Information System (PHLIS) is used for the investigation and surveillance of outbreaks of specific notifiable diseases in the U.S.307 Outbreak-specific Data Entry Screens are created and distributed to all reporting sites electronically. Data input and reporting occurs within hours. In addition, there is relational database software on individual personal computers with connections to laboratory and public health computers and databases. An evaluation of PHLIS used different methods to calculate expected values of Salmonella serotype Enteritidis.307 This retrospective analysis demonstrated that using a 5-week mean as the threshold for what constituted an outbreak was more sensitive than using either the 5-week median or the 15-week mean. Using the 5-week mean as the threshold for determining an outbreak, the system had 3 false negatives, 76 percent sensitivity, 95 percent specificity, and 77 percent false-positive rate.307 The authors remarked that their evaluation may have been affected by their definition of an outbreak (the system would have missed an outbreak of 3 cases if it occurred in a season with a high background number of reported cases).307 The authors report that this system, which detected an outbreak of Salmonella serotype Stanley in May 1995, resulted in a more timely investigation of the outbreak than would have occurred with conventional surveillance -- the investigation implicated alfalfa sprouts as the vehicle of infection resulting in "the prompt development of prevention measures."307
Among the laboratory systems specifically designed for surveillance of antimicrobial resistance, only The Surveillance Network™ (TSN™) Database-USA has been clinically evaluated. Laboratory data obtained from 229 laboratories "chosen for participation on the basis of their geographic and demographic characteristics" and ability to perform antimicrobial susceptibility testing are sent to the TSN™ Database-USA. One study compared in vitro susceptibility testing of isolates from 27 hospital laboratories with surveillance data for 200 laboratories from TSN™ Database-USA for the current status of fluoroquinolone activity against gram-negative species. Despite the fact that the comparison data was from different time periods and for different samples, the 2 surveillance systems agreed in many areas. The authors suggested that the discrepancies between the systems may be due to reporting practices of clinical laboratories, the geographic distribution or number of isolates, or the number of laboratories involved in the studies.308
In general, the evaluative and descriptive reports of the systems collecting laboratory and antimicrobial resistance data suggest that the electronic systems improve the timeliness and sensitivity of conventional methods. Few reports specifically described how laboratory samples are handled, methods for confirmation of laboratory samples, acceptability, or cost of implementation.
Laboratory testing will be an essential component of any bioterrorism response effort. Systems that facilitate the collection, analysis, and reporting of notifiable pathogens and antimicrobial resistance data could potentially facilitate the rapid detection of a biothreat agent. However, none of these systems has been evaluated for such use.
Limitations of the current U.S. laboratory system for bioterrorism surveillance include lack of staffing and equipment for rapid detection in local laboratories and lack of a robust communication infrastructure among the different levels of laboratories. Efforts are ongoing to correct some of the gaps. Specifically, the Laboratory Response Network, which builds on existing laboratory capacity and is currently under active expansion, was designed so that it can be integrated into surveillance networks (such as NEDSS) and communication networks (such as California's Rapid Health Electronic Alert, Communication, and Training system (RHEACT)).
The primary objective of hospital surveillance is to track hospital-acquired infections, not to identify undiagnosed infections from the community. However, hospital epidemiology systems could play 2 roles in the early detection of a covert bioterrorist attack: the identification of a cluster of recently admitted cases suggestive of a community-based outbreak, and/or the identification of a cluster of cases within the hospital suggestive of patients with an unrecognized communicable disease.
A bioterrorist attack resulting in a large number of people seeking medical attention at the same time would likely be identified by other means. However, a smaller release of a biothreat agent may result in the hospitalization of a few sentinel patients who could be detected by a hospital surveillance system. For example, if a biothreat agent that causes meningitis was released in a covert attack, patients receiving a large inoculum may present to the emergency department of their local hospitals and be admitted to the ICU. Patients receiving a small inoculum may present to their internists or urgent care clinics and be admitted to the medical wards of the same community hospital. Another group of patients may present to a neurologist and be admitted to the neurology service of the hospital. Under these circumstances, no single team of clinicians will have cared for these patients and each is likely to be unaware of the other similar cases within the hospital. The detection of such a cluster would fall to the hospital infection control service.
Alternatively, if cases of smallpox with its characteristic rash present to the emergency department, astute clinicians may recognize the disease and institute appropriate isolation measures. However, if a few patients present with acute onset of fever, chills, myalgia, cough, and chest X-ray revealing patchy infiltrates, they are not likely to be placed in respiratory isolation until Y. pestis is found in their blood cultures. The early identification of patients with contagious infections and their prompt isolation depend on the effectiveness of the hospital epidemiology service.
The hospital epidemiology IT/DSSs that are likely to identify a bioterrorism event are those that function in a timely and sensitive manner. Traditional hospital infection control surveillance relies on the manual review of suspected cases and the retrospective analysis of aggregated surveillance data. This approach tends to be labor intensive, expensive, and slow. Efforts to improve hospital surveillance with IT/DSSs have focused on automated alerts of abnormal microbiology cultures (see also the section of this chapter on Reporting and Communication Systems), identification of high-risk patients, and the detection of infection rates above a statistical background rate.
| System name | Purpose | Geographic location | Population or process under surveillance | Data used in surveillance system | Method of data collection | Sponsoring agency |
|---|---|---|---|---|---|---|
| BOSS™ Computerized Surveillance System348 | To provide surveillance of nosocomial infections and outbreaks. | Not applicable. | Bacterial isolates leading to nosocomial infections. | Laboratory data, not including site of isolate. | The BOSS™ software program reviews isolate data prospectively and establishes baseline thresholds; it then provides reports and threshold diagrams that must be further analyzed to identify infections and outbreaks. | None |
| Clinical Data Repository (CDR)349 | For infection control including antibiotic resistance, drug usage and nosocomial infections. | U.S., Beth Israel Deaconess Medical Center, MA | Resistant organisms, such as methicillin-resistant S. aureus and vancomycin-resistant enterococci, are tracked in relation to nosocomial infections, organism susceptibility, and antibiotic use. | Demographic, testing and treatment data, including laboratory and radiology results, ICD-9 codes, and inpatient medications. | Developed via Microsoft™ Access, the main body of CDR consists of 47 Structured Query Language tables. Real-time updates occur for some tables, such as those for laboratory results and medications, while other tables are updated on a daily basis. | Beth Israel Deaconess Medical Center, Boston, MA |
| Computer-Assisted Infection (CAI) Monitoring Program350 | To integrate patient, lab, and epidemiologic surveillance of antibiotic resistance data in order to manage nosocomial infections in ICU patients. | Germany | Nosocomial infections in ICU patients. | Patient, laboratory and epidemiologic data, including diagnoses and therapeutic decisions. | Physicians enter data into UNIX® workstations at the patient's bedside. CAI is electronically linked to other data management systems in the ICU. The data are evaluated with the query-by-example method. | University Hospital of T?bingen, T?bingen, Germany; Niemetz GmbH Weiss-kirchen, Austria |
| Danish National Hospital Discharge Registry344 | To develop a discharge registry of all patients admitted to Danish hospitals (except psychiatric); an algorithm was developed to see if data source is useful for surveillance. | Denmark | All hospital patients except psychiatric cases. | Clinician ICD10 codes. | Upon discharge, all patients are automatically registered with their discharge diagnoses in the database. This information is then transferred to the Danish National Registry of Patients. | Danish government |
| Decentralized hospital computer program (DHCP) of the VA Hospitals351 | To identify infection trends within VA hospitals by ward and time. | U.S. | Infections in the VA hospital system. | Selected data from electronic medical records, including microbiologic and epidemiology reports. | Uses the electronic medical record to create reports that facilitate infection control and monitor trends in hospital infections over time and space. | VA |
| GermWatcher340-343 | To detect outbreaks of new infections and rising endemic rates of preexisting infections. | U.S. | Nosocomial infections. | Positive microbiology culture results from hospital patients. | Each morning, positive lab results are automatically transferred to GermWatcher. The system makes recommendations to keep, discard or watch the cultures (based on the CDC's criteria for potential nosocomial infections). This classification is reviewed by 1 of 3 infection control nurses and changed if necessary. | Washington University School of Medicine and Department of Medical Informatics, St. Louis, MO |
| Haemsept352 | To detect infections among hospitalized patients and to provide guidance for antibiotic prescribing on a hematology unit. | U.K. | Septicemia among hospitalized patients on a hematology unit. | Demographic, laboratory, and treatment data, including sensitivity testing, underlying illness, white blood cell count, and predisposing factors. | Clerical officers enter patient information into computer. In the future, the program will be linked with hospital computer systems for automatic collection. | Royal Victoria Hospital, Belfast, Northern Ireland |
| Henri Mondor University Hospital Clinical Information System234, 353 | To assist in identification of nosocomial infections, and to track resistance patterns and antibiotic prescriptions. (Alerts clinicians and infection control personnel to the presence of multi-resistant organisms and provides patient-specific data on positive cultures. | France | Hospital-acquired infections, multi-resistant bacteria and antibiotic prescribing patterns. | Demographic, discharge, bacterial infection data; antibiotic characteristics and orders. | Separate data reports are created for each of the 3 processes under surveillance using data extracted from a hospital information system. Clinicians can access reports and alerts at a secure Web site. | Hospital Information Department, Henri Mondor University Hospital AP-HP, Creteil, France |
| Hospital Infection Standardized Surveillance (HISS)346, 347 | To decrease time and increase standardization of data collection on hospital data for infections. | Australia | Nosocomial infections. | Hospital data, such as demographic and laboratory information. | Data are entered into a handheld PC, and then electronically transferred to a desktop PC for storage and analysis using specialized software. | New South Wales Health Department, Australia |
| Knowledge-based Information Network Giessen (WING)354 | To detect nosocomial infections, even when only limited amounts of clinical data are available. | Germany | Hospital-acquired infections. | Information from the hospital information system, including lab data. | Data are automatically abstracted from the electronic medical record by the surveillance system. Five different engines are used for each part of the main task: patient pre-selection to remove all patients definitely not at risk based on inclusion criteria; detecting patients at high risk for a nosocomial infection, using a rule-based reasoning process; an alert process; an explanation process; and statistical tools. | University of Giessen, Germany |
| LDS Data Mining Surveillance System (DMSS)186, 191 (part of HELP) | Please see section on Management/Prevention Systems for a description of this system. | |||||
| National Nosocomial Infections Surveillance (NNIS) System 355, 356 | To collect nosocomial infection surveillance data that can be aggregated into a national database for monitoring of trends in infections and risk factors. | U.S. | Nosocomial infections in acute care general hospitals. | Detailed epidemiologic and lab information, including antimicrobial resistance and outcomes data, from approximately 315 voluntary hospitals. | Trained infection control personnel collect data using protocols aimed at inpatients with high risk factors. Information is entered into computers using Interactive Data Entry and Analysis System (IDEAS) software. IDEAS allows the creation of reports and graphics, as well as the transfer of information to a central CDC database. | Hospital Infections Program (CDC) http://www.cdc.gov/ncidod/hip/surveill/nnis.htm |
| The Royal Hospitals' Automated Infection Surveillance System357, 358 | To improve accuracy and decrease time needed to conduct in-hospital infection surveillance. | U.K. | Infections in hospitalized patients. | Demographic, procedural, and infection information. | Automated data entry is performed using an optical scanner and Formic for Windows. | The Royal Hospitals, Belfast, Northern Ireland |
| Tucson VA Nosocomial Infection Surveillance System345 | To identify potentially preventable nosocomial infections. | VA Hospital, Tucson, AZ | Identifies excessive rates of positive cultures for nosocomial infections. Excessive positive culture rate is greater than or equal to 4 times the monthly baseline culture rate. | Laboratory results, including patient location, culture site, organism identified, and susceptibility patterns. | Data are entered into a computer program by typing a single number that correlates with a list of appropriate responses. Average entry time is 30 seconds for all information from a single report. | VA |
| University of Alabama Data Mining Surveillance System (DMSS)213, 214 | To automatically identify new, unexpected, and interesting patterns in surveillance data for infections that are not constrained to outbreaks for user-defined outcomes. | U.S., Alabama | Nosocomial outbreaks or new microbial resistance patterns. | Inpatient culture data from electronic medical records; currently only for bacteria (not viruses). | Data mining is performed on data from patient electronic medical records. Currently, the system is only capable of performing analysis on discrete/categorical (not continuous) variables. | University of Alabama |
| University of Iowa Hospitals and Clinics (UIHC) Program of Hospital Epidemiology (PHE) Surveillance System359 | To manage surveillance of nosocomial infections. | U.S., Iowa | Hospital-acquired infections, including bacteremias and endemic pneumonias. | Microbiology, treatment, and risk information. | Data on patient cases are entered into computer system by technicians each day. This system is linked with electronic databases for surgical procedure data and microbiology data. | UIHC |
Five of the 16 systems have been evaluated for their detection capabilities (others have been evaluated for other outcomes such as user acceptability which will be discussed at the end of this section). As we discussed earlier in this Report, the HELP system at the LDS Hospital in Salt Lake City uses the Data Mining Surveillance System (DMSS), which has the demonstrated ability to identify unusual patterns in surveillance data from sources including the microbiology laboratory, nurses' charts, chemistry laboratory, surgical record, and pharmacy.186, 191
Researchers at the University of Alabama at Birmingham have developed a system that they also call the Data Mining Surveillance System (DMSS).213, 214 They performed an evaluation of the DMSS using all positive inpatient microbiology cultures for a 15-month period. Each month of the study, 475 to 677 records were used in the algorithm with a running time of less than 4 minutes. The system detected 2 outbreaks of a highly resistant strain of Acinetobacter baumannii and changes in the incidence of multi-drug resistant Klebsiella pneumoniae that were not detected by conventional hospital surveillance. It also provided geographic information suggesting that in some units (e.g., the surgical ICU) patterns of antibiotic use may have been associated with changes in antimicrobial resistance, which was unknown before the use of the system.214
GermWatcher is a system designed to detect both outbreaks of new infections and rising endemic rates of preexisting infections in hospitals.340-343 Each morning, positive laboratory results are automatically transferred from the hospital database to GermWatcher. The system makes recommendations to the infection control officer to keep, discard or watch the cultures, based on the CDC's criteria for potential nosocomial infections. The system was evaluated by comparing these recommendations to those of the hospital control officers. Changes in the detection algorithms between GermWatcher Versions 1 and 2 resulted in improvement from 14 percent to less than 2 percent in rates of disagreement between the system and infection control officers. The final version of the system misclassified 3.5 percent of the 1851 cultures evaluated (2.8 percent false positives and 0.7 percent false negatives).341 The interpretation of these results is difficult because no additional information was provided regarding the use of the infection control officer as a gold standard (i.e., the sensitivity and specificity of their decision making).
The Danish National Hospital Discharge Registry, a registry of all non-psychiatric patients admitted to Danish hospitals, was developed to determine if hospital discharge data (clinician-provided ICD10 codes) could be used for the detection of bacteremia.344 Out of 45,000 patients discharged from Aalborg Hospital, Denmark in 1994, a diagnosis of septicemia or sepsis was found in the discharge database 207 times for 186 patients. Of these, 183 episodes (88 percent) were not in the bacteremia database maintained by the regional department of clinical microbiology. Using the clinical microbiology bacteremia database as the gold standard, the sensitivity of septicemia and sepsis registration in the Danish National Hospital Discharge Registry ranged from 4.4 percent to 5.9 percent (18 to 24 cases out of 406) depending on the definition of bacteremia used. The positive predictive value of the registry was 21.7 percent (95 percent confidence interval: 12.8 to 30.5 percent) since only 18 out of 83 episodes of septicemia found in the Danish National Hospital Discharge Registry were confirmed by the clinical microbiology data.344 These data suggest that the use of ICD10 data as collected in that hospital discharge registry do not significantly add to the information already available in the clinical microbiology data.
The Tucson VA Nosocomial Infection Surveillance System was designed to identify potentially preventable nosocomial infections.345 During a 6-month study, of the 19 clusters of infections and 3 outbreaks of intravenous catheter-related bacteremias identified by a combination of the system and epidemiological investigations, only 3 of the 22 were identified by standard surveillance methods. Standard surveillance found no additional infections.345 We conclude that this system significantly improved the detection of nosocomial infections; how this relates to the detection of inpatients with bioterrorism-related illness is unclear.
The other evaluations of the hospital surveillance systems reported similar improvements in the ability to detect hospital-based infections above what would have been detected by manual methods alone. Several evaluation reports did not report detection outcomes, rather, they primarily presented data regarding the acceptance of the system by hospital infection control officials. For example, the Hospital Infection Standardized Surveillance (HISS)346, 347 system was designed to improve the timeliness and standardization of data collected about nosocomial infections. Data are entered into a handheld computer, and then electronically transferred to a desktop personal computer for storage and analysis using specialized software. In an evaluation of HISS for the reporting of procedure-related infections in public acute care hospitals in New South Wales, Australia, the authors reported that although all 9 infection control officers who used the handheld device "had initial difficulties in adapting to the new technology, personnel at 7 sites consider it a useful tool. The near 100 percent completion rate for the data sets achieved by all 9 sites testify that the handheld computer assisted ICPs (infection control professionals) to collect large data sets."346
In general, the systems collecting hospital-based infections data with the most compelling evidence for effectiveness are those that are integrated into a robust hospital IT infrastructure. The different systems had different purposes and therefore varied considerably with respect to the information they collected and the kinds of alerts sent to hospital infection control officials. None was specifically designed for integration into a national bioterrorism surveillance program, and it is not clear how to evaluate which of the systems presented would be best suited for that purpose. These incorporate local estimates of pathogenic prevalence and resistance with clinical data to provide hospital infection control personnel with timely, sensitive, and relatively specific analyses. None of the included systems described cost of implementation.
The hospital surveillance systems that automate the collection of data from hospital-based laboratories and clinical records (i.e., the LDS and University of Alabama DMSSs, CDR, DHCP in VA hospital, and WING) are likely to be more timely than the manual systems that they replaced.
From the reports of the 5 systems that have been evaluated for their detection capabilities, we conclude that some of these systems could be a valuable tool for hospital infection control officers. However, there is little evidence demonstrating that they have sufficient sensitivity, specificity, or timeliness to detect a community-based bioterrorism event.
The integration of data from hospital-based surveillance systems, already collected in electronic format for use by hospital infection control officers, could be a valuable addition to a surveillance system organized by local public health officials. Similarly, the collection and reporting of hospital-based infections data from networks of hospitals (such as the VA) could contribute to a national bioterrorism surveillance system.
On September 17, 1984, the Wasco-Sherman Public Health Department received the first case reports of the 751 victims of the intentional contamination of salad bars in The Dalles, Oregon with Salmonella typhimurium.7 The lengthy epidemiologic and criminal investigations that followed demonstrated that the nation's largest foodborne outbreak of 1984 was the result of a bioterrorist attack by followers of Bhagwan Shree Rajneesh.7
As evidenced by this event, terrorists can exploit vulnerabilities to our food supply. Agroterrorists may be motivated to cause human morbidity and mortality through contamination of foods during harvest, processing, or preparation, or they may be interested in creating the economic burden resulting from reduced food supply.360 Attacks may be made against food and agriculture transportation systems, on water supplies, on farm workers, on food handlers, or on processing facilities.360 Insect hosts for diseases affecting crop plants or livestock may be imported and released with the intent of creating an epidemic or influencing a nation's ability to export agricultural products abroad.360 U.S. reliance on imported fresh fruits and vegetables has grown to such an extent that the safety of imported foods has become a source of major public health concern.360
Most industrialized nations mandate the reporting of foodborne illnesses and have established surveillance systems for foodborne illnesses that collect data from health officials or clinical laboratories. In the U.S., reportable foodborne diseases and organisms include: botulism, brucellosis, cholera, E. coli O157:H7, hemolytic uremic syndrome, post-diarrheal salmonellosis, shigellosis, typhoid fever, Hepatitis A, cryptosporidiosis, cyclosporiasis, and trichinosis.361
Ours is not a comprehensive review of agroterrorism. We direct interested readers to other reports on the topic.7, 82, 360, 361 Our search produced 2 types of IT/DSSs for surveillance of foodborne illness: those that collect and analyze reports from clinicians and laboratories about the incidence and laboratory characteristics of foodborne pathogens, and those that model microbial growth responses to various food production methods. Our search also found 2 studies on active monitoring systems using implantable sensors in livestock to allow for identification of animals and surveillance of the animals' vital signs.362, 363 Additionally, we found reports on 3 DSSs designed to assist veterinarians and animal handlers with diagnosis and management of common animal diseases (EPIZOO, Associate, and BOVID-3).364, 365 However, these systems are outside the scope of our project and are not described further in this section.
| System name | Purpose | Geographic location | Population or process under surveillance | Data used in surveillance system | Government agency |
|---|---|---|---|---|---|
| Foodborne illness and contamination surveillance systems | |||||
| Food and Animal Residue Avoidance Databank (FARAD)378, 379 | To build a computerized databank of data necessary to solve a drug or chemical residue problem in food-producing animals. | U.S. | Drug and chemical levels in animals are used to estimate decontamination times for specific drugs and animals. | Data entry at regional access centers by those requesting information. | USDA http://www.farad.org/ |
| Foodborne Diseases Active Surveillance Network (FoodNet)366-372 | To monitor foodborne diseases. | Eight sites in the U.S. (California, Connecticut, Georgia, Maryland, Minnesota, New York, Oregon, and Tennessee) | Outbreaks of 9 foodborne illnesses in humans (Campylobacter, Cryptosporidium, Cyclospora, E. coli O157:H7, Listeria, Salmonella, Shigella, Vibrio, and Yersinia). | Laboratory reports and reports by public health officials. | CDC; USDA; FDA http://www.cdc.gov/foodnet/ |
| Joint United Nations Environment Programme (UNEP)/Food and Agriculture Group of the United Nations (FAO)/WHO Food Contamination Monitoring Programme380 | To compile food contamination monitoring data in Europe. | Europe | Nineteen contaminants including selected pesticides, industrial chemicals and naturally occurring toxins. | Reports from participating countries. | UNEP; FAO; WHO |
| National Molecular Subtyping Network for Foodborne Disease Surveillance (PulseNet)381 | To create a national molecular subtyping network for foodborne bacterial disease surveillance. | U.S. | Four foodborne pathogens suspected of causing an outbreak: E. coli O157:H7, Listeria monocytogenes, nontyphoidal Salmonella, and Shigella. | Pulsed-field gel electrophoresis characteristics of bacterial strains. | CDC http://www.cdc.gov/ncidod/dbmd/pulsenet/pulsenet.htm |
| Salmonella Data Bank (SDB)382 | To create a single Salmonella reporting system that combines case reports, laboratory data, veterinary data, agricultural data, and labor statistics. | Germany | Human Salmonella infection. | Case reports from district public health offices to the National Reference Laboratory. | Municipal Medical Investigation Office, Frankfurt an der Oder, Germany |
| Salmonella Outbreak Detection Algorithm (SODA)371 | To track, via serotyping and a statistical algorithm, outbreaks and clinical isolates of Salmonella. | U.S. | Outbreaks of Salmonella. | Electronic state public health laboratory reports collected through PHLIS (see Table 18); isolates are compared by serotype and week to a 5-year historical baseline. | CDC PHLIS Helpdesk 404-639-3365 http://www.cdc.gov/ncidod/dbmd/phlisdata/ |
| WHO Surveillance Program for the Control of Foodborne Infections and Intoxicants in Europe380 | To monitor and register foodborne diseases and contamination. | Europe | Food and outbreaks of foodborne illness in humans. | Reports from participating countries. | WHO |
| Foodborne illness predictive microbiological models | |||||
| New York State Department of Health's Bureau of Community Sanitation and Food Protection (BCSFP)383 | To analyze epidemiologic data and relate incidence of disease to specified food preparation practices. | Not applicable. | Scenarios of major risks in food production processes generated by computer. | Models based on known characteristics of microorganisms. | New York State Department of Health |
| Stepwise and Interactive Evaluation of Food safety by an Expert System (SIEFE)376, 377 | To provide microbiologic quantitative risk assessment for food products and their production processes. | Not applicable. | System provides decision makers with quantitative insights into food production processes and their risk determining factors. | Models based on known characteristics of microorganisms. | Not applicable. |
| U.K. Food Micromodel384 | To allow prediction of organism responses under a variety of conditions. | Not applicable. | Food processing. | Models predicting the behavior of Salmonella, L. monocytogenes, S. aureus, Y. enterocolitica, E. coli O157:H7, B. cereus, B. subtilis, psychotropic strains of C. botulinum, Campylobacter jejuni, C. perfringens and Aeromonas hydrophila. | Not applicable. |
In the U.S., the national system for surveillance of food-borne illnesses is dependent on voluntary reporting from clinicians and laboratories. In addition, the Foodborne Diseases Active Surveillance Network (FoodNet), the principal foodborne-disease component of the CDC's Emerging Infections Program (EIP), is an active system that collects data from clinical laboratories and public health officials to estimate the burden and sources of specific foodborne diseases in the U.S.366-372 FoodNet is a collaborative project among the CDC, the 8 EIP state health department sites, the Food Safety and Inspection Service of the United States Department of Agriculture (USDA), and the Food and Drug Administration (FDA). Results from FoodNet are used to identify infection control points, focus future prevention strategies and decision making within food safety regulatory agencies, measure changes in the burden of disease, and evaluate the effects of interventions on rates of infections over time. FoodNet collects data about 9 foodborne diseases (Campylobacter, Cryptosporidium, Cyclospora, E. coli O157, Listeria, Salmonella, Shigella, Vibrio, and Yersinia) in 8 U.S. sites (California, Connecticut, Georgia, Maryland, Minnesota, New York, Oregon, and Tennessee).
In general, the relatively small number of organisms for which data are collected limits the usefulness of systems that perform surveillance of specific foodborne pathogens. Additionally, the lack of published evaluative information prevents a clear understanding of how these systems would function in the event of a bioterrorist attack on the food supply.
Technologies like SODA and PulseNet have been used extensively in foodborne outbreak investigations with success -- even with investigations of outbreaks resulting from intentional contamination of the food supply. However, none of the foodborne illness surveillance systems that collect disease incidence data was specifically designed for the early detection of bioterrorist attacks on the food supply, nor has any been evaluated for that purpose. We found no evidence regarding the potential sensitivity, specificity, or timeliness of FoodNet, the active surveillance system collecting data to estimate the burden of foodborne illnesses in the U.S. Moreover, even if FoodNet was sufficiently sensitive and timely to be useful for agroterrorism detection, it is limited in that it only collects data from 8 states on 9 foodborne illnesses. The primary means for detecting an agroterrorist attack outside these states or using a different organism would be based on the analysis of voluntary reports from clinicians and laboratories.
On the morning of March 18, 1996, John Major, then Prime Minister of the U.K., was presented with a memo from the Health and Agriculture Ministers confirming a link between bovine spongiform encephalopathy (BSE) in cattle and Creutzfeldt-Jakob disease (CJD) in humans.385 The memo described reports of people contracting CJD by consuming beef infected with BSE.385 Response to BSE in the U.K. has cost the country an estimated £4 billion, caused significant damage to the agricultural industry, and harmed Britain's relations with its European Union partners, who rapidly banned the importation of British beef.385 Similarly, concerns exist that a bioterrorist attack could involve the dissemination of a zoonotic illness among animal populations with the intention of infecting humans or livestock to cause economic and political chaos.386-388
| System name | Purpose | Geographic location | Population or process under surveillance | Data used in surveillance system | Government agency |
|---|---|---|---|---|---|
| Australian National Animal Health Information System (NAHIS)397 | To monitor animals' health status and aid in decision making. | Australia | Animal disease and mortality. | Laboratory data, reports by coordinators, studies. | Australian government |
| California Encephalitis Program390 | To monitor for mosquito-borne viruses such as West Nile Virus, St. Louis encephalitis, and Western equine encephalomyelitis. | California | Viral activity in mosquitoes and chickens. | Mosquitoes are collected and tested for the presence of the viruses and 200 flocks of sentinel chickens are maintained through the state to monitor viruses in the mosquitoes that bite them. | California Department of Health Services |
| EpiMAN-FMD395, 396 and EpiMAN-SF398 | To assist disease control authorities in the containment and eradication of animal disease outbreaks. | New Zealand | Animal disease and mortality. | Laboratory data, information from the field. | Not applicable |
| International Office of Epizootics (OIE) 399, 400 | To collect and disseminate the information gathered by national surveillance programs. Also to alert countries threatened by an epizootic, strengthen international cooperation on the control of animal diseases, and facilitate trade in animals and animal products. | International | Animal disease and mortality. | Reports from participating countries. Studies describing the OIE have noted problems with the timely filing of reports and adequate monitoring on the part of member nations. | OIE |
| National Animal Health Monitoring System (NAHMS)391-394 | To collect data on animal disease incidence and prevalence, mortality, management practices, and disease costs. | U.S. | Animal disease and mortality. | Laboratory data, reports by veterinary practitioners, studies. | USDA |
| National West Nile Virus Surveillance System (ArboNet)389 | To track West Nile virus activity in humans, horses, other mammals, birds and mosquitoes. | U.S. | West Nile virus activity. | Laboratory data. The CDC strongly encourages "real-time" reporting by telephone, electronic mail, fax, or data entry into a Web-based database. | CDC http://www.cdc.gov/ncidod/dvbid/westnile/surv&control.htm. |
The National West Nile Virus Surveillance System (ArboNet)389 is an electronic-based surveillance and reporting system, developed by the CDC to track West Nile virus activity in humans, horses, other mammals, birds, and mosquitoes. The California Encephalitis Program390 monitors mosquitoes and flocks of sentinel chickens for mosquito-borne viruses such as West Nile Virus, St. Louis encephalitis, and Western equine encephalomyelitis. ArboNet and the California Encephalitis Program are the only zoonotic surveillance systems in our Report. Reports of these programs describe the type of data they collect and report the weekly incidence of the diseases under surveillance. They provide no additional information about the sensitivity, specificity, representativeness, acceptability, or other criteria by which we evaluated surveillance systems.
The National Animal Health Monitoring System (NAHMS) is an integrated national (U.S.) surveillance system that collects data on animal disease incidence and prevalence, mortality, management practices, and disease costs.391, 392 One of its surveillance programs is the Veterinary Diagnostic Laboratory Reporting System. Data are compiled from several sources including national animal disease control and eradication programs, information on patterns of disease based on laboratory data, selected etiologic agents, global disease distribution and reports of "unusual" laboratory findings. Results are published quarterly in DxMonitor. Another animal health surveillance program within NAHMS is the Sentinel Feedlot Monitoring program, which relies on health data collected from veterinary practitioners, who report inventory and morbidity by cause. A pilot study, involving a large commercial beef feedlot, was performed in 1988.393 NAHMS also conducts national studies of livestock, which involve processing data obtained from questionnaires, logs, and laboratory analysis of biological specimens.391 These studies include the swine health survey in 1990392 and the 1995 National Swine Study.394 Data from these principally describe animal disease incidence data and related mortality and cost information but not evaluative information about the system itself.
In New Zealand, an epidemiological information system has been developed to assist disease control authorities in the containment and eradication of animal disease outbreaks. The system was initially developed to control a potential incursion of foot and mouth disease, and is called EpiMAN-FMD. By combining disease and viral spread models, farm maps, and epidemiologic parameters, the system provides statistical reports and decision support to help with the management of an outbreak. While the system has not been tested in an actual foot and mouth disease outbreak (or any bioterrorism events), it has been tested in hypothetical scenarios, and the developers hope to adapt EpiMAN-FMD to other veterinary issues.395, 396
None of the reports of the systems in this section described the hardware requirements, the system's security measures, or the costs of operating the system.
Bioterrorism involving zoonotic and animal diseases represents a substantial threat to our national security and economy. Early detection of such an event requires effective rapid detection systems for use by farm workers, meat inspectors, and veterinarians with real-time reporting capabilities to public health officials. The surveillance systems described above provide some organizational support for national and international anti-agroterrorism efforts. However, none has been evaluated for this purpose. Our search found reports of only 2 zoonotic surveillance systems -- a major gap in bioterrorism surveillance efforts. Most of the reports provided little or no information about the timeliness of these systems; those that did suggest lag times that would be too long for effective bioterrorism surveillance.
In this section, we present information about surveillance systems that met our inclusion criteria but collected sufficiently different surveillance data that they could not readily be described in the previous sections. These systems could be valuable additions to surveillance networks that integrate surveillance data from clinicians, hospitals, and laboratories. There are likely to be many other systems collecting data with potential utility for bioterrorism surveillance that have not been described in published reports or fall outside the scope of our primary research focus. For example, by September 2002, the American Association of Poison Control Centers will require all of its approximately 60 member centers to electronically report all call data at the time it is collected to the Toxic Exposure Surveillance System (TESS) database.401 Efforts are currently underway to develop analysis algorithms to systematically search this database for clusters of cases that may represent public health problems.
| System name | Purpose | Geographic location | Population or process under surveillance | Data used in surveillance system | Method of data collection | Sponsoring agency |
|---|---|---|---|---|---|---|
| Data Web407 | To access and analyze demographic, economic, environmental, health, and other data already collected by a variety of organizations in the U.S. | U.S. (piloted in Connecticut) | Any surveillance topic of interest for which data are available in the Data Web databases. | Data Web is a collection of systems and software that provides access to demographic, economic, environmental, health, and other data located in organizations across the U.S. | Users submit queries on Data Web site. Additional developments will include adding functions such as mapping, graphics, and statistics; development of standardized reports to state and local health departments and the CDC; increase in the number of agencies whose data are available on Data Web. | CDC http://www.cdc.gov/programs/research.htm |
| Dialysis Surveillance Network (DSN)408 | To monitor bloodstream and vascular infections at dialysis centers nationwide. | U.S. | Rates of vascular access infections and other bacterial infections for both individual centers and the overall provider population. | Data are collected on the presence or absence of criteria for infections, not infections themselves; a computer algorithm is used to determine whether the data support the case definition of an infection. | Data from voluntary providers can be submitted either through the Web or on paper. Those submitting data electronically can access and print reports at any time, while those submitting data on paper receive a quarterly analysis report. | CDC http://www.cdc.gov/ncidod/hip/DIALYSIS/dsn.htm |
| Disease Early Warning System (DEWS)409, 410 | To detect and predict outbreaks and epidemics in Pakistan. | Pakistan | Outbreaks and epidemics, not otherwise specified. | Case reports and data from a mobile laboratory. | No additional information available. | Pakistan's National Institute of Health |
| EPIFAR402 | To track individual prescription histories in order to provide estimates of disease prevalence. | Italy | All Italian citizens, as members of the National Health Service (NHS). | Individual drug prescription forms, including patient demographic data and drug information. | All drug prescriptions are routinely collected and processed by the NHS. A computer program is used to analyze these data and determine the prevalence of selected diseases. | National Health Service (NHS) |
| Global Public Health Information Network (GPHIN)240 | To monitor reports on communicable diseases and communicable disease syndromes on the Internet. | Global | Reports of communicable diseases and communicable disease syndromes. | Electronic reports available on the World Wide Web in electronic discussion groups, newswires, and elsewhere. | GPHIN's powerful search engines actively crawl the World Wide Web. Searches are in English and French and will eventually expand to all official languages of the WHO. | Health Canada; WHO http://www.cdc.gov/ncidod/eid/vol4no3/heymann.htm |
| HAWK404, 405 | To track reportable diseases in Kansas using an electronic system that follows NEDSS standards. | Kansas | Vaccine-preventable diseases, tuberculosis, and infectious diseases, with the intention of including HIV and STD records through a data warehouse model. | Notifiable disease reports from clinicians and public health agencies. | HAWK is a central, statewide database of reportable diseases that can be accessed remotely by authorized public health officials via the Internet. Remote users can report disease occurrence and generate summary statistics and reports to assist them in evaluating the overall effectiveness of public health efforts in their area. | Kansas Department of Health and Environment |
| Medical Historian403 | To ask a patient questions about their medical history to derive a historical database of patient information that can be queried for the purpose of disease surveillance. | Not applicable | Any disease outbreak. | Patient history information. | The system performs a computerized elicitation of a review of symptoms. "Diseases" are defined as constellations of symptoms. The database is then scanned to search for outbreaks of "diseases" (symptoms clusters). | Not applicable |
| National Surveillance System for Healthcare Workers (NaSH)411, 412 | To allow the CDC to monitor trends, detect emerging occupational hazards, and evaluate prevention policies for infectious disease exposure of health care workers. | U.S. | Infectious disease exposure and infection in health care workers. | Includes health care worker demographics, baseline vaccinations, bloodborne pathogen exposures, and post-exposure prophylaxis. | Health care facilities voluntarily provide data. No other information available. | CDC http://www.cdc.gov/ncidod/hip/SURVEILL/nash.htm |
| Peace Corps Surveillance413 | To monitor common ailments in overseas Peace Corps volunteers. | Global (outside of U.S.) | Common illnesses in Peace Corps volunteers. | Reports from Peace Corps health care facilities. | Study tabulates results of defined group of illnesses in Peace Corps volunteers and groups them into quarterly reports, with data represented as number of cases per 100 volunteers. | Peace Corps |
| Traveler's Diarrhea Network414 | To detect outbreaks of traveler's diarrhea in locations that travelers frequent. | Japan | Traveler's diarrhea. | Reports include names of isolated pathogens, date of onset, and suspected place of infection. | The Infectious Disease Surveillance Center in Tokyo is connected with hospitals and airport quarantine stations by an e-mail network system. Data are reported weekly. | National Institute of Infectious Diseases, Japan |
| Unexplained Deaths and Critical Illnesses Surveillance System238, 312, 415 | To improve the CDC's capacity to rapidly identify the cause of unexplained deaths or critical illnesses and to improve understanding of the causes of specific infectious disease syndromes for which an etiologic agent is frequently not identified. | U.S. | Unexplained deaths and illnesses, including acute pulmonary syndrome and infection of the central nervous system. | Reports and clinical specimens from coroners and medical examiners. | Data collection is Web-based. Active population-based surveillance is conducted in 4 Emerging Infections Program sites with a total population of 7.7 million 1- to 49-year-olds. National and international surveillance efforts are passive for clusters of unexplained deaths and illnesses. | CDC http://www.cdc.gov/ncidod/dbmd/diseaseinfo/unexplaineddeaths_t.htm |
These systems differ with respect to the types of surveillance data they collect and therefore differ with respect to sensitivity, timeliness, and cost. For example, EPIFAR collects drug prescription data, which may be more sensitive for a bioterrorism event than a system like Data Web, which performs surveillance on administrative databases of demographic, economic, environmental, and health data collected by a variety of organizations in the U.S. However, Data Web uses data already collected for other purposes and therefore minimizes collection burden.
Two of these 11 systems have been clinically evaluated. EPIFAR is designed to track individual prescription histories in order to provide estimates of disease prevalence in Italy.402 All drug prescriptions are routinely collected and processed by the Italian National Health Service (NHS). EPIFAR is a computer program used to analyze these data and determine the prevalence of selected diseases. A retrospective study of 2,550 patients who received 1 of 12 drug regimens designed for tuberculosis found that 7 times as many tuberculosis patients were identified by EPIFAR as were officially reported by clinicians.402 However, 88 of 250 total notified cases (35.2 percent) for the study period were not found in the EPIFAR system.402 Also, in a survey of physicians prescribing tuberculosis drugs for the included patients, physicians denied any tuberculosis-related problem for nearly 25 percent of the EPIFAR-identified patients. The positive predictive value of the model differed between drug regimes, with a range of 50 to 76.9 percent.402 To understand whether a system such as EPIFAR is likely to be useful for bioterrorism surveillance, it would be important to have a detailed understanding of the sensitivity, specificity, and timeliness of each of the drugs (or combinations of drugs) under surveillance. For example, if EPIFAR collected surveillance data exclusively for a medication such as isoniazid, which is typically used exclusively for the treatment of mycobacterial diseases, it is likely to have better specificity (albeit perhaps decreased sensitivity) than a combination of medications including those that are used for many diseases. Given the current evidence, it is not clear which drugs (or combination of drugs) should be selected for a bioterrorism surveillance program that attempts to achieve a given degree of sensitivity for all or most of the most worrisome biothreat agents.
Medical Historian electronically asks patients questions about their medical histories to derive a historical database of patient information that can be queried for the purpose of disease surveillance. The system performs a computerized elicitation of a review of symptoms. "Diseases" are defined according to symptom-clusters. The database is then scanned to search for outbreaks of symptom-clusters. In a retrospective evaluation of 288 patients with cough and rhinitis, the computer diagnosis was within the 95 percent confidence interval of expert panel diagnosed-conditions for upper respiratory infection. Sensitivity varied from 27 to 90 percent depending upon the symptom-cluster used. No assessments of specificity were provided. Data on the sensitivity of Medical Historian for 4 other computer-surveyed diseases were not stated, but the authors did note that "the number of encounters identified electronically was always fewer than the number identified by the physician panel."403 It is difficult to interpret these results in terms of surveillance of bioterrorism-related illness.
The HAWK system for surveillance of reportable diseases in Kansas is a promising system that collects notifiable disease reports from clinicians and a variety of public health agencies in the state using a data warehouse model.404, 405 Authorized public health officials can remotely access the data via the Internet to perform analyses. No evaluative data are available about HAWK, which could serve as a model for similar statewide reporting systems.
None of the reports of the systems in this section described the type of hardware required, the system's security measures, or the costs of operating the system.
.
However, there are significantly fewer descriptions of systems
collecting the earliest surveillance data -- a significant gap in the
literature. For example, given that the 53.1 million school-aged
children (aged 5 to 17) represent 18.9 percent of the total U.S.
population (281.4 million) and that their absenteeism is collected on a
daily basis nationally (albeit not always electronically), an evaluation
of the sensitivity, timeliness, and cost of this source of surveillance
data seems warranted.406 The surveillance of work absenteeism rates is complicated by the
decline in the types of industries that require employees to clock in
and out of work and by the lack of these data in an electronic format.
Systems that collect pharmaceutical data, like EPIFAR, hold particular
promise for bioterrorism surveillance. Pharmaceutical data, particularly
over-the-counter medication sales data, could provide an early, if not
specific, indication of an outbreak. Additionally, most pharmaceutical
sales are tracked electronically. The evaluation of EPIFAR emphasizes
the complexity of selecting specific pharmacy data for bioterrorism
surveillance. A careful analysis of the detection characteristics of
common prescription and non-prescription medications used for the
bioterrorism-related syndromes (e.g., antipyretics, cough suppressants,
and antidiarrheals) will be required to determine the utility of these
data for bioterrorism surveillance.It is clearly not the function of surveillance to predict the long-range future, but it is only prudent to anticipate the immediate problems that can be expected on the basis of presently known facts and presently accepted concepts, erroneous though some must be.
-- A. Langmuir, 1963236
Once surveillance data are collected, analysis -- typically by public health officials -- is required to identify patterns suggestive of bioterrorism-related illness. Our search identified a number of methods for analysis of surveillance data. We note however, that we have not conducted a systematic review of surveillance analysis methods per se but have identified methods that have been or could be applied to the analysis of bioterrorism-related illnesses. In this section, we describe the important features of analytic methods for bioterrorism surveillance data.
The primary analytic question in prospective disease surveillance is whether the currently observed disease process differs from the expected process. Before answering this question, the analyst must determine what elements of the data will be modeled: individual attributes (e.g., gender, age), time, and/or space. Additionally, the analyst must consider how the expected process (i.e., the baseline) will be modeled and what threshold will be used to establish a significant change from baseline (i.e., an aberrant disease process). Traditionally, time has been the major modeling consideration,416 but recent developments in technology and statistics have greatly facilitated the consideration of spatial information in surveillance analyses. We first examine modeling decisions concerned with generating the expected disease process, and then discuss developments in spatial analysis and their implications for bioterrorism detection.
Models of the expected disease process are typically derived from the historical pattern of disease, either over the immediately preceding time interval (e.g., 30-day moving average), or over one or more historically corresponding intervals (e.g., the mean rate for the first week in January over the past 5 years).417, 418 This approach is straightforward, but it ignores the underlying dynamics of disease propagation through the population. Several articles identified in our search describe methods for stochastically modeling the spread of communicable disease epidemics.419-423 These methods have not been widely used for the modeling of disease surveillance, but they may allow more accurate determination of the expected disease rates and deviations from the expected.
More accurate descriptions of the expected and observed disease rates should enable more accurate identification and characterization of data aberrations and disease outbreaks. Because the need for timeliness is particularly acute in bioterrorism surveillance, application of these methods to disease surveillance merits further attention. It should also be possible to apply similar methods to model the disease processes associated with outbreaks of non-communicable diseases of bioterrorism, such as anthrax. Potential drawbacks of this modeling approach include its added complexity and the need to collect or estimate data for a number of parameters, such as the proportion of susceptible individuals, herd immunity, and mixing rate.
The importance of spatial location for understanding disease processes has been appreciated for many years. Spatial location has a central role in the epidemiological triad of "person, place and time," and maps have long been used to identify important patterns in diseases such as cholera, cancer, leprosy, pneumonia, and smallpox.252 Consideration of the spatial dimension of surveillance data may enable more timely identification and characterization of important patterns. Recent developments in IT and statistics, most notably the development of user-friendly geographic information system (GIS) technology, can facilitate consideration of space in surveillance data; however, the effective application of these developments to surveillance analysis is not yet well established.424 In this section, we examine how GIS technology and other spatial analysis tools have been applied to bioterrorism-related diseases.
A GIS is an automated system for collecting, organizing, analyzing and presenting geographically referenced data. Beyond simple mapping, a GIS can perform complex functions such as automated address matching, distance calculations, buffer analyses (i.e., calculation of a buffer zone of variable width around a point, line, or area), spatial queries (i.e., the ability to select observations based on their geographic characteristics), and linking data sources by spatial location.425 The primary effort to develop global mapping capability for public health data has come from the joint WHO/UNICEF program HealthMap -- a data management, mapping and GIS system for public health. The program was initially created in 1993 to establish a GIS to support management and monitoring of the Guinea Worm Eradication Programme.426 Since 1995, however, in response to the increasing demand for mapping and GIS technologies from a much wider range of public health administrators, the scope of the work has been broadened to include the promotion and use of GIS technology for other disease control programs.426
Most applications of GIS that we identified in our review performed retrospective analyses to identify spatial patterns of disease and/or disease vectors. However, several of the syndromal surveillance systems described in this Report (such as ESSENCE and RSVP®) use GIS or spatial analysis in an ongoing and systematic manner to identify disease outbreaks or data aberrations. Some studies used Global Positioning System technology to locate disease cases.427, 428 Analytic methods employed within a GIS environment ranged from simple, descriptive spatial analyses,429-431 to more complex spatial432, 433 and space-time modeling.427, 428 The main objective of the analyses was usually to characterize the relationship between disease vectors and cases. In some instances, this relationship was used to forecast future disease outbreaks based on predicted disease vector ecology.433-436
The studies presented, with their focus on predicting new outbreaks from previous outbreaks and disease vector distributions, may seem unrelated to bioterrorism surveillance. The lack of previous large-scale bioterrorism attacks, and the fact that most agents of bioterrorism are not vector-borne, prevent this type of predictive modeling for bioterrorism. However, a number of the specific technologies and GIS/spatial analytic methods employed in these studies may be directly applicable to the analysis of bioterrorism surveillance data. These methods and technologies include remote sensing, data integration, spatial interpolation, and space-time statistical analysis. For example, remote sensing data may help to identify potential effects of meteorological conditions for airborne dispersal of a bioterrorism agent. The ability of a GIS to integrate disparate data sources (e.g., pharmacy sales data, ambulance activity, and ICD9 codes) according to their spatial location may enable the identification of important relationships that indicate early disease behavior. Spatial interpolation could be used to predict risk in locations for which similar data are not available. Finally, advanced space-time analytic methods can take advantage of the spatial dimension of data to detect aberrations with greater sensitivity and timeliness. The combination of these analytic methods with rich descriptions of the expected disease process (as described previously) may provide even greater sensitivity for identifying bioterrorism attacks in the midst of normally occurring disease outbreaks. We note that no published report has specifically evaluated whether a surveillance system that uses both temporal and spatial analyses is likely to be more timely or sensitive than a system that performs only temporal analyses.
On March 20, 1995, the Aum Shinrikyo sarin attack on the Tokyo subway system resulted in 7 deaths and the medical treatment of an additional 250 people.10 A group of physicians in Matsumoto, Japan, who had treated victims of a sarin release by the Aum Shinrikyo a year before, heard about the attack, and sent information about typical case presentations to Tokyo hospitals and the Ministry of Health and Welfare.10 Although this information was reportedly helpful,10 a more systematic approach to the dissemination of medical and intelligence information could have benefited clinicians, public health officials, and victims. In particular, public health officials could have used intelligence information from law enforcement officers regarding the increased likelihood of such an attack and disseminated medical information about the clinical presentation and therapeutics for nerve agents before such an attack occurred.10
Source: Modified, with permission from SH Cody, Santa Clara County Department of Public Health243
Note: in some states, the state health department (not the local health department) has primary responsibility over communication of public health information.
). During October and November 2001, the dissemination
of information about confirmed and suspected cases of anthrax in the U.S.
and methods for its detection, laboratory testing, treatment, and
prophylaxis tested this pathway. The media reports of anthrax cases created
a demand for information that exceeded the capacity of national, state, and
local health departments. The information on laboratory protocols, reagents,
and training available before October 2001 was not sufficiently detailed to
meet the information needs of first responders, clinicians, and others.437 Just as had been predicted by the Dark Winter Exercise, asynchrony in
the provision of information between the media and public health officials
contributed to the perception that the public health officials had lost
control of the situation.11
After the anthrax cases of late 2001, the National Association of Counties conducted a telephone survey of county public health directors. Completed surveys were obtained from 300 of the 946 county public health directors.438 Thirty-five percent of county public health directors indicated that insufficient communication networks were considered obstacles to their health department's ability to respond to a bioterrorism or chemical warfare crisis.438
The discrepancy in reporting time of public health information between the media and public health officials derives principally from the different missions of these organizations. Whereas it is the media's duty to collect, verify, and report news as it becomes available, public health officials' primary obligation is to ensure the public health, which often requires time-consuming analyses of available data and consideration of appropriate responses before notification of the public. For international news organizations to be able to report on a 24-hour news cycle, they must be supported with sophisticated information systems that facilitate the management of their data. As evidenced by the findings in this section, public health officials typically do not have similarly robust communication infrastructures at their disposal.
Alarmed by news of the anthrax cases and contaminated mail, patients sought information about biothreat agents from credible sources such as local public health departments as well as from their personal physicians. Efforts to respond to calls and faxes from concerned patients and clinicians stretched the limited resources of public health departments. Similarly, emergency departments, urgent care clinics, and clinicians' offices strained to meet the information needs of the "worried well."
Reporting and communication systems that facilitate the secure delivery of information from public health officials to the public and from clinicians to their patients could have helped to dispel the perception of chaos and inundation. Specifically, if clinicians had had an effective means of communicating electronically with their patients, they might have been able to provide reassurance without necessitating office visits and phone calls. Similarly, they could use such a system to notify patients of a suspected bioterrorist attack, describe the characteristic signs and symptoms, and disseminate criteria for seeking medical attention. In a recent survey, 13 percent of doctors responded that they e-mail their patients.439 This low number may be related to the belief, held by some physicians, that electronic communication carries hazards of its own. Barriers to implementation of electronic communication between physicians and patients include physician concerns that this additional mode of communication will add to already busy practices,439 that e-mails from patients regarding matters requiring urgent attention that will be missed due to delays in checking e-mail,439 and that certain e-mail systems will not comply with HIPAA.
On September 11, 2001, as in other times of crisis, the volume of calls from around the world into the affected areas exploded. Local telephone systems overloaded and played a standard message saying there was no phone access to that zone -- a message that, in its lack of specificity about what was happening in the area, may have served to increase the callers' apprehension.172 Additionally, much of the communication infrastructure in the affected area of New York City was physically interrupted by the loss of electric power, cables, servers, and radio transmitters. One lesson that can be learned from these experiences is that, after a bioterrorist attack -- particularly if it is combined with other acts of terrorism such as a physical, radiologic, or chemical attacks -- local phone, fax, and phone-modems may be unavailable. Clinical information systems that rely on these modes of communication (e.g., laboratory data about patients and surveillance alerts from public health officials to clinicians) may be affected. Under these circumstances, access to the Internet will be limited to wireless connections or cable television.172 Similarly, e-mail will not be available via the phone lines but only via Web mail.
Security is one of the most critical features of a communication system for bioterrorism. The 3 main types of security concerns for these systems are: (1) maintaining patient confidentiality by ensuring that the information is disclosed only to authorized persons (i.e., this issue is addressed by HIPAA regulations and by systems that use role-based access to information); (2) maintaining the accuracy and completeness of the data (i.e., preventing unintended changes to the original data that would compromise subsequent analyses and conclusions); and (3) maintaining the availability of the system so that it is functional when it is needed (i.e., preventing system overloads so that it will be useful to responders in the event of a bioterrorist attack). Security violations can disturb all 3 of these elements of a system. In the event of a bioterrorist attack, it is possible that there may also be cyberterrorist attacks on information and communication resources. Alternatively, systems may simply overload if demand for access exceeds capacity. Communication systems for bioterrorism must include adequate redundancy to avoid overloads, as well as security measures to prevent and respond to cyberattack.
We identified 26 ITs that could be used to support the reporting and communication needs of decision makers during a bioterrorism event, 7 of which have been described in a peer-reviewed evaluation report (Tables 3 and 24; Appendix H).
The systems vary with respect to their purposes: 8 for communication among public health officials at local, state and federal levels; 2 for communication among public health officials, clinicians, and the public; 4 for communication between patients and clinicians; 5 for the automated communication of information from electronic medical records of patients to clinicians; 1 for communication from professional clinician organizations to clinicians; 3 for communication between emergency departments and first-line emergency response personnel; and 3 for other kinds of communication.
The technologies that have been subjected to the most evaluations are those that send an alert to a clinician based on a worrisome finding in the patient's medical record (typically a laboratory test result, radiologic finding, or medication error). All 5 of the systems in this category have been clinically evaluated, typically for outcomes related to clinician acceptability, time to respond to alerts, or changes in numbers of medication errors. Although clinicians reported being annoyed by erroneous pages or alerts, in general these systems tended to decrease the interval between a laboratory result becoming available and action being taken by the clinician. If these systems increase the speed with which clinicians receive new information that may affect their management decisions, they may provide a useful tool in the event of a bioterrorist attack.
The Program for Monitoring Emerging Infectious Diseases (ProMED©)440-442 is an independent system that was specifically designed by the Federation of American scientists to provide early warning of possible bioterrorism events caused by infectious agents or toxins, including agroterrorism. Reports of outbreaks of human, animal, and crop diseases from around the world are screened and assessed for quality before being distributed (24 hours a day, 7 days a week) to 24,000 subscribers in over 150 countries. These subscribers include clinicians, public health officials, first responders, veterinarians, bioterrorism and agroterrorism experts, members of the CIA and FBI, school teachers and their pupils, and newswire services and newspapers.172 ProMED© is both a surveillance and communication system that increasingly serves as a model for the development of similar systems within individual countries.172 In an internal study, ProMED© provided more timely and more numerous outbreak alerts for emerging diseases and toxins as compared to the WHO, CDC and Cable News Network (CNN).172 For example, ProMED© reported cholera outbreaks in 11 countries in 1999 between 3 days to 11 weeks earlier than the WHO.441 Over 97 percent (342 out of 351) of ProMED©'s alerts for a 7-month period were subsequently confirmed by official sources (including 6 official reports that were later retracted).441
Whereas a system like ProMED© reports the same information about a potential disease outbreak to all subscribers, other types of systems are required to disseminate secure outbreak information to limited members of the public health community. Specifically, the many levels of the U.S. public health system (local, state, and national) need an integrated communication system to which all officials have access. The Health Alert Network (HAN)443, 444 is a secure Web-based information and communication system designed by the CDC to link local and state public health agencies with each other and with other organizations responsible for responding to a bioterrorism attack. Public health officials and first responders can access a variety of useful data, including disease reports, response plans, and management guidelines. In addition, the nationwide system includes early warning broadcast alert and distance-learning functions. Since September 11, 2001, HAN has distributed multiple health alerts via e-mail and fax about bioterrorism preparedness.
| System name | Method of information transfer | Description | Active/Passive on the part of the recipient | Evaluation data | Contact information |
|---|---|---|---|---|---|
| Among public health officials at local, state and federal levels | |||||
| Computerized Information System for Infectious Diseases (CISID)451 | E-mail, Web-based information. | For the collection, management, and analysis of infectious disease data in Europe (e.g., tuberculosis). Features of the system include: program specific data entry modules, public access to online query facilities, Excel download of retrieved information, auto-generation of program-specific indicators, data feedback and progress tables showing key surveillance indicators and compiled country profiles, e-mailing of reports, dynamic maps, multi-lingual capability. Currently used by EuroTB (see Table 16). | Both. | None available. | Department of Infectious Diseases; Informatics Support Unit of the WHO Regional Office for Europe http://www.who.dk/id/facmp.html |
| Electronic Foodborne Outbreak Reporting System (EFORS)452 | Web-based. | Web-based reporting of foodborne diseases and outbreaks by state health departments to the CDC. | Passive. | None available. | CDC http://www.cdc.gov/ncidod/dbmd/outbreak/report_o.htm |
| Emerging Infections Network (EIN)453, 454 | E-mail and Web site; postings are screened. | A forum for reporting, discussion, and dissemination of information regarding unusual infectious disease cases. | Both. | None available. | Asian Pacific Economic Cooperation (APEC) member states |
| Epidemic Information Exchange (Epi-X)455 | Web site; secure e-mail; on-line database; tele-communication capabilities; telephone and pager auto-activation. | Epi-X provides communication and information functions for use in both routine and emergency public health issues. In addition to receiving e-mail, telephone and/or pager alerts, on-line users can create reports and review information on earlier outbreaks and unusual events. | Primarily passive, because of daily e-mail messages and alerts; also active, due to on-line features. | None available. | CDC http://www.cdc.gov/programs/research5.htm |
| Health Alert Network (HAN)443, 444 | Web site; secure e-mail. | HAN is a secure Web-based information and communication system designed by the CDC to link local and state public health agencies with each other and with other organizations responsible for responding to a bioterrorism attack. Public health officials and first responders can quickly access a variety of useful data, including disease reports, response plans, and management practice guidelines. In addition, the nationwide system includes early warning broadcast alert and distance-learning functions. | Primarily passive, but also active because of Web-based information features. | None available. | CDC http://www.phppo.cdc.gov/han/ |
| HEALTHCOM279, 456 | E-mail. | HEALTHCOM is the Web-based communication system connecting over 4,000 computers of the 58 county health departments of New York State. It enables reciprocal Web-based communications between county and state health departments. The system supports health-related data submissions such as electronic birth certificates and child blood-lead reporting. | Both. | None available. | New York State Department of Public Health |
| Information Network for Public Health Officials (INPHO)457, 458 | Web site; local- and wide-area networks; secure e-mail; emergency electronic alerts. | CDC's INPHO facilitates communication and information exchange between public health practitioners across the U.S. In particular, it is designed to improve the rapid collection of health data and its transformation into useful trends and statistics, which can then be shared across many domains. Public health calendars, health information and CDC guidelines are available through its Web site. | Both. | None available. | CDC |
| Rapid Health Electronic Alert, Communication, and Training system (RHEACT)303, 445, 446 | RHEACT is a secure Web portal that enables alert notification, knowledge sharing and training. Collaborative workspaces have been established for: Emergency Response and Planning, Epidemiology and Investigation, Biologic Lab, Chemical Lab, Distance Learning and Local Assistance and Local Public Health Jurisdictions. Each of the workgroups utilizes the tools for document control that allows posting, editing, clearance processes and subscribing to documents. RHEACTS provides the ability to connect and communicate via telephone, cell phone, pager, Web browser or via traditional e-mail. The features include voice command, text-to-speech, and teleconferencing. The security is administered by organization by role to allow the transition of individuals into roles and for individuals to have multiple roles. | Both. | None available. | California Department of Health Services. Project | |
| Among public health officials, clinicians, and the public | |||||
| CDC WONDER/PC 459-461 | Web site. | CDC WONDER/PC is an integrated information and communication service that allows access to public health data from the CDC and other sources. Among other things, the software allows one to query a number of databases maintained by the CDC, communicate with public health officials and other WONDER/PC users, and search MMWR. In addition, data from codebooks such as the Health Interview Survey and the National Hospital Discharge Survey are available for use in surveys. WONDER also allows users to post notices, general announcements, data files, or software programs that may be of interest to public health professionals. | Active. For instance, in order to query the databases, the user must first put together a data request using the WONDER request screen. The request is usually processed in a few minutes, allowing the raw data to be downloaded by the user into a text or spreadsheet file. | None available. | CDC http://wonder.cdc.gov/ |
| Program for Monitoring Emerging Infectious Diseases (ProMED)440-442 | Web site; e-mail alerts, which are transferred to remote locations using phone, satellite and ground radio links. | ProMED is an early warning system for emerging infectious diseases and toxins, including agroterrorism. It is unique in that it reports disease outbreaks in humans, plants, and animals, unlike other surveillance systems. Operated by the International Society for Infectious Diseases, it disseminates information on outbreaks to a global network of public health officials and health care workers. All sources of information are included on ProMED, including official reports and subscriber observations. Reports are screened and assessed for quality standards by infectious disease specialists before being distributed. ProMED-mail currently reaches about 24,000 subscribers from at least 150 countries. Sublists are produced in Spanish, Portuguese, Chinese and Japanese; more languages may be offered in the future. Both Brazil and the Netherlands plan to launch similar services in the near future. There are no subscription costs for using this service. Annual budget to run the program $500, 000. | Primarily passive, due to e-mailed alerts. However, all e-mailed materials are also posted on the ProMED Web site, thereby providing an active component. | In an internal study, ProMED-mail provided more timely and more numerous outbreak alerts for emerging diseases and toxins as compared to WHO, CDC, and CNN.172 For instance, ProMED reported cholera outbreaks in 11 countries in 1999 between 3 days to 11 weeks earlier than WHO. Over 97% (342 out of 351) of ProMED's alerts for a 7-month period were subsequently confirmed by official sources (including 6 official reports that were later retracted).441 | http://www.promedmail.org |
| Between patients and clinicians | |||||
| Comprehensive Health Enhancement Support System (CHESS)462, 463 | Web-linked computer system. | CHESS is an interactive information system for people addressing health-related challenges. It is user-friendly and includes access to a broad range of medical literature, answers to commonly asked questions, references to providers, and information on available social services. In addition, users can submit questions anonymously to experts via a Web connection and receive a response within 24 to 48 hours. Users can also participate in Discussion Groups, which anonymously link them to other patients. Programs have been developed in 5 specific topic areas: Academic Crisis; Adult Children of Alcoholics; AIDS/HIV Infection; Breast Cancer; and Sexual Assault. | Primarily active, in terms of patient use; also passive, due to communication function. | A controlled study of 132 HIV-positive subjects found that CHESS was used an average of 20 times per week, for a total of 138 minutes, by the experimental group. CHESS use was not correlated with education.462 A separate study of 21 patients with either breast cancer or AIDS/HIV infections found that it was used frequently and perceived as useful. The most often-used function was communications. 463 | CHESS Project Center for Health Systems Research and Analysis 1120 WARF Building 610 Walnut St Madison, WI 53705-2397 608-263-0492 Attn: CHESS Project |
| ComputerLink 464 | Networked computer programs. | Patients with AIDS or other chronic illnesses can use ComputerLink from their home to access nurse-supervised information, decision support and communication services. Specialized functions of the system include the Electronic Encyclopedia, with over 200 pages of disease-specific information, and peer discussion groups. In addition, patients can anonymously e-mail questions to a nurse, whose response can be viewed by all program users. | Both. | A controlled study of 51 AIDS patients living at home showed that experimental subjects (n=26) used ComputerLink an average of 297 times over a 6-month period, with an average log-on of 12.5 minutes. The most frequently used function was communications.464 | School of Nursing K6/346 CSC 600 Highland Avenue Madison, WI 53792 |
| Healinx™465 | Secure e-mail communications; other Web-based interactions. | Patients and clinicians register as Healinx™ users on the Healinx™ Web site. They may then send secure messages in order to ask/answer clinical questions, request prescriptions and renewals, schedule appointments, and request referrals to specialists. System is HIPAA-compliant. In addition to secure messaging, Healinx™ provides structured interviews on a variety of clinical topics (e.g., fever, cold and flu symptoms, and rash) called webVisits™. These webVisits™ guide patients through an interactive questioning process and send the patient responses to the clinician in a structured message (i.e., chief complaint, history of present illness, review of symptoms, etc.). The clinician, or any member of his or her staff, can triage or respond to webVisit™ communications just as they would for office visits or telephone calls. Clinicians can also generate new prescriptions using Healinx™. Each prescription is automatically screened for interactions and checked for compliance with the patient's formulary. Healinx™ also routes each prescription to the patient's pharmacy of choice, eliminating the need for pharmacy call-ins and callbacks. | Passive for physician. | None available. | Healinx™ 950 Marina Village Parkway, Suite 100 Alameda, CA 94501 http://www.healinx.com |
| Your Practice Online466 | Secure e-mail communications; individual physician practice Web sites. | Your Practice Online allows physicians to build a customized Web site, with both information and communication capabilities. Through secure messaging, approved patients can send and receive e-mails through the physician's Web site. Patient messages are categorized as appointment requests, prescription refill requests or general messages and questions. Each Web site also provides access to Medem™'s Medical Library, a comprehensive source of peer-reviewed medical literature. | Both. | None available. | Medem, Inc. 877-926-3336 http://www.medem.com |
| From electronic medical records of patients to clinicians | |||||
| Automated Late-Arriving Results Monitoring System (ALARMS)467 | Daily electronic log. | Developed for use in emergency departments, ALARMS monitors hospital laboratory and emergency department registration databases for late-arriving abnormal lab results. | Passive. | By generating a daily log of all relevant cases, it may improve documented follow-up of late-arriving lab results. As of 2000, the system was developed but not implemented. | Harvard Children's Hospital, Division of Emergency Medicine and Informatics Program, Boston, MA |
| Brigham Integrated Computing System (BICS)468, 469 | Electronic medical record- and pager-based alerts in the event of a critical laboratory value or medication error. | As new data enter BICS (e.g., new laboratory results or medication orders), a copy is sent to an inference engine. The engine examines a knowledge base to determine if any of the clinical alarm rules have become true as a result of the new data. If 1 or more rules are true as a result of the new data, the physician on record for the patient is paged. If the page is not acknowledged within 15 minutes, the borders of the workstations on the patient's floor turn red. If the alert has not been viewed for more than 30 minutes, an operator calls the nurses' station on the patient floor and reads a staff member the alert. | Passive. | The system was evaluated over a 6-month period. For 122 of 1852 (6.6%) alerts, no physician or the wrong physician was paged. It was corrected in 18 of these cases. Of the 1,730 alerts sent out, physicians acknowledged 1,214 (70.2%). Physicians reported that they would take an action as a result of the alert approximately 70% of time that they acknowledged one.468 In a separate study that involved 192 alerting situations (94 experimental, 98 controls), the use of BICS resulted in a 38% shorter median time interval between when a critical laboratory result was available and when appropriate treatment was ordered (1.0 vs 1.6 hours, p = 0.003; mean, 4.1 vs 4.6 hours, p = 0.11).469 | Brigham and Women's Hospital, Boston, MA |
| Clinical Event Manager (CEM)470 | Either text pager or e-mail. | CEM is a program that monitors patient data en route to existing clinical repositories, examines that data for critical items (i.e. an abnormal lab value), generates a message alerting the clinician of the new information, and delivers that message to the clinician electronically. CEM sends the message to either a text pager or to an e-mail account. Clinicians subscribe to CEM and indicate which of the 400 available alerts they wish to receive, and which format (text pager vs. e-mail) they prefer. | Passive | A 90-day study involving 99 participants at 2 hospital sites reported that 95% percent of survey respondents (60/63) felt the alerts were helpful most of the time, especially for microbiological data. 71% of the participants wanted to keep using the service after the study was completed. However, the pages were occasionally found annoying by almost half of the participants. In addition, participants favored receiving patient data from the hospital computer system over pager and e-mail. These preferences were due to both user-friendliness and comprehensiveness of the data.470 | Mysis Healthcare Systems, Inc. http://www.mysishealthcare.com |
| Computerized Lab Alert System (CLAS)471 | Daily computer alerts. | CLAS monitors pathologic and missing lab results for hospital patients on a daily basis. It is part of a larger electronic medical records program known as CLICKS (Clinical Records). Alert messages generated by the system do not disappear until the physician either repeats the lab test or otherwise follows up on the issue raised in the alert. | Passive | A 33-day study on a psychogeriatric ward found that over 21% of the CLAS alerts generated (181/864) led to repeat lab tests or initiation of a treatment decision. However, direct questioning of physicians a few hours after their documented review of CLAS alerts demonstrated that they were unaware of half of the day's messages. All messages not addressed were related to non-urgent chronic physical ailments, thereby indicating that physicians may have intentionally ignored such alerts in anticipation of later followup. 471 | Not available |
| Minnesota Microbiology Information System472 | Web-based program. | This system provides Web-based access to in-patient microbiology results. It includes summarized information regarding a patient's current or past lab results. Data are grouped into 2 commonly used clinician categories, "Summary of cultured organisms" and "Custom ravelers' id display." The program also links cost, dosing and use information for antimicrobial agents to results regarding antimicrobial susceptibility. | Active. | An unblended cross-over study involving 16 health care workers and 2 cases of real patient data found that the experimental system was more user-friendly, quicker, and was associated with fewer major and minor user data retrieval errors than a conventional display system. 472 | Departments of Laboratory Medicine and Pathology and Medicine University of Minnesota, Minneapolis, MN |
| From professional clinician organizations to clinicians | |||||
| National Guideline Clearing-house™ (NGC)473 | Web site; e-mail notifications to registered users. | The NGC is a public resource for evidence-based clinical practice guidelines. It is sponsored by AHRQ in partnership with the American Medical Association and the American Association of Health Plans. It is a clearinghouse of clinical practice guidelines on wide-ranging topics in clinical medicine. In response to recent anthrax cases, the NGC has posted guidelines on topics such as anthrax, botulinum toxin, and plague as biological weapons (medical and public health management); use of anthrax vaccine in the U. S.; and recommendations of the Advisory Committee on Immunization Practices. | Primarily active, since interested individuals can browse the NGC Web site for relevant guidelines. However, it also has a passive component, as users may register to receive e-mail notification of new guidelines. | None available. | http://www.guidelines.gov |
| Between emergency departments and first-line emergency response personnel | |||||
| EMSystem™/EMSurvey474 | Web-based system. | EMSystem™/EMSurvey are products designed to provide custom communication, benchmarking, and surveillance technology to emergency departments and Emergency Response Services. The Web-based communication system can integrate emergency care sites and monitor important information about use of services, types of cases seen, and other relevant public health data. | Passive. | None available. | 888-290-6710 info@emsystem.com http://www.EMSystem.com |
| Motorola Emergency Medical Communications System475 | Radio signals. | The Motorola Emergency Medical Communications System is a wide-area radio communications network designed to enhance the delivery of emergency medical assistance to the public. It requires a control center and dispatchers who can track ambulances and other emergency service vehicles throughout a metropolitan area, allowing efficient and monitored dispatching and effective surveillance of emergency situations. Ambulance personnel are notified via 2-way radios or alphanumeric pagers. It operates as part of the 911 emergency calling system. | Passive. | None available. | Motorola Land Mobile Products Sector 800-247-2346 |
| Rapid Emergency Digital Data Information Network (ReddiNet®)20 | Microwave radio technology. | ReddiNet® is a computerized microwave radio technology that allows hospitals and Emergency Medical Services personnel to communicate with each other. A centralized authority sends out alerts, updates, and treatment recommendations in the event of an outbreak. It is most useful for pre-hospital triage and dissemination of key information in the event of a known outbreak, but could also be used as a data source of early detection. | Passive. | None available. | California Emergency Medical Services Authority, Disaster Medical Services Division http://www.reddinet.com |
| Other | |||||
| ePocrates Rx™/ePocratesID™476 | Software and updates can be downloaded from the Internet to a Personal Digital Assistant (PDA); also e-mail updates. | EPocrates Rx™ is a drug information program for use on handheld devices by clinicians (similar to the antibiotic recommendation programs in the Management and Prevention section of this Report). More than 500,000 users have downloaded the application onto their handheld devices. It includes drug dosing and toxicity information and can be linked to an associated program, ePocrates ID™, which provides antimicrobial recommendations according to the type of suspected infection. EPocrates Rx™ sends e-mail notices of updates to its software to users. Developers are working on a template that would enable clinicians to report notifiable diseases or drug reactions to public health officials. | Passive. | None available. | EPocrates, Inc. 120 Industrial Road San Carlos, CA 94070 650-592-7900 http://www.epocrates.com |
| Indianapolis Network for Patient Care (INPC)477 | Electronic system for sharing laboratory data from all Indianapolis hospitals. | Investigators at the Regenstrief Institute at the Indiana University Hospital, in collaboration with all 5 major Indianapolis hospital systems, have created a citywide electronic medical record called the Indianapolis Network for Patient Care (INPC). At the time of an emergency room visit, with a patient's permission, information from each of the separate hospital data vaults is presented to the attending physician in a single virtual record. The data from this citywide laboratory data repository are automatically searched for reportable communicable diseases, and alerts are sent to the Indiana State Department of Health. Early trials suggest that automated reporting delivers signals about new outbreaks faster and more comprehensively at very low marginal cost. | Passive. | Evaluation currently in progress. | Regenstrief Institute, Indiana University Hospital |
| Medcast478 | Daily Web-based transfer of material onto physician's hard drive. | Medcast is a commercial information service for practicing physicians. Five nights a week, current medical news stories are summarized and formatted for delivery to the physician's office. Each night, 4.5 MB of compressed information in text, audio, and graphic formats are transferred via modem to the physician's hard drive. Once a week, a larger transfer of data updates the information sent on a daily basis. As of 1999, the service was offered at no charge to practicing physicians. | Passive. | An independent survey found that 90% (66 out of 73) of physician users believed that use of Medcast enhanced their practice. In addition, 70% (51 out of 73) of the respondents considered the Medcast system "very easy to use."478 | WebMD, Inc. http://www.medcast.com |
Two additional systems, currently under development, show particular promise for serving the communication needs of clinicians and public health officers in the event of a bioterrorist attack: the Global Disaster Information Network (GDIN) and the Urban Security Initiative. GDIN is an interagency disaster system for information and decision support being developed to reduce the impact of disasters by integrating relevant information from all sources and making it available to decision makers and the public.447 GDIN was piloted in hurricane simulation exercises in Florida and was reported to be "extremely useful;" however, no additional details about its functions or evaluations are publicly available.447 The Urban Security Initiative is under development at the Los Alamos National Laboratory.448 It will create centralized computer systems to help cities respond to emergency situations, including chemical and biological attacks.448 The objective of the Urban Security Initiative is to develop an Internet-based Web environment that allows multiple organizations to solve collective problems. Its pilot project is a Web-based emergency planning effort in which 20 different agencies are working together for earthquake preparedness in the Los Angeles area. The disaster-preparedness Web environment consists of 3 components: (1) detailed scenario data from earthquake simulations, along with damage estimates from different potential earthquakes; (2) information about the mission and capabilities of each of the 20 involved agencies; and (3) an interactive area where each agency can sort disaster planning issues according to importance, order necessary actions in time sequence, and request resources.
In the event of a bioterrorist attack, it is essential that clinicians, public health officials, and other users of public health information have access to information that is easy to find, current, and correct. We found many physician groups, professional organizations, and news services that send e-mail notifications of news stories, articles, and clinical updates to their members; however, we have no evidence about the quality of the information or how it is used. One example is a Web site developed by researchers in the Center for Disaster Preparedness at the University of Alabama at Birmingham with sponsorship from AHRQ that offers 5 online courses on the clinical features of bioterrorism-related illnesses with free continuing education credits to 315,000 clinicians.449
The CDC and WHO Web sites satisfy some of the information needs of public health officials and clinicians; however, these organizations have limited resources to devote to immediate, real-time dissemination of information regarding an outbreak resulting from bioterrorism. The CDC regularly holds press conferences and issues press releases and other media alerts on outbreaks. It maintains a Web site, accessible from the CDC home page, which provides updates on various biothreats and ongoing investigations.450 It incorporates a Media Relations tab, which contains updated press kits and press releases, an archive of press releases, and telebriefing access.
In terms of security measures, the systems vary greatly. Many of the systems described being "HIPAA-complaint" without additionally specifying the measures to maintain patient confidentiality. No system described measures to prevent attack from cyber terrorists or to maintain adequate capacity in the event of a surge in demand.
The 26 systems described in this section represent only those for which peer-reviewed analyses, government reports, or Web-based information was available. There are undoubtedly many similar systems for which published data were unavailable. In particular, the development and implementation of systems for use by public health officials is likely to occur without publication of reports since these officials' responsibilities do not typically include preparation and publication of journal articles.
Broken arrows indicate those relationships that are not currently supported by an IT or surveillance system.
An * indicates that an information system described in this section facilitates communication between the noted parties.
An S indicates that a surveillance system described in a previous section communicates information between the noted parties.
, we mapped each of the
included reporting and communication systems to each communication need
identified in Figure 11 (and
described in our Task list, Table 2). Arrows marked
with an asterisk indicate that an IT described in this section transmits
information between the noted parties. Arrows marked with an
"S" indicate that a surveillance system described in a previous
section transmits information between the noted parties. Broken arrows
indicate those relationships that are not currently supported by a specific
IT or surveillance system included in this Report.
The systems with the most evidence for effectiveness are the alert systems that notify clinicians of abnormal findings in their patient's electronic medical records. Although these systems are limited to institutions with electronic medical records, they could play an important role in improving the timely recognition of disease associated with a biothreat agent.
As was demonstrated in the Dark Winter exercise, formal communication systems between members of the media and public health officials are lacking.11 The media page of the CDC's Web site is a passive means of communication between public health and the media. ProMED© has subscribers from all of the groups depicted in the communication diagram, including the media, and currently serves as an independent source of bioterrorism and agroterrorism information to all groups with a need to know. ProMED© is a recognized leader in the international effort to rapidly report and disseminate information on a wide range of biothreat agents. However, information from ProMED© to the media does not serve the same purpose as a system specifically designed for public health officials to communicate information about outbreaks to the public. This represents a major gap in the currently available technologies.
We found 8 separate systems that link various members of the public health community. There are ongoing efforts to integrate communication of public health information vertically and horizontally within the U.S. public health system.
In the event of a bioterrorism event, clinicians must be able to rapidly communicate with their patients. Systems exist that enable Web-based communications between these parties in a HIPAA compliant manner. However, their utility in crisis situations may likely remain limited unless their use for routine communications increases. Robust security measures will be a necessary component of any bioterrorism-related communication system.
Some systems are designed primarily for facilitating command and control functions. Typically, these systems are designed to collect a stream of data -- some from detection systems and others from meteorologic forecasts -- and apply a mapping function to perform analyses and make predictions for decision makers in a command and control center. Although they are principally designed for use by incident commanders, we have included these systems because they may have some utility for public health officials.
| System name | Purpose | Manufacturer | Contact information |
|---|---|---|---|
| Automated Decision Aid System for Hazardous Incidents (ADASHI)481 | To improve the response of military and civil personnel to a biological or chemical incident. Has integrated, automatic functions to track and monitor essential data regarding the incident, including hazard source analysis, hazard area prediction, detection planning and sampling, and medical treatment. Decision support provided through direct questions and memory prompts regarding operational options, as well as projected consequences of decisions. Will be available in early 2003. | Edgewood Chemical Biological Center | Edgewood Chemical Biological Center Aberdeen Proving Ground, MD 21010-5424 410-436-1915 |
| Chemical/Biological Operational Decision Aid (CODA)482 | For the prediction of casualty and human performance degradation analysis for military operations in the chemical, biological, and radiological environment. | Veridian Corporation | 1400 Key Blvd, Ste 100 Arlington, VA 22209-2369 703-516-6372 http://www.veridian.com/offerings |
| LandScan483, 484 | To provide detailed worldwide population information for estimating ambient populations at risk during hazardous releases (e.g., chemical, biological, radiological). The database is available on compact disc and is not an integrated system. | Oak Ridge National Laboratory | Oak Ridge National Laboratory P.O. Box 2008, M.S. 6237 Oak Ridge, TN 37831 http://www.ornl.gov/gist/projects/LandScan/landscan_doc.htm |
| Meteorological Information and Dispersion Assessment System Anti-Terrorism (MIDAS-AT)479, 480 | To model attacks involving weapons of mass destruction (WMD) using real-time meteorological data. Hazard predictions updated every 5 minutes using live sensor and weather tower data. Includes Urban Terrain and Inside Building models to increase accuracy of modeling predictions. | PLG of EQE International, Inc. | http://www.midas-at.com/plg-home.html |
| Nuclear-biological-chemical (NBC) Analysis485 | To serve as a tool for risk management in emergency and training incidents involving hazardous materials. Can integrate both mapping and sensor data. Calculates the predicted hazard areas using a standard algorithm. | Bruhn NewTech | http://www.newtech.dk/cer-products.htm |
| Nuclear-biological-chemical (NBC) Command and Control486 | To provide decision support during nuclear, biological and chemical weapons events. Includes functions for real-time mapping of assets and toxic clouds, support for medical response and asset placement decisions, and planning and prediction tools for multiple sensors. | Litton Integrated Systems | http://www.nbcindustrygroup.com/handbook/pdf/COMMUNICATIONS.pdf |
| Systematic Approach for Emergency Response (SAFER®) Real-Time System487 | To model toxic releases using real-time weather information. Integrates information from toxic gas sensors, weather stations, and mobile detectors to provide timely modeling of vapor clouds. | SAFER® Systems, L.L.C. | 5141 Verdugo Way, Suite B Camarillo, CA 93012 800-621-7237 http:/www.safersystem.com/RealTime.htm |
In addition to their utility in managing actual crises, these systems may have utility in preparing for events and training personnel. For example, the Meteorological Information and Dispersion Assessment System Anti-Terrorism (MIDAS-AT)479, 480 could be used by event planners to consider vulnerabilities in event security and train staff accordingly. Most of these systems include mapping functions and several report having analysis capabilities to provide decision makers with current information about the status of the event (e.g., where sensor data are reporting abnormal aerosols and where hospitals have exceeded capacity) and predict the spread of the outbreak.
The information about these systems was primarily derived from Web-based information provided by the manufacturers. These descriptions provided no data about timeliness, security measures, hardware requirements, or most of the criteria by which we had intended to evaluate them.
These systems are designed to be used by incident commanders, emergency management personnel, and the military; none has been clinically evaluated. However, data from these systems could be integrated with data from environmental detection and surveillance systems. Related projects, combining multiple sources of disparate data for combined surveillance, communication, and incident command capabilities, are currently in development.
In this section, we present the results of 2 quality evaluations of articles that reported evaluations of systems. First, we applied the quality guidelines from researchers at McMaster University33, 34 to the peer-reviewed evaluations of IT/DSSs for detection, diagnosis, management and prevention, or reporting and communication. Second, we applied the CDC's quality guidelines35 to peer-reviewed evaluations and descriptions of surveillance systems. If a given quality criterion was not specifically described by the reference or it was not applicable to the material presented in the article, we so noted. We did not attempt to independently evaluate any system; instead, we relied exclusively on the authors' reports.
We evaluated each of the included 48 peer-reviewed reports of evaluations of the 35 IT/DSSs for detection, diagnosis, management/prevention or reporting/communication based on 5 criteria: method of allocation to study groups (e.g., random), unit of allocation to study groups (e.g., clinician), baseline group differences (e.g., no baseline differences between study and control groups), type of outcome measures (e.g., objective), and completeness of followup (Evidence Table 4). A study of the DERMIS system178 is represented twice in Evidence Table 4 because it included 2 separate evaluations of 2 different outcomes. Therefore, the total number of evaluations included in Evidence Table 4 is 49. Twenty-eight of 49 studies evaluated the IT/DSS using objective outcome measures. Fifteen of the 49 studies reported followup rates in excess of 90 percent. However, for 3 of the quality criteria -- unit of allocation, method of allocation, and baseline differences -- fewer than 10 studies fully or partially satisfied the criteria. The overall quality of the studies was difficult to assess because the studies often did not report sufficient information for us to rate the quality criteria (Figure 13
).We reviewed each of the 31 peer-reviewed evaluations and 30 descriptive reports of 39 systems (note, many systems were reviewed in multiple articles) (Evidence Table 5). A total of 61 articles were reviewed. For each of these articles, we abstracted whether the authors specifically described the following characteristics: usefulness, importance, timeliness of the information, flexibility of the system, acceptability of the system, system sensitivity and specificity, simplicity of system use, and representativeness. Often, the discussion of these characteristics in the report was quite modest, and based on opinion about the system rather than a formal evaluation of the characteristic. No reports addressed all characteristics. At least 40 of 61 reports of evaluations or descriptions of surveillance systems described the timeliness, importance and usefulness of the system (Figure 14
). However, less than one third of the
reports of evaluations of surveillance systems described the
representativeness, simplicity, sensitivity, acceptability or flexibility of
the system.The evaluation articles of IT/DSSs for detection, diagnosis, management/prevention, or reporting/communication did not report many of the important characteristics of a clinical evaluation. We emphasize that just because the articles did not report a particular characteristic, does not mean that the study was not appropriately designed. It merely indicates that we are not able to make a judgment about that element of the study design. Similarly, many of the articles on surveillance systems did not report some of the most important characteristics of these systems including their sensitivity and acceptability. Consideration of the quality of the evidence regarding the effectiveness of the systems reported by these articles is thereby limited. The literature on the utility of available IT/DSSs to meet the information needs of clinicians and public health officials would benefit from additional articles reporting in sufficient detail so that the evidence may be objectively considered.
We abstracted technical information about each IT/DSS including what kind of hardware platform it uses, whether it uses standard vocabularies, what kind of reasoning it uses, what security measures it has, and whether access to the system is restricted by user type (Appendix C). Security concerns are important for all systems. The other questions were least relevant to the detection systems and most relevant to the diagnostic, management/prevention, and reporting/communication systems.
In general, very few of the reports of the IT/DSSs provided complete technical information. For those reports that did describe these technical characteristics, we have included this information in the relevant Table and/or Evidence Table about that system.
Most diagnostic DSSs are operated on desktop personal computers (PCs); however, the radiologic and telemedicine systems require additional hardware, and newer handheld versions of some PC-based systems are increasingly available. The reasoning used by these systems includes both probabilistic (Bayesian) and rules-based methods. Additionally, some of the systems relied on neural networks to generate their outputs. Generally, only the radiologic systems were incorporated into the IT infrastructure of the hospital in which they were operating. None of the reports of these systems specifically described restricting access to the system by user type or other security measures.
Many of the management and prevention systems were dependent on an electronic medical record. As with the diagnostic systems, the inference engines used a wide variety of reasoning methods. Some systems specifically reported using the Systematized Nomenclature of Medicine (SNOMED®) vocabulary; others simply indicated that "a standard vocabulary was used" but did not specify which one. Since most users of these systems had to log in to the hospital system before any recommendations were generated, this provided some measure of security. None of the reports of these systems specifically described restricting access to the system by user type.
The collection of surveillance data depends on reports sent via mail, e-mail, fax, and the Internet. Very few of the descriptions of surveillance systems addressed security concerns.
The reporting and communications systems vary enormously with respect to their technical characteristics. Most of the clinician alerting systems required an electronic medical record system with a rules-based reasoning system that sent e-mail or pager alerts. Generally, the communication systems within the public health system and between clinicians and their patients are Web-based, require passwords, and have some security mechanisms including encryption of patient information. Several systems reported being HIPAA-compliant, and one system has role-based access for users. None of the reports of these IT/DSSs described measures to prevent system overload in times of surges in demand or to thwart cyber attackers.
Key Question 1: What are the information needs of clinicians and public health officials in the event of a bioterrorist attack?
) and these tasks in the
Task Decomposition
(Table
2).
Briefly, clinicians require the necessary information to make diagnostic, management, prevention, and reporting decisions. Diagnostic decisions require information to accurately estimate the pre-test probability of disease for a given patient. This includes information about the probability of a patient's exposure to a biothreat agent, their susceptibility for developing bioterrorism-related illness, and the clinical syndromes associated with bioterrorism-related illnesses. Clinicians' interpretation of test results requires information about the sensitivity and specificity of the test. The probability of disease, given a positive or negative test, cannot be calculated without knowledge of the sensitivity and specificity of the test. Because clinicians' decisions will depend on their assessment of the probability of disease after testing, lack of information about sensitivity and specificity critically limits diagnostic decision making. Management decisions require information about how to appropriately distinguish between those patients who need treatment and those who do not, how best to treat the acutely ill, whom to isolate and how, how to manage scarce resources, and how to maintain personal safety. Prevention decisions require information about prophylaxis and vaccination protocols. Reporting decisions rely on information about what constitutes a reportable case or cluster of cases, and about the kinds of data that public health officials seek.
Similarly, the information that public health officials require to prepare for and respond to a bioterrorism event can be considered in relation to the decisions they have make. The decision to perform an outbreak investigation requires information about the baseline characteristics of the surveillance data and threshold levels that suggest that an outbreak resulting from naturally occurring or bioterrorism-related illness may have occurred. This information includes (for each source of surveillance data): timeliness, sensitivity, and specificity, expected value of rates being monitored, and method for determining the outbreak threshold. Once a bioterrorism event has been identified, public health officials require information that will enable them to perform ongoing surveillance in the midst of the crisis to track the extent and spread of the epidemic. The decisions regarding the institution of epidemiologic control measures that prevent the spread of disease require information about the transmissibility of the suspected biothreat agent(s) and about the criteria for and effectiveness of prophylaxis and quarantine strategies. Decisions to issue a surveillance alert require information about the nature of the suspected bioterrorist attack and the characteristics and expected natural history of the suspected biothreat agent(s). Other communication decisions relate to the specific information that needs to be conveyed to other public health officials, clinicians, the media, and other decision makers.
Key Question 2: Based on the information needs identified for these decision makers, what are the criteria by which IT/DSSs should be evaluated with respect to usefulness during a bioterrorism event?
For detection systems, important evaluation criteria in this Report include the following: the purpose of the system, information regarding the type of sample collected, portability, and methods for maintaining the security of the sample. For the collection systems we also evaluated the collection efficiency, limits of size of particulate collected, and flow rate. For the particle counters, biomass indicators, and identification systems we also evaluated sensitivity, specificity, the upper and lower limits of the size of particles that can be counted (for the particle counters), and the concentration of organisms that can be detected (for the biomass indicators). For the identification systems we also evaluated the amount of time it takes to run a sample, the number of samples that can be run at a time, the number of biothreat agents it can identify, and whether it can identify both toxins and organisms. We evaluated each of the reports of integrated collection and identification systems according to the same criteria for each of the component systems.
For diagnostic DSSs, important evaluation criteria include the following: the purpose of the system, the type of information required by the DSS (e.g., a manually-entered list of signs and symptoms provided by the clinician), the type of information provided by the DSS (e.g., a list of differential diagnoses with or without associated information about the diseases of interest), diagnostic sensitivity and specificity, whether the biothreat agents and their associated illnesses are included in the knowledge base, the method of reasoning used by the inference engine, information regarding the ability to update the probability of biothreat-related illness as the epidemic progresses, and the type of hardware required.
For management systems, important evaluation criteria included the following: the purpose of the system, the type of information required by the system (e.g., patient information from an electronic medical record), the type of information provided by the system (e.g., antibiotic recommendation or quarantine recommendation), information about the manner in which the management recommendations are provided (e.g., whether the recommendations are provided in an unprompted manner to the user), timeliness of management recommendation, the accuracy of the management recommendations, whether the biothreat agents and their associated illnesses are included in the knowledge base, the method of reasoning used by the inference engine, whether the system uses a standard vocabulary, information regarding the ability to update recommendations as the epidemic progresses, type of hardware required, and the system's security measures.
For surveillance systems, important evaluation criteria included the following: the purpose of the system, the type and method of surveillance data collected by the system; timeliness of data collection, analysis and presentation to the decision maker; methods for determining when an outbreak has occurred; geographic area under surveillance; the type of hardware required; the system's security measures; and information regarding the public health importance of the health event under surveillance, the system's usefulness, simplicity, flexibility, acceptability, sensitivity, specificity, representativeness, and the direct costs needed to operate the system.
For reporting and communication systems, important evaluation criteria included the following: the purpose of the system, the type of information the system is intended to communicate, the intended provider of the information being communicated, the intended recipient of the information, whether the recipient has to actively seek the information from the provider (e.g., by visiting a Web site) or the information is transmitted by the provider by phone, fax, e-mail, or other means to the recipient (i.e., passive on the part of the recipient), the timeliness of the system, type of hardware required, and the system's security measures.
Key Question 3: When assessed by these criteria, in what ways could existing IT/DSSs be useful during a bioterrorism event? In what ways are they limited?
Our review identified 217 IT/DSSs, few of which were designed specifically for response to bioterrorism. Rather, most included systems had other intended purposes, but could conceivably be useful to clinicians or public health officials in response to a bioterrorism event. The evidence on which to judge the usefulness of these systems is limited. Many of the systems were not evaluated even for their intended purpose. Of the studies that did evaluate systems for their intended purpose, few adhered to published criteria for high-quality evaluations. Even if a system was found useful for its intended purpose, we cannot infer that the system necessarily would be useful for response to bioterrorism. For example, surveillance systems that may function effectively for their intended purpose (e.g., the detection of naturally occurring outbreaks such as influenza) may not provide information quickly enough to be useful in rapid detection of a bioterrorism event. We now describe the evidence we identified in more detail.
The 55 collection, particulate counters and biomass indicators, and rapid identification systems described in this Report have critical roles to play in the detection of a covert release of a biothreat agent. Additionally, they are required by first responders and clinicians to test environmental and clinical samples after a known release. However, the paucity of comprehensive evaluative information about these systems prevents conclusions about whether or not one or more of these systems is likely to be useful for these purposes.
The evidence on detection systems was descriptive and predominantly collected from government sources and manufacturers' Web sites. We note that the definitions of what constitutes a "rapid" or "portable" test varied widely. We found no reports that directly compared 2 or more of the commercially available systems in any given category. Additionally, few of these systems have been compared to a gold standard and their sensitivity and specificity remain poorly characterized in the publicly available literature. The few reports of evaluations of the antibody-based systems, which are available for less than 10 of the most worrisome biothreat agents, have been characterized by high false positive rates. The nucleic acid-based systems are limited by the availability of sensitive probes but are promising in terms of portability, timeliness, and ability to communicate results to decision makers at remote locations. A significant gap in the literature is an analysis performed by an independent research group comparing the most promising technologies to each other and to a gold standard. For most systems, the available information does not describe if reagents are widely available, how difficult it is to train first responders in the use of these systems, how difficult it is to use these devices to collect and analyze samples in a secure manner in the event that that are used as evidence in a criminal investigation of the bioterrorist attack, and how much it would cost to fully implement these systems.
We identified 23 diagnostic systems with potential utility for enhancing the likelihood that clinicians consider the possibility of bioterrorism-related illness. None of these DSSs has been evaluated formally with respect to bioterrorism response. Three of the general diagnostic DSSs have been evaluated for their intended (non-bioterrorism related) purposes. In these evaluations, the general diagnostic DSSs typically performed better than physicians-in-training but not as well as experienced clinicians, and they performed better on more straightforward cases but less well for difficult cases.
The evidence for the utility of telemedicine systems for bioterrorism is mixed. Telemedicine systems are most useful in areas with limited direct access to medical specialists. Because acts of bioterrorism against civilian populations may be more likely to occur in population centers than in remote areas, the usefulness of these systems may be limited. However, since few practicing primary care or emergency physicians have ever seen the rashes associated with smallpox or other bioterrorism-related illness, the use of teledermatology technologies may increase the likelihood of a timely diagnosis by facilitating access to dermatologic experts. Additionally, in the event of a widespread epidemic reaching geographically isolated areas, public health officials could use existing telemedicine infrastructures to relate public health information and alerts to clinicians.
| System name | Purpose | Description | Diagnostic accuracy | Contact information |
|---|---|---|---|---|
| University of Chicago - Computer Aided Diagnosis of Interstitial Lung Disease153-155 | To aid in the detection of interstitial lung disease in chest radiographs. | System digitizes radiograph and analyzes specific regions of interest (ROIs) for abnormalities. The computer then quantifies the proportion of abnormal ROIs and makes a determination as to whether the image is normal or abnormal. | Areas under the ROC curve obtained with and without computer-aided diagnostic output were 0.970 and 0.948 (p = 0.0002), respectively.155 | Department of Radiology Kurt Rossmann Laboratories for Radiographic Research, MC 2026 University of Chicago 5841 S. Maryland Ave. Chicago, IL 60637 |
| University of Chicago - Artificial Neural Network for Interstitial Lung Disease155 | To help radiologists differentiate among 11 interstitial lung diseases. | The artificial neural network uses 10 clinical parameters (age, sex, duration of symptoms, severity of symptoms, temperature, immune status, underlying malignancy, smoking history, dust exposure, and drug treatment) and 16 radiographic findings in 3 categories (distribution of the infiltrates, characteristics of the infiltrates, and other findings) to develop a differential diagnosis. | Areas under the ROC curve obtained with and without the system output were 0.911 and 0.826 (p < 0.0001), respectively.155 | Department of Radiology Kurt Rossmann Laboratories for Radiographic Research, MC 2026 University of Chicago 5841 S. Maryland Ave. Chicago, IL 60637 |
Management and prevention systems are designed to make recommendations to clinicians by abstracting clinical information from electronic medical records to make patient-specific recommendations. None of the 18 systems identified in this review has been specifically designed or evaluated for utility in providing management or prevention recommendations during a bioterrorism event. Moreover, we have no information as to whether the knowledge bases of these systems include comprehensive information about bioterrorism-related illnesses. The systems that are not linked to electronic medical records share many of the limitations of the general diagnostic systems -- including, that clinicians may not use the system to seek advice for patients presenting with common viral syndromes (i.e., the bioterrorism-related syndromes). Expert systems that continuously search electronic medical records (including data from the laboratory, radiology reports, and physician notes) for new evidence of an infection and apply clinical practice guidelines to those data have potential utility in bioterrorism management. However, to establish their utility for improving management or prevention decision making during a bioterrorism response, evaluations of the hospital IT infrastructures and methods for the incorporation of clinical practice guidelines for biothreat-related illnesses are required. Antibiotic recommendation programs are typically designed to provide recommendations for antibiotics with the narrowest possible spectra, thereby reducing the risk of developing resistant organisms. If clinicians make antibiotic selection decisions while unaware of the true bioterrorism-related diagnosis and select narrow-spectrum antibiotics, they may not be effective against the pathogens. Therefore, whether the use of these systems would be helpful or detrimental is not known.
). However,
less than one-third of the reports of evaluations of surveillance systems
described the representativeness, simplicity, sensitivity, specificity,
acceptability or flexibility of the system. The quality of the evidence
regarding the effectiveness of the systems reported by these articles is
therefore limited.
Syndromal surveillance systems have been designed with the intention of collecting data that could provide an early indication of a bioterrorism event. However, we have no evidence to determine which of the methods of collecting syndromal data (e.g., triage nurses collecting syndromal data on patients presenting to emergency departments, clinicians providing syndromal reports on suspicious patients, or using administrative data such as ICD9 codes or school absenteeism data) provides the most sensitive, timely, acceptable, and low cost data.
| System name | Geographic location | Data used in surveillance system | Method of data collection | Sponsoring agency |
|---|---|---|---|---|
| 121 Cities Mortality Reporting System291, 292 | U.S. | Reports from cities that summarize the total number of deaths in each area and the number attributed specifically to pneumonia and influenza. | The CDC receives weekly influenza and pneumonia mortality reports from (now) 122 cities and metropolitan areas in the United States within 2-3 weeks from the date of death. The reports received through this system are published as Table 4 of MMWR. | CDC http://www.cdc.gov/epo/dphsi/phs.htm#121 |
| AAH Meditel United Kingdom General Practitioner Reporting System287 | United Kingdom (U.K.) | Clinician reports of influenza cases. | Each night, data are downloaded via the phone line from the computers of 850 sentinel general practitioners to the mainframe computer at AAH Meditel. The system was originally intended to collect prescribing data but has been extended to collect other patient data as well. | AAH Meditel |
| Arbeitsgemeinschaft Influenza (AGI) Sentinel Surveillance System293 | Germany | Clinician reports of influenza cases, including number of acute respiratory infections (by age group) and number of patient consultations; viral samples; hospital admissions; mortality rates; work and school absenteeism. | Over 400 volunteer physicians submit patient data to a central database via a networked computer in their office, mail or fax. Throat swabs are collected 1 day per week from 30 randomly chosen clinician offices and submitted to 1 of 3 reference labs for analysis. | AGI |
| California Influenza Surveillance Project294 | California | Kaiser-Permanente inpatient admission diagnoses, Kaiser-Permanente outpatient pharmacy prescriptions for antivirals, outpatient influenza-like illnesses from sentinel physicians, and respiratory virus isolations and detections from laboratory sources. | Kaiser-Permanente inpatient admission diagnoses and pharmacy data are reported weekly. The number of influenza admissions is divided by the total number of hospital admissions for the same day (excluding pregnancy, labor and delivery, birth, and outpatient procedures) to give the percentage of influenza admissions. Kaiser pharmacy data are also reported on a weekly basis by all Kaiser outpatient pharmacies in California. Sentinel physicians throughout California report the number of outpatient visits for influenza-like illness and the total number of visits per week. Virus isolation and characterization data comes from hospital, academic, private and public health laboratories located throughout California. During the influenza season, these laboratories report the number of laboratory-confirmed influenza and other respiratory virus detections and isolations on a weekly basis. An unspecified portion of influenza viruses isolated at County and Kaiser laboratories are forwarded to the Viral and Rickettsial Disease Laboratory for further antigenic and genetic characterization. Additional specimens also come from sentinel physicians.294 | California Department of Health Services, Viral and Rickettsial Disease Laboratory |
| DOD Influenza Surveillance Program,285, 286 formerly known as Project Gargle245 | Nineteen sentinel U.S. military bases, 43 non-sentinel bases, and overseas locations in Argentina, Peru, Ecuador, Panama, Japan, Korea, Thailand, and Nepal. | The reports of results of throat swabs for influenza. | Samples from both military personnel and local residents are shipped to the virology lab at the Epidemiology Surveillance Division, Brooks Air Force Base for culture. Selected isolates are then sent to the CDC for further analysis. | U.S. Air Force; DOD-GEIS |
| Dutch Medimatica Influenza System290 | The Netherlands | Clinician reports of influenza cases and viral samples. | Voluntary input of suspected influenza cases and collection of viral samples by clinicians. | National Influenza Centre, Erasmus University |
| European Influenza Surveillance Scheme (EISS),295 formerly CareTelematics296, 297 | Eight networks in 7 European countries | Clinician reports of influenza cases and virological influenza data. | Data are sent electronically from each national network to a central computer in Paris via the Internet; users may query the database 24 hours a day. | Institut Pasteur |
| National Flu Surveillance Network (NFSN)298 | U.S. | Clinician reports of influenza cases. | The NFSN consists of nearly 6,300 volunteer physicians at over 1,100 surveillance sites located in all 50 states that provide Web-based reports of influenza cases. Each day, the sites report the flu test results from ZstatFlu®, a throat swab test reported to be 99% specific (no sensitivity data provided).298 These electronic results are posted on http://www.FluWatch.com. The Web site received over 11,000,000 page views during the last flu season. Alerts are issued when influenza is being reported at least every other day in moderate numbers. A warning is issued when flu is being reported daily in high numbers. Since September 11, 2001, each member has received an alert that the NFSN could assist in countering a biological agent attack as well as information for review of possible biological agents and their symptoms. | FluWatch http://www.FluWatch.com. |
| Regional Influenza Surveillance Group (GROG)288, 289 | France | Clinician reports of influenza cases, viral samples, emergency physician workloads, absence from work less than 15 days in duration, selected drug prescription and use, hospital admissions. | Almost 1,500 voluntary general practitioners, pediatricians, pharmacists, and military medical officers provide a weekly activity summary, including number of sick-leaves prescribed, via the phone. Patient samples are submitted directly to the Northern France Reference Centre. Hospital admission data from electronic medical records are collected weekly by telephone from the "Assistance Publique" (Health and Social Security Services). The distribution of 10 common influenza-related drugs is obtained weekly from the Pharmaceutical Commercial Office (OCP), a major drug distributor in France. Additional work absence data are collected from a random sample of Social Security offices chosen to participate in GROG. One study showed that the most sensitive health services-based indicators are data on sick-leave, emergency home visits, and general practitioner reports of patients with influenza-like illnesses. A regression-based method for early recognition of influenza epidemics based on time series analysis is used to evaluate the incoming data. | Northern France Reference Centre http://www.grog.org/ |
| U.S. Influenza Sentinel Physicians Surveillance Network238, 299 | U.S. | Clinician reports of influenza cases. | Approximately 500 physicians around the country give weekly reports via a secure Internet site on the total number of patients seen and the number of those patients with influenza-like illness, by age group. Some clinicians also submit viral samples.300 | CDC http://www.cdc.gov/ncidod/diseases/flu/weekly.htm |
| WHO Influenza Surveillance240, 284/FluNet301 | Global | Clinician reports of influenza cases. | 110 collaborating laboratories in 82 countries perform analyses of local influenza isolates. Providers complete Web-based data entry form. Weekly entries include the number of patient specimens tested and the number of influenza virus isolates. | WHO; INSERM |
Although none of the surveillance systems that collect influenza data has been evaluated specifically for the detection of bioterrorism-related illness, they are potentially useful for bioterrorism surveillance in 3 ways. First, sentinel clinicians who report on patients with suspected influenza are experienced at applying a case definition to a clinical population for the collection of public health data. Because many bioterrorism-related illnesses present with a "flu-like illness," this network of trained sentinel clinicians could provide valuable surveillance data. (We note that the evaluation of these sentinel clinicians is derived from heterogeneous surveillance networks in North America, Europe, and Australia. It is difficult to know whether the cultures of medicine, the training that sentinel clinicians receive, and their commitment to public health surveillance efforts is sufficiently similar that we can assume that the results of an evaluation of a surveillance network in France will be generalizable to clinicians in the U.S.) Second, most influenza surveillance systems integrate clinical and laboratory data for the detection of influenza outbreaks. Surveillance for bioterrorism may be aided by similar integration of multiple data sources. Finally, influenza surveillance -- like bioterrorism surveillance -- requires a coordinated global effort. New programs for the surveillance of bioterrorism-related illness could be derived from the existing IT infrastructures and the historical relationships that have been developed for influenza surveillance. Several of the influenza systems rely on weekly reporting by clinicians -- for bioterrorism surveillance, this time lag is likely to be problematic.
Surveillance systems for hospital-acquired infections were included in this Report because of the possibility that data from these systems, already collected in electronic format for use by hospital infection control officers, might be a useful addition to an integrated surveillance system organized by local public health officials. However, the reports of the surveillance systems for hospital-acquired infections suggest that although these systems could be a valuable tool for hospital infection control officers, there is little evidence to suggest that they have sufficient sensitivity, specificity, or timeliness to detect a community-based bioterrorism event.
When we consider the 90 surveillance systems described in this Report, there are relatively few systems collecting the earliest surveillance data -- such as school and work absenteeism, calls to telephone care nurses, over-the-counter pharmacy sales, or veterinary or zoonotic illness -- a potentially significant gap in available surveillance systems.
Key Question 4: In areas where existing IT/DSSs do not meet the information needs of clinicians or public health officials, what functional and technical considerations are important in the design of future IT/DSSs to support response to bioterrorism events?
There are at least 3 explanations for why the evidence about an existing IT/DSS may fail to demonstrate its utility to meet the information needs of clinicians and public health officials. First, it may be that the evidence actually demonstrates that the system fails to support the information needs of clinicians and public health officials. Second, it may be that the evaluation data are of sufficiently poor quality that they cannot support the conclusion that the system may serve the information needs of clinicians and public health officials. Third, the system may have demonstrated efficacy in a clinical trial; however, when used in a real-world environment or clinical setting, the system lacked effectiveness. We did not identify evaluations or studies that directly assess the functional and technical requirements that are important for future IT/DSSs. In this section, we provide our interpretation of factors that should be considered for the design of future IT/DSSs.
| System name | Purpose | Test characteristics | Availability |
|---|---|---|---|
| AMEBA Biosensor62, 63 | To detect biothreat organisms using microorganisms as sensors. | No information available. | Currently available through Gensor Inc. |
| Digital Smell/Electronic Nose59, 64 | To detect volatile gases given off by microorganisms during normal metabolism. | Holmberg et al. demonstrated that an array of 15 sensors was able to correctly classify 68 of 90 colonies containing 0 or 1 of 5 test organisms and an uninoculated control. (However, it registered 22 of 90 as false positives).59 | Currently available through Osmotech Inc. |
| Interim Biological Agent Detector (IBAD)10, 36, 39, 65, 66 | To continuously monitor the air for a significant rise in particulate concentrations. | No information available. | Limited to military use. |
| Long Range Biological Standoff Detection System (LR-BSDS)10, 36, 39, 67 | To be flown in helicopters for the detection of aerosol clouds resulting from long-line source attacks. | No information available. | Limited to military use. |
| Met One Aerocet 531 Mass/Particle Counter43, 68 | To detect statistically significant rises in aerosol concentration over background. | Battery-powered handheld counter with a particle range of 0.5 to 10.0 µm (Met One makes several other particle counters and the particle detection characteristics vary by model; the handheld Model GT-321 can detect particles with a lower threshold of 0.3 µm). Has a computer interface with extended data storage of up to 4,000 data records. | Currently available through Pacific Scientific Instruments. |
| Model 3312A Ultraviolet Aerodynamic Particle Sizer (UV-APS) and Fluorescence Aerodynamic Particle Sizer-2 (FLAPS-2)10, 36, 39, 57, 58 | To detect living organisms in aerosols and nonvolatile liquids. | Detectable particle range: 0.5 to 15 µm. In field tests, FLAPS-2 was able to detect 39 of 40 blind releases of simulant aerosols at a distance of about 1 kilometer with no false alarms over a 3-week period. In another trial, it was able to detect as few as 10 agent-containing particles per liter of air. | Currently limited to military use. Built by TSI for Defense Research Establishment Suffield (DRES). |
| Model 3321 Aerodynamic Particle Sizer Spectrometer (APS 3321) and Fluorescence Aerodynamic Particle Sizer (FLAPS-1)10, 36, 39, 43, 57, 69 | To rapidly distinguish aerosol clouds that could contain biothreat particles from background particles. | Detectable particle range: 0.37 to 20 µm. | Currently limited to military use. Built by TSI for DRES. |
| Portable Biofluorosensor (PBS)43 | To identify the presence of biological compounds in aerosols. | Interference is minimized but false positives occasionally occur. Better analysis is achieved with solubilized spores in comparison to airborne samples. | Currently limited to military use by the DOD. |
| Program for Response Options and Technology Enhancements for Chemical/Biological Terrorism (PROTECT)60, 61 | To distinguish between naturally occurring and abnormal aerosols. | A network of particulate collectors is set up in public places (e.g., subway stations). The data from these are analyzed in an attempt to distinguish between naturally occurring and abnormal aerosols. | Under development at Argonne National Laboratory (DOE). |
| Short-range Biological Standoff Detection System (SR-BSDS)10, 36, 39, 70 | To detect biologically active aerosol clouds at distances up to 5 kilometers. | No information available. | Under development; limited to military use. |
| Single Particle Fluorescence Counter (SPFC)43 | To count airborne particles. | No information available. | Limited to military use. |
| Spreeta™ 71 | To detect and quantify biological particles in a sample. | No information available. | Currently available through Texas Instruments Incorporated. |
| Vertical Cavity Surface Emitting Laser (VCSEL)72, 73 | To detect changes in human red blood cells indicative of exposure to biothreat agents. | No information available. | Limited to military use. |
| XM2 and PM1010, 36, 39, 74 | To provide particulate air samplers that can be mounted in high-mobility, multi-purpose wheeled vehicles. | No information available. | Currently limited to military use by the DOD. |
Note: for additional information on these systems, see Evidence Table 1.
| System name | Purpose | Description | Diagnostic accuracy | Contact information |
|---|---|---|---|---|
| DXplain™ 126, 131-134 | To provide a differential diagnosis based on clinician-entered signs and symptoms. | DXplain™ is primarily a rules-based system that relates over 5,000 findings to 2,000 diseases in a semi-quantitative fashion. A Bayesian analysis is performed to assess the likelihood of each condition, given the terms used to describe the case. The system includes descriptions and findings for potential agents of bioterror, and is updated weekly to account for potential outbreaks. Users access the system via the Internet where the knowledge base and associated rules are continually updated by its developers. The system prompts the user for more specific information when vague terms, such as "fever," are entered. Desktop computer accessible. | In an evaluation of 103 consecutive internal medicine cases, DXplain™ correctly identified the diagnosis in 73% of cases, with an average rank of 10.7 (the rank of a diagnosis refers to its position on the differential diagnosis -- for example, the diagnosis with the greatest likelihood of being the actual disease is ranked first, the next most likely diagnosis is ranked second, and so on).131 The differential diagnosis included an average of 59.3 diagnoses per case. | Developed at Massachusetts General Hospital, DXplain™ is currently available through: Laboratory of Computer Science Massachusetts General Hospital Harvard Medical School Boston, MA http://www.lcs.mgh.harvard.edu/ |
| Iliad126, 135-138 | To provide a differential diagnosis based on clinician-entered signs and symptoms. | Iliad uses both Bayesian and Boolean methods to link findings with possible diagnoses. The knowledge base is focused in internal medicine and contains information on 1,200 diseases and 14,000 manifestations. The differential diagnosis generated by Iliad is not dependent upon the level of training of the user.135 Iliad is available as a CD-ROM, which was last updated in 1997. There are no plans for future updates.138 Medical HouseCall137 is a system for consumers derived from Iliad with a novel user interface. | In a 1996 multi-center evaluation designed to assess the impact of Iliad on users at different levels of training, each of 33 users analyzed 9 cases selected at random from 36 diagnostically difficult cases. On average, Iliad included the correct diagnosis in its list of possible diagnoses for 4 of the 9 cases, and included the correct diagnosis within its top 6 diagnoses for 2 of the 9 cases.135 | Developed at the University of Utah, Iliad is currently available through: A.D.A.M. Inc. 1600 River Edge Parkway, Suite 800 Atlanta, GA 30328 Ph: 770-980-0888 http://www.adam.com |
| Quick Medical Reference (QMR)126-129, 139-147 | To provide a differential diagnosis based on clinician-entered signs and symptoms. | QMR is the most widely distributed and evaluated general diagnostic DSS. It is available on CD-ROM and operates on a stand-alone PC. QMR uses rules-based logic to associate case findings with the 600 diseases in its knowledge base. It requires that clinicians provide specific information about a case, but unlike DXplain™, does not prompt users to provide more detailed descriptions of signs and symptoms. Clinicians vary significantly in the terms entered into the system for the same case. This variability has, in turn, led to the generation of significantly different differential diagnoses, some of which do not contain the actual diagnosis.127-129 In a study of physician acceptance of QMR, clinicians found QMR to be more useful for difficult cases, in cases for which it was predicted that QMR could provide good information, and when diagnostic confidence was lower.144, 145 | One prospective study used QMR to assist in the management of 31 patients whose cases were felt to be diagnostically difficult.139, 140 Only cases for which the anticipated diagnoses existed in the QMR knowledge base were included. In the 20 cases for which a diagnosis was ultimately made, QMR included the correct diagnosis in its differential in 17 cases (85%) and listed the correct diagnosis as most likely in 12 cases (60%). An evaluation at the University of Toronto found that use of QMR improved medical interns' diagnostic accuracy.143 With the assistance of QMR, interns' differential diagnoses and diagnostic plans for hypothetical difficult cases more closely matched those of senior subspecialists, than those prepared without QMR. One study compared the diagnostic ability of QMR to interns and chief residents for 40 actual patients whose cases were of varying difficulty.141 Interns included the correct diagnosis in 84% of cases, chief residents in 90%, and QMR in 64% of cases when using the intern data and 62% of cases when using the chief resident data. Unlike other systems, QMR was significantly less accurate in more difficult cases. QMR was also used to analyze 154 cases admitted to a tertiary care hospital for an undiagnosed illness of less than 6 months duration and for which a diagnosis was ultimately confirmed.129 For 137 of the 154 cases, the correct diagnosis was present in the QMR knowledge base. Two physicians, blinded to the actual diagnosis, independently entered data for each case into QMR. Physician A obtained the correct diagnosis in 62 cases (40%) while physician B obtained the correct diagnosis in 57 (36%). | Developed at the University of Pittsburgh, QMR is currently available through: First DataBank, Inc. 1111 Bayhill Drive San Bruno, CA 94066 Ph: 800-633-3453 http://www.firstdatabank.com |
| Associate for Public Health148 | To provide a differential diagnosis based on clinician-entered signs, symptoms, laboratory data, and exposures. | Associate for Public Health is a decision support tool designed for clinicians and public health personnel to build a list of possible infectious or parasitic diseases based on clinical features of each patient. Texas Medical Informatics has 2 other products geared toward infectious/parasitic diseases in cats and dogs. These products also provide links to other medical and veterinary resources. | No information available. | Texas Medical Informatics, Inc. 3588 Preakness Circle College Station, TX 77845 Ph: 979-690-0844 http://www.texmedinfo.com |
| DiagnosisPro® 149, 150 | To assist clinicians in the generation of differential diagnoses, provide general information about clinical conditions and appropriate therapeutics, and advise on diagnostic tests for a given case. | Users enter up to 10 attributes about a patient and the system searches its database of 20,000 attributes to create a list of differential diagnoses. From the source material provided by the manufacturer, it does not appear that the system uses Bayesian logic to rank order the differential diagnoses. The manufacturer reports that diseases from Manson's Text of Tropical Medicine are included in the knowledge base but does not further specify which potential biothreat agents are included. DiagnosisPro® is available for desktop and handheld computers. | No information available. | MedTech USA Inc. 6310 San Vicente Blvd. Suite 425 Los Angeles, CA 90048 Ph: 800-640-8000 http://www.MedTech.com |
| Problem-Knowledge Couplers® (PKC)151 | To couple patient-specific information with medical knowledge. | PKC is a Web-based system into which users enter patient-specific information and receive information about the underlying cause of the patient's symptoms. | No information available. | PKC Corporation One Mill Street, Box A8, Suite 355 Burlington, VT 05401-0530 Ph: 800-752-5351 http://www.pkc.com |
Incorporating information from radiologic systems with other information from patients' medical records and knowledge bases about the clinical presentations of bioterrorism-related illnesses could be a useful innovation. Specifically, radiologic systems could serve as a component of an integrated management system that incorporates radiologic as well as other clinical information with clinical practice guidelines for the management and reporting of suspected bioterrorism-related illness.
A well-accepted list of the key syndromes for surveillance and detailed definitions of these syndromes could facilitate the integration of numerous sources of surveillance data. For example, an improved definition of "flu-like illness" could include its clinical characteristics so that triage nurses and clinicians can clearly identify patients with the syndrome, the specific ICD9 codes and other administrative data likely to be associated with it, the pharmaceuticals likely to be used to treat it, and the laboratory tests likely to be ordered to diagnose it. Then, each source of syndromal surveillance data can be systematically mapped to each of the syndromes, facilitating ongoing efforts to integrate multiple sources of surveillance data into a single system. The factors that affect the timeliness of a clinician surveillance report are: the interval between when the clinician first decides to report a given patient and actually sends the report, the interval between transmission and receipt of the surveillance report, the interval between receipt of the report and completion of data analysis, and the interval between analysis and communication of the results to decision makers who can respond appropriately. No system included in this Report has been evaluated to determine how long each of these intervals take, which intervals are rate-limiting, and what steps can be taken to increase their efficiency.
Surveillance efforts for bioterrorism will benefit from a detailed analysis of the sensitivity, specificity, and timeliness of each source of surveillance data; from improved spatial and temporal analysis methods; from evaluations of methods for the integration of multiple sources of surveillance data; from global investments in laboratory and communications infrastructures; from systems that collect sources of data reflecting disease earlier in the course of illness (e.g., school and work absenteeism and over-the-counter pharmacy sales); and from systems that facilitate ongoing outbreak investigation during the midst of a response to bioterrorism.
All IT/DSSs require security measures to protect patient confidentiality and thwart efforts of cyber attackers.
This Report describes 217 IT/DSSs identified in our searches because they may be useful for responding to a bioterrorism attack. Our goal was to delineate the information needs of clinicians and public health officials, and to assess whether available IT/DSSs have potential utility, relative to those needs. We used a formal approach to describe the decisions that clinicians and public health officials would make during a response, and thereby elucidated their information needs. In devising our framework for the evaluation of IT/DSSs, we focused on critical decisions that must be made by clinicians and public health officials. For clinicians, these decisions relate to diagnosis and detection, management, prevention of further exposure or illness, and communication with public health officials. For public health officials, they include decisions relating to surveillance (such as how to interpret surveillance data, when to initiate outbreak investigations, and how to track the extent and spread of the epidemic during the crisis), when to alert clinicians and the public about a potential outbreak, what type of measures to take to control the outbreak or epidemic.
This framework guided both our search for, and evaluation of systems. We anticipated that many potentially useful systems would not have been described in the peer-reviewed literature, and therefore performed extensive searches of the Internet. In fact, about 20 percent of the 217 systems that we found were described only on the Web. Some of the information that we found on the Internet is no longer available because the authors of Web sites removed the information after the bioterrorism attacks in Fall 2001.
The anthrax attacks in Fall 2001 created enormous interest in systems that could be used for surveillance, detection and diagnosis, in both the public and private sector. Intensive research funded by the CDC, DOD, DOE, WHO, and the private sector is ongoing. Because some of the information is proprietary, and because developers are concerned about publicizing some of the more sensitive information, it will be increasingly difficult for potential users to identify the available systems and to understand their capabilities.
The vast majority of systems that we found, both from the peer-reviewed literature and from other sources, have not been rigorously evaluated. Very few of the systems have been specifically evaluated for use in response to a bioterrorism event or threat. Thus, overall, there is a lack of evidence to indicate whether these IT/DSSs would be helpful. We emphasize that lack of evidence about effectiveness is not evidence for lack of effectiveness. We recognize that this may be a particular difficulty for systems designed for use by public health departments since public health officials are not likely to have the incentive to publish the results of their experiences with a given system. Many of the systems seem promising; careful evaluation of the systems could determine whether this promise is realized.
Increasingly, bioterrorism preparedness efforts are underway to create integrated "meta-systems" that combine surveillance, laboratory, and communication functionalities for both naturally-occurring and bioterrorism-related illness. An example of this is the RHEACT project, which is intended to provide standard Web portals that serve as the consistent interface to integrate multiple users, multiple sources of NEDSS compliant surveillance data (including reports from the Laboratory Response Network) with communication and alerting capabilities. It may be useful to the developers and evaluators of these "meta-systems" if the efficacy of each of its components has been rigorously evaluated.
The efficacy of any system will, in part, be dependent on the task it is expected to perform during a bioterrorism response. For a covert release of biothreat agents, the initial purpose of information systems will be the early detection of agents and bioterrorism-related illness. After the bioterrorist event has been made known, systems will be required to treat exposed people and to minimize additional exposures. In the sections that follow, we discuss examples of systems that could be used during covert and known bioterrorist attacks. However, we emphasize that the publicly available evidence on these systems is quite limited.
The detection of a covert bioterrorist attack on human or animal populations or food or water supplies requires the capacity to collect surveillance data and communicate this information to decision makers in a sufficiently timely manner that actions may be taken to prevent resulting morbidity or mortality. These surveillance data are not just from laboratories and clinicians but include information from the FBI and other members of the intelligence community that may suggest the possibility of a bioterrorist attack. Surveillance systems have the additional role of tracking the extent of an outbreak once they have been discovered. Here, we briefly describe surveillance systems that could contribute to surveillance efforts for the detection of and then monitoring the extent of a covert bioterrorist event. We do not specifically describe the necessary communication systems that facilitate the secure and timely transmission of surveillance information to the public health decision makers but recognize that this is a critical component of any surveillance system for the detection of a covert bioterrorist attack.
For the early detection of a covert release of biothreat agents, integrated surveillance systems that collect, analyze, and communicate data from multiple sources would be useful. There is no evidence that any currently available technology can perform all of these functions. However, efforts like the Program for Response Options and Technology Enhancements for Chemical/Biological Terrorism (PROTECT) project of the DOE hold promise with respect to integrating data from multiple detection systems in locations of likely attack. Networks of disparately deployed collection systems, particulate counters and biomass indicators (with or without the identification systems described in the next section) sending electronic reports of their results to a central surveillance analysis center could provide the earliest possible evidence of a bioterrorist attack -- even before the first patient becomes ill.
There are a number of syndromal surveillance systems in use. Devices such as Health Buddy® and the Patient Encounter Module of LEADERS can be used in multiple settings (e.g., clinics, airports, sporting events) to present questions to multiple users (e.g., clinicians, patients, teachers, immigration officers, food inspectors) for input into a surveillance system. The Rapid Syndrome Validation Project (RSVP®) is a Web-based surveillance tool that enables clinicians to report data on patients presenting with one or more of the syndromes of interest. The Early Warning Outbreak Recognition System (EWORS) of the DOD-GEIS has demonstrated that personnel with limited training can use a simple computer program to collect demographic and syndromal data in remote regions of the world, and then transfer this data to analysis centers on a daily basis. A program like the Electronic Surveillance System for the Early Notification of Community-based Epidemics (ESSENCE) requires administrative data in an electronic format, but has the advantage of minimizing the burden of collection by using routinely collected data for syndromal surveillance. ESSENCE has demonstrated remarkable scalability, performs the most comprehensive analysis of syndromal data of all the syndromal systems, and, as is true for many of these systems, is currently under evaluation to determine its sensitivity, specificity, and timeliness in detecting infectious disease outbreaks.
Because clinicians may be the first to recognize unusual or suspicious illnesses, reports from clinician networks are an essential source of surveillance data for detection of bioterrorism-related diseases. In particular, because individuals exposed to a variety of biothreat agents may present with influenza-like illness initially, sentinel clinicians in the U.S. and around the world, already experienced in the reporting of influenza cases in accordance with case definitions provided by public health officials, may be useful for the detection of a bioterrorism event. None of the surveillance systems collecting clinician reports has been evaluated specifically with respect to biothreat agents. However, the evaluations of these systems for other purposes have demonstrated their ability to detect outbreaks more quickly than the manual reporting systems that they replaced.
Laboratories are critical to the rapid detection of a covert biothreat agent release. Bioterrorism surveillance in the U.S. is limited by the lack of coordination among the many types of laboratories, reporting and communication systems for laboratory surveillance data, and personnel and equipment for rapid detection of biothreat agents. Therefore, the plans for the Laboratory Response Network, with its use of NEDSS-compliant reporting, address many of the current gaps in U.S. laboratory response capabilities. In addition, efforts such as those of the DOD-GEIS to make laboratory services and training available in developing nations and at U.S. military facilities worldwide respond to the needs for global laboratory surveillance.
The dissemination of a common organism with uncommon antimicrobial resistance could be highly desirable by bioterrorists interested in minimizing the likelihood of detection of the release, and in reducing the efficacy of available therapies. Surveillance systems that collect antimicrobial resistance data could potentially detect bacterial bioterrorism agents with unusual resistance patterns. However, none of these systems has been evaluated for such use.
The contamination of salad bars in The Dalles, Oregon with Salmonella typhimurium by followers of Bhagwan Shree Rajneesh was intended to be a covert attack on the U.S. food supply. Technologies like the Salmonella Outbreak Detection Algorithm (SODA), the National Molecular Subtyping Network for Foodborne Disease Surveillance (PulseNet), and the Foodborne Diseases Active Surveillance Network (FoodNet) have all been used to detect and track foodborne outbreaks, but they are all limited to the detection of fewer than 10 species. Moreover, none of the foodborne illness surveillance systems that collect disease incidence data were specifically designed for the early detection of bioterrorist attacks on the food supply, nor has any been evaluated for that purpose.
For the management of a known attack, first responders will require portable detection systems to evaluate suspicious materials or potentially contaminated environments. Clinicians will require rapid diagnostic tests to evaluate patients with potential exposures; diagnostic systems to aid in the recognition of bioterrorism-related disease; and management systems for advice regarding triage, treatment, and prevention measures. Public health officials will require information systems to help them conduct ongoing outbreak investigations in the midst of the response, manage the results from public health laboratories, and implement epidemiologic control measures to prevent the further spread of disease. Finally, there will be a need for flexible, secure communication tools that can facilitate the transmission of information among geographically separated decision makers, the media, and the public.
Of the available portable collection systems for use by first responders, only the BioCapture™ device has reports in the publicly available literature of controlled evaluations of its flow rate and collection efficiency. Even if collection and rapid identification systems could be deployed widely, they would still be constrained by the inherent limitations of currently available detection systems: the lack of tests for many biothreat agents (e.g., smallpox); the lack of available probes for many nucleic acid tests; and the lack of tests that facilitate the evaluation of a given sample for multiple biothreat agents. Moreover, given the inadequacy of sensitivity and specificity data, decision makers may have difficulty determining the appropriate action to take given an environmental sample with either a positive or negative result.
Of the available DSSs for diagnosis, only 2 systems (GIDEON and The Computer Program for Diagnosing and Teaching Geographic Medicine) were specifically described to have knowledge bases that contain the most worrisome potential biothreat agents. Several evaluations of diagnostic DSSs suggest that the differential diagnosis produced by the DSS is highly dependent on the terms entered about the patient. The need for clinicians to manually enter patients' signs and symptoms into diagnostic DSSs may be eliminated by systems that automatically collect patient data from an electronic medical record -- although none of the systems currently available have this capability.
The management and prevention systems identified have diverse functions and purposes. None has been evaluated with respect to usefulness in response to bioterrorism. However, systems like Health Evaluation through Logical Processing (HELP) at the LDS Hospital in Salt Lake City and the Clinical Event Monitor at Columbia-Presbyterian Medical Center, which combine rich electronic medical record data with robust inference methodologies and detailed knowledge bases of infectious diseases, may be modifiable to better serve the needs of clinicians faced with management decisions. In particular, these systems use many elements of the medical record (e.g., laboratory results, clinical information, demographic data, pharmacy data and radiology reports) to generate patient-specific recommendations about antibiotic therapies, hospital infection control measures, the admission of patients with community-acquired pneumonia, and the isolation of patients with suspected active pulmonary tuberculosis. These systems would be more useful for bioterrorism response planning if their knowledge bases were updated to reflect current information about biothreat agents.
As was demonstrated in the Dark Winter tabletop exercise and in the months since the first inhalational anthrax cases in the U.S., communication systems are a critical element in the response to bioterrorism. In particular, effective communication is needed among levels of public health officials, between public health officials and the media, and among all decision makers responding to a bioterrorist crisis. The Web-based and other communication systems commonly used among public health officials at local, state, and federal levels have not been critically evaluated. Several promising efforts to improve reporting and communication include the RHEACT project in California to integrate the public health communication infrastructure. The Urban Security Initiative project of the Los Alamos National Laboratory is piloting a project to use Web-based technology to facilitate communication among 20 agencies. The Indianapolis Network for Patient Care (INPC), currently under evaluation, is a mechanism for communication between local hospitals and public health officials.477
We have evaluated IT/DSSs that affect the information needs of clinicians and public health officers in response to a bioterrorism attack. Therefore, systems designed for other decision makers (e.g., hazardous materials personnel or incident commanders) that could not also be used by clinicians or public health officers were excluded. Additionally, our focus was on detection and response to biological events; we did not include those IT/DSSs for response to chemical or nuclear weapons unless they could also be of use against biothreat agents.
Our search strategy was designed to identify all potentially relevant articles in the peer-reviewed medical literature, in government reports and on the Internet. We recognize that there are articles from other disciplines (e.g., biology and computer science) that likely contain relevant literature that were not identified by our search. We surmise that the literature we did not capture may pertain primarily to detection systems and communication systems. Additionally, all material presented in this Report was publicly available. There may be systems directly relevant to this Report that have only been described in classified documents.
We limited our literature searches to exclude reports of IT/DSSs from before 1985 on the advice of our informatics experts, who suggested that those systems were not likely to be relevant today. However, many of our references are from the late 1980s and early 1990s, and the information on systems presented in them is likely to have changed considerably. In particular, it was difficult to determine which systems are currently available. Also, many systems have had multiple names as they evolved. We have made every effort to combine reports of the same system; however, if we could not trace the name changes, we may have included duplicate reports on the same IT/DSS.
There are many details of the features of the systems included in this Report that we were not able to obtain from the readily available published information about the systems. For example, cost will greatly affect which systems public health officials, municipal leaders, hospitals and others will consider using. Unfortunately, the cost of a system was almost never mentioned in the reports that we reviewed. Similarly, information regarding the security measures of systems was rarely included in the available descriptions. Certainly, some of the missing information could have been obtained by directly contacting the developer or manufacturer of each system; however, a survey of that nature was outside the scope of this project.
Many of the IT/DSSs included in this Report have not been subjected to critical evaluations. Moreover, the quality evaluations we performed demonstrate that the evaluations of the included systems that have been performed rarely meet published criteria for the high-quality evidence that decision makers ideally require in deciding which systems best meet their needs. Throughout Chapter 3 (Results) of this Report, in the summary comments on each of the sections we discussed the gaps in the available literature. In this section, each of the 4 types of systems is discussed briefly in turn, identifying critical areas for additional systems. We then discuss other expansions of the research described in this Report.
Users of these systems require a comprehensive understanding of the test characteristics of these systems to be able to interpret their results. Therefore, studies that compare similar systems with each other and with a gold standard are required. First responders, clinicians, food safety inspectors, and veterinarians need rapid, portable detection systems, preferably with the capacity to electronically report test results to decision makers at a remote location. Studies are needed that demonstrate successful translation of the use of some of the laboratory-based systems into portable devices for use in the field. Additional tests for biothreat agents are needed, as are sensitive and specific probes for nucleic acid tests; tests that facilitate the evaluation of a given sample for more than one biothreat agent; and integration of these test results into systems that directly analyze samples and communicate results to decision makers.
Additional research is required to evaluate the utility of systems based on electronic medical records to make patient-specific recommendations for management decisions. A further understanding of the most effective means of improving inference engines, updating knowledge bases, and presenting recommendations to clinicians will be useful as these systems are developed.
There are several ongoing efforts to integrate disparate sources of data into a single system, thereby maximizing the use of routinely collected data supplemented by reports from experienced sentinel clinicians, syndromal surveillance tools, laboratories, and pharmacies. Critical evaluations of these systems are complicated by the fact that there is no gold standard by which to compare such a system. Moreover, there is no available evidence to confirm the hypothesis that the integration of multiple data sources is actually more sensitive, specific, or timely than the collection and analysis of a single source of data. Studies evaluating the characteristics of different sources of surveillance data and alternative means of integrating them are required to test this fundamental hypothesis. Additional methods for the analyses of these data in both space and time will facilitate interpretation of the data collected by these systems.
Recent events suggest that systems for the effective communication of secure data among all levels of public health officials, and between public health and the media, are urgently needed. This Report focused on the reporting and communication needs of clinicians and public health officials, only a few of the many essential decision makers in a bioterrorism response. Additional research is required to develop and evaluate an integrated communication system that includes all parties responsible for bioterrorism preparations and response, including the FBI, Federal Emergency Management Agency (FEMA), the National Security Council, the Office of Emergency Preparedness, the Office of the Secretary of HHS, and the Office of Homeland Security, among many others. These systems will require additional methods for the secure transfer of patient-specific information and will have to comply with standards developed through HIPAA. The assumption that systems that actively send data to the recipient (i.e., that call, e-mail, or fax the recipient directly) are more effective than passive systems in communicating critical information remains to be formally tested. A better understanding of the barriers to mandated clinician reporting of communicable diseases in this era of increasing use of computers in the routine care of patients could help in the design of systems for this purpose.
In this Report, we developed a conceptual model of the decisions and tasks clinicians and public health officials would have to make in the event of a bioterrorism event. The influence diagram and task decomposition serve as the framework by which we evaluated IT/DSSs for bioterrorism preparedness and response. We defined clinicians as all personnel who would be directly involved in the care of patients resulting from a bioterrorist event in a clinic or hospital. These include physicians, nurses, nurse practitioners and respiratory therapists. We used the term public health official (unless otherwise specified) to refer to all professionals at the local, state, national and international levels responsible for preparing for and responding to acts of bioterrorism to ensure the public health. Because each of these groups includes many different types of decision makers, the influence diagram, and task decomposition could be expanded to describe these differences in detail. Similarly, our conceptual model could be expanded to evaluate the decisions and tasks of other groups of relevant decision makers (e.g., laboratory personnel, first responders, veterinarians, and hospital administrators).
The purpose of this project was to perform a comprehensive review of the available literature of IT/DSSs for clinicians and public health officials in the event of a bioterrorism event. Methodologies other than systematic review would provide additional valuable insight into the answers of the Key Questions addressed in this Report. Specifically, surveys of clinicians and public health officials could be used to better describe the information needs of these groups in preparing for and responding to bioterrorist events, the IT/DSSs currently in use, and the performance of these systems in routine use and times of crisis. There is also the need for additional research on how to provide effective training in the use of IT/DSSs and how to maintain the security and availability of systems in times of crisis.
In conclusion, from our systematic review of the literature, we have identified the important decisions and tasks of clinicians and public health officials in preparing for and responding to bioterrorism. The IT/DSSs described in this Report may assist with these decisions, but most were designed for purposes other than a response to bioterrorism and have not been evaluated even for their intended purpose. Many of these systems are reasonable candidates for further evaluation. Such evaluations would clarify their value both for response to bioterrorism, and for the other purposes for which they were designed.
This Evidence Table presents selected abstracted data for 4 categories of detection systems: Collection Systems, Particulate Counters and Biomass Indicators, Identification Systems, and Integrated Collection and Identification Systems. For each system we include its name, purpose, a description, the biothreat agents it can be used to detect, and, if available, information about the system's accuracy.
| Name: | BioCapture™ |
| Purpose: | Portable collection system for use by first responders. |
| Description: | Captures airborne bacteria, spores and other pathogens 0.5 microns and larger into a small volume of liquid. The collector uses rotating impeller arms to impact airborne particles, while simultaneously washing the surface with a fluid. The liquid can then be deposited onto handheld assays, the BTA™ Test Strips (see below, under Identification Systems). Two models are currently available, the BT-500 and the BT-550. The collection systems are identical. The BT-550 has an integrated automatic sample delivery system designed for use with Tetracore BTA™ test strips. It is currently fielded in fire departments in Seattle, Los Angeles, New York City, San Diego, El Paso, Montgomery County, MD, and the 9th WMD Civil Support Team in California. |
| Flow rate/collection efficiency: | External validation of the BioCapture™ was conducted at the Dugway Proving Ground in December 2000. During these tests, single cell, 1 micron B. globigii was aerosolized. The performance of BioCapture™ was compared with an All Glass Impinger (AGI) that collects into liquid and a slit sampler that impacts bacteria directly onto growth media. In general, the collection efficiency of the BioCapture™ was 50-80% relative to the AGI collection, and 60%-125% relative to slit sampler collection. |
| Biothreat agents: | Determined by the handheld tests used in conjunction with the collector. |
| IT or DSS: | Neither when operated as a stand-alone unit. |
| Source of information: | No peer-reviewed evaluations. Other Web site.42 |
| Contact information: | MesoSystems 1021 N. Kellogg St. Kennewick, WA 99336 Ph: 509-737-8383; Fax: 509-737-8484 http://www.mesosystems.com (MesoSystems also manufactures the BioVic™, an aerosol collector that serves as the front-end air sampler for biological detection systems. The BioVic™ preconcentrates the air stream, capturing particles into 1 of 3 media: a small volume of liquid, a small air stream, or onto a solid surface for delivery into a sensor.) |
| Name: | Portable High-Throughput Liquid Aerosol Air Sampler System (PHTLAAS) |
| Purpose: | Portable system for detection of aerosolized and insect-carried biowarfare agents. |
| Description: | Per report from the manufacturer, after reducing contaminants from large volumes of air into small volumes of liquid, this handheld device uses "water analyzers" (not otherwise specified) to detect bacteria, viruses, and fungi.44 |
| Flow rate/collection efficiency: | No information available. |
| Biothreat agents: | Determined by the identification system used. |
| IT or DSS: | Neither when operated as a stand-alone unit. |
| Source of information: | No peer-reviewed evaluations. Government report43 and other Web site.44 |
| Contact information: | Zaromb Research Corp. 9S706 William Drive Hinsdale, IL 60521 http://www.zaromb.com/generic.jhtml?pid=4 |
| Name: | SASS 2000 Plus™ Chem-Bio Air Sampler |
| Purpose: | Portable system for collecting aerosolized samples. |
| Description: | Low-power device developed for use with a rapid identification system, manufactured specifically for use with the RAPTOR™ system but can be used with other systems as well (see below, under Identification Systems). The SASS 2000 Plus™ is a multi-stage, wetted-wall cyclone sampler that extracts chemical and particulate-based threat agents from surrounding air and transfers them to a liquid phase for detection and analysis. It concentrates airborne particles by several hundred thousand times into a small amount of water. The system has a total weight of 3.8 kg with a battery and 2.8 kg without battery, and dimensions of 14.2 centimeters (cm) width (W) by 19.8 cm diameter (D) by 30.5 cm height (H). |
| Flow rate/collection efficiency: | The SASS 2000 Plus™ has a flow rate of 260 L/min and is designed to capture particles ranging in size from 2-10 micrometers. |
| Biothreat agents: | Determined by the identification system used. |
| IT or DSS: | Neither when operated as a stand-alone unit. However, it can be connected to other sampling, detection or communication systems using an RS-232 link transforming it into an IT. |
| Source of information: | No peer-reviewed evaluations. Government report,36 other Web sites,39, 45, 46 and other.10 |
| Contact information: | Research International, Inc. 18706 142nd Ave. N.E. Woodinville, WA 98072 Ph: 425-486-7831; Fax: 425-485-9137 http://www.resrchintl.com/raptor_source.htm |
| Name: | SpinCon® Advanced Air Sampler |
| Purpose: | Portable collection system for both soluble vapors and particulate matter for use in infectious disease investigations in public buildings, workplace exposure, and clean room monitoring. |
| Description: | Collects soluble vapors and particulate matter including: volatile compounds (e.g. Sarin), nonvolatile compounds (e.g. acids derived from nerve agents), semi-volatile compounds (e.g. organophosphonates), low and moderate vapor-pressure chemical compounds, hot saturated stack gases from combustion sources, molds, pollen, fungi, bacteria, viruses, and bacteriaphages. The system can be tailored to a specific class of vapors or particulate matter through selection of the sampling fluid. Target particles or chemical compounds are transported in the sampling fluid medium, which can then be analyzed separately. System dimensions: 18"H x 12"W x 8"D; weight approximately 35 pounds and power requirements of 120 VAC +/- 10% 60 Hz, 320 watt maximum. |
| Flow rate/collection efficiency: | The system is capable of sampling over 1000L/min and can operate in batch or continuous monitoring mode with automatic or manual controls. |
| Biothreat agents: | Multiple types of agents as above; determined by the identification system used. |
| IT or DSS: | Neither when operated as a stand-alone unit. |
| Source of information: | No peer-reviewed evaluations. Government report,36 other Web sites39-41 and other.10 |
| Contact information: | Midwest Research Institute Ph: 816-753-7600, ext. 1507 |
| Name: | AMEBA Biosensor |
| Purpose: | Uses microorganisms as rapid, flexible sensors for detecting biothreat organisms. |
| Description: | The AMEBA biosensor monitors physiological response data from microorganisms exposed to aerosolized samples. The manufacturer reports that it can identify the presence of biological organisms in an aerosol sample in 1 second. The basic biosensor can come with a "Physics Package" that includes a collection apparatus. Additionally, the manufacturer produces a "Systems and Electronics Package" to provide signal image processing, decision support and the infrastructure to transmit data from up to 100 sensors to a central controller through either wire, wireless or radio frequency connections. |
| Sensitivity/specificity: | No information available. |
| Biothreat agents: | No information available. |
| IT or DSS: | Neither when operated as a stand-alone unit. However, when combined with the "Systems and Electronics Package," it appears to have both IT and DSS capabilities. |
| Source of information: | No peer-reviewed evaluations. Other Web sites.62, 63 |
| Contact information: | Gensor Inc. 2333 Huntingdon Pike Huntingdon Valley, PA 19006 Ph: 215-938-7800; Fax: 215-938-1551 http://www.gensor.com |
| Name: | Digital Smell/Electronic Nose |
| Purpose: | Diagnostic system based on volatile gases given off as metabolites by microorganisms. |
| Description: | Numerous articles describe different electronic noses. In general, they consist of an array of gas sensors with different selectivity patterns, a signal-collecting unit and pattern recognition software. The identification has been based on either statistical methods or neural networks. Arrays of electronic noses have been used to measure bacterial growth as a function of concentrations of substrates and bacterial metabolites and products. Three devices marketed by Osmetech, Inc. (for use at the laboratory, point of care, and handheld/portable) are commercially available for the detection of microorganisms causing bacterial pharyngitis, pneumonia in ventilated patients, urinary tract infections, and bacterial vaginosis. |
| Sensitivity/specificity: | Holmberg et al. demonstrated that an array of 15 sensors (including ones for oxygen and carbon dioxide) were able to correctly classify 68 of 90 colonies containing 1 of the following organisms: E. coli, Enterococcus sp, Proteus mirabilis, Pseudomonas aeruginosa, Staphylococcus saprophytica, and an uninoculated control (22 of 90 false positives in this experiment).59 |
| Biothreat agents: | No information available. |
| IT or DSS: | IT |
| Source of information: | Peer-reviewed article59 and other Web site.64 |
| Contact information: | Osmetech Inc. 500 West Cummings Park, Suite 4200 Woburn, MA 01801 Ph: 781-759-1112 http://www.aromascan.com |
| Name: | Interim Biological Agent Detector (IBAD) |
| Purpose: | Shipboard cyclone sampler that continuously monitors the air for a significant rise in particulate concentrations. |
| Description: | If a significant rise over background is detected, the instrument automatically collects an aerosol sample and alerts the ship's damage control center so the crew can collect an air sample and screen it with a handheld antigen test. IBAD was first deployed in 1994; 20 are currently in use. IBAD can identify a biothreat agent in 45 minutes. |
| Sensitivity/specificity: | No information available. |
| Biothreat agents: | Will be a function of the handheld tests used in conjunction with it. |
| IT or DSS: | It is neither when operated as a stand-alone unit. |
| Source of information: | Peer-reviewed article,65 government reports,36, 66 other Web site39 and other.10 |
| Contact information: | Naval Research Laboratory (NRL), Washington, D.C. |
| Name: | Long Range Biological Standoff Detection System (LR-BSDS) |
| Purpose: | Designed to be flown in helicopters for the detection of aerosol clouds resulting from long-line source attacks. |
| Description: | This system, which is not currently commercially available, uses infrared lasers to detect an aerosol cloud at a standoff distance of up to 30 kilometers. The objective of this system is to provide early warning in order that potentially exposed individuals may adopt personal protective measures. An improved version is in development to extend the range to 100 kilometers. The LR-BSDS does not discriminate biological material. Therefore other systems must be used in conjunction with the LR-BSDS to determine whether the aerosol contains biologically active particles and if so, to identify them. |
| Biothreat agents: | Does not specifically identify any agent. |
| Sensitivity/specificity: | No information available. |
| IT or DSS: | IT |
| Source of information: | No peer-reviewed evaluations. Government reports,36, 67 other Web site39 and other.10 |
| Contact information: | Los Alamos National Laboratory |
| Name: | Met One Aerocet 531 Mass/Particle Counter |
| Purpose: | Low-power aerosol particle sizer and counter that detects statistically significant rises in aerosol concentration over background. |
| Description: | This particle sizer and counter is the size of a large handheld calculator and is typically used to monitor clean rooms. This device draws an air sample through a laser-illuminated sample volume where airborne particles scatter light. The scattered lighted is detected by a photodiode. Sample times can be set for up to 24 hours. Count data are stored in memory and transferred to a printer or computer. |
| Biothreat agents: | Determined by the identification system used. |
| Sensitivity/specificity: | Can detect particles from 0.5 to 10.0 microns. |
| IT or DSS: | Neither when operated as a stand-alone unit. |
| Source of information: | No peer-reviewed evaluations. Government Report43 and other Web site.68 |
| Contact information: | Pacific Scientific Instruments 481 California Ave Grants Pass, OR 97526 Ph: 541-472-6515 http://www.particle.com/home.htm |
| Name: | Model 3321 Aerodynamic Particle Sizer® Spectrometer (APS® 3321) and Fluorescence Aerodynamic Particle Sizer® (FLAPS-1®) |
| Purpose: | Designed to rapidly distinguish aerosol particles containing living organisms from all other background particles. |
| Description: | APS® 3321 measures aerodynamic size as well as relative light-scattering intensity. Detectable particle size ranges from 0.37 to 20 mm, with high-resolution sizing from 0.5 to 20 mm aerodynamic diameter. APS® 3321 weighs approximately 10 kg with dimensions of 38 cm H by 30 cm W by 18 cm D. Sampling time is from 1 second to 18 hours per sample, depending on the user's needs. Included with Model 3321 APS® is the Aerosol Instrument Manager software to enable the user to perform computer-controlled operations and data interpretation. |
| Sensitivity/specificity: | No information available. |
| Biothreat agents: | Determined by the identification system used. |
| IT or DSS: | APS 3321 is an IT due to the software discussed above. FLAPS-1® is neither when operated as a stand-alone unit; however, when combined with APS® 3321, it is an IT. |
| Source of information: | No peer-reviewed evaluations. Government reports,36, 43 other Web sites,39, 57, 69and other.10 |
| Contact information: | Built by TSI according to DRES specifications. Particle Instruments Division P.O. Box 64394 St. Paul, Minneapolis 55164-0394 Ph: 800-677-2708 particle@tsi.com |
| Name: | Model 3312A Ultraviolet Aerodynamic Particle Sizer® (UV-APS®) and Fluorescence Aerodynamic Particle Sizer-2® (FLAPS-2®) |
| Purpose: | Detection of living organisms in aerosols and nonvolatile liquids. |
| Description: | The success of the FLAPS-1® project led to a second contract with TSI to construct the UV-APS®. The UV-APS® is designed to measure aerodynamic diameter, light-scattering intensity and fluorescence intensity of airborne solids and nonvolatile liquids. The UV-APS® is able to measure these parameters in real-time (programmable from 1 second to 18 hours), providing rapid measurements of aerodynamic size and scattered light for particles ranging in size from 0.5 to 15 mm, in addition to identifying fluorescence characteristics of particles and thereby distinguishing airborne biological particles from most inanimate material. FLAPS-2® adds to this basic particle-sizer the ability to measure the intrinsic fluorescence produced by living organisms which contain the bio-active molecule NADH or other similar flavinoid molecules. By itself, UV-APS® produces raw data without analysis or display capabilities. For FLAPS-2®, proprietary display and alarming software was developed by DRES with Dycor (Edmonton, Alberta) to automatically log all particle and fluorescence data and automatically trigger an alarm when an unusual proportion of fluorescent particles are detected. |
| Sensitivity/specificity: | In field tests, FLAPS-2® was able to detect 39 of 40 blind releases of simulant aerosols at a distance of about a kilometer with no false alarms logged over a 3 week period. In another trial it was able to detect as few as 10 agent-containing particles per liter of air. |
| Biothreat agents: | Determined by the identification system used. |
| IT or DSS: | UV-APS® is neither when operated as a stand-alone unit. FLAPS-2® is an IT. |
| Source of information: | No peer-reviewed evaluations. Government report,36 government Web site,57 other Web sites,39, 58 and other.10 |
| Contact information: | TSI Particle Instruments Division P.O. Box 64394 St. Paul, Minneapolis 55164-0394 Ph: 800-677-2708 particle@tsi.com |
| Name: | Portable Biofluorosensor (PBS) |
| Purpose: | Identification of the presence of biological compounds in aerosols. |
| Description: | Used during Operation Desert Storm, this system detects the excitation of airborne aerosols and aerosols dissolved in water. Excitation of the molecules is achieved using ultraviolet light from a xenon flash lamp. The emission of light of particular wavelengths when the molecules return to their unexcited state allows non-specific identification of biological compounds. |
| Sensitivity/specificity: | Interference is minimized but false positives occasionally occur. Better analysis is achieved with solubilized spores in comparison to airborne samples. |
| Biothreat agents: | Determined by the identification system used. |
| IT or DSS: | Neither when operated as a stand-alone unit. |
| Source of information: | No peer-reviewed evaluations. Government report43 |
| Contact information: | No information available. |
| Name: | Program for Response Options and Technology Enhancements for Chemical/Biological Terrorism (PROTECT) |
| Purpose: | To distinguish between naturally occurring and abnormal aerosols in order to rapidly detect and respond to a chemical or biological attack. |
| Description: | PROTECT is designed by the DOE specifically for detection of a biological or chemical attack on large, interior public locations (e.g., airports and subway systems) or interior buildings at special events. The system consists of a network of chemical and biological sensors, as well as computer programs that model airflow through the area to determine the possible spread of the contaminant. In addition, it includes tools to enable prompt and effective decision-making during such an attack, training exercises, and decontamination procedures. |
| Biothreat agents: | Determined by the identification system used. |
| Sensitivity/specificity: | Determined by the identification system used. Currently under evaluation. |
| IT or DSS: | IT |
| Source of information: | No peer-reviewed evaluations. Government Web sites60, 61 |
| Contact information: | DOE Argonne National Laboratory, Decision and Information Sciences |
| Name: | Short-Range Biological Standoff Detection System (SR-BSDS) |
| Purpose: | Designed to detect biologically active aerosol clouds at distances up to 5 kilometers. |
| Description: | SR-BSDS is designed to detect and track biological aerosol clouds while discriminating between biological and non-biological aerosols as well as hard targets. The detection is based upon an infrared beam that automatically scans for aerosol clouds and, if a cloud is detected, a laser-induced ultraviolet beam is used for determining the makeup of the cloud. This information is transmitted over a radio to a command post. Once the system is set up it operates autonomously. This scanning procedure continues to operate without any need for additional intervention. One drawback of the system, however, is its large size (50" W, 53" D, and 56" H, weighing approximately 1050 lbs). |
| Sensitivity/specificity: | No information available. |
| Biothreat agents: | Determined by the identification system used. |
| IT or DSS: | IT |
| Source of information: | No peer-reviewed evaluations. Government report,36 other Web sites,39, 70and other.10 |
| Contact information: | Collaborative effort of Fibertek and U.S. Army Fibertek, Inc. 510 Herndon Parkway Herndon, VA 20170 Ph: 703-471-7671 http://www.cbwsymp.foa.se |
| Name: | Single Particle Fluorescence Counter (SPFC) |
| Purpose: | Counts airborne particles. |
| Description: | Employs continuous airflow across a 780 nm laser-diode beam, resulting in scattering from individual aerosol particles in the air. The total intensity of scattered light is measured, and particle size is calculated. This event also triggers a 266 nm UV laser pulse that causes fluorescent particles to emit light at different wavelengths. |
| Sensitivity/specificity: | No information available. Sensitivity and specificity will also be affected by the identification system. |
| Biothreat agents: | Determined by the identification system used. |
| IT or DSS: | Neither when operated as a stand-alone unit. |
| Source of information: | No peer-reviewed evaluations. Government report.43 |
| Contact information: | Naval Research Laboratory (NRL), Washington, D.C. |
| Name: | Spreeta |
| Purpose: | To detect and quantify biological particles in a sample. |
| Description: | A rapid surface plasmon resonance biosensor. Alone, the Spreeta sensor only measures the refractive index of non-specific materials using the Kretschmann geometry. However, the presence of a biolayer allows Spreeta to detect and quantify specific biological agents in a manner that is rapid, accurate, and portable. The sensor is inexpensive and can be easily integrated with proprietary hardware and software. |
| Sensitivity/specificity: | No information available. |
| Biothreat agents: | No information available. |
| IT or DSS: | Neither when operated as a stand-alone unit. |
| Source of information: | No peer-reviewed information. Other Web site.71 |
| Contact information: | Texas Instruments, Incorporated Ph: 888-438-2214 tisensors@ti.com |
| Name: | Vertical Cavity Surface Emitting Laser (VCSEL) |
| Purpose: | Detection of changes in human red blood cells indicative of exposure to biothreat agents. |
| Description: | Instead of creating beams that pass through blood cells and then yield data, researchers insert blood samples into the laser itself to become part of the generation process of the VCSEL laser beams, altering them as they are formed. Above a specialized semiconductor, a coated glass mirror forms one end of the laser generating area. A blood sample is pumped through etched microgrooves in the glass. This design allows blood components (red cells, white cells, and pathogens) to become part of the lasing process. The components of the blood modify the lasing light as it is created in the tiny laser cavity, thus permitting output light to be analyzed in a spectrometer to detect changes in cell sizes and shapes. For victims of terrorist biological or chemical attacks, the transportable unit is expected to greatly reduce the time needed to analyze dangerous materials invading the blood stream. Diagnosis could be made on the spot, facilitating treatment when speed is crucial. |
| Sensitivity/specificity: | No information available. |
| Biothreat agents: | No information available. |
| IT or DSS: | Neither when operated as a stand-alone unit. |
| Source of information: | No peer-reviewed evaluations. Other.72, 73 |
| Contact information: | Patented jointly by Sandia National Laboratory and the National Institutes of Health. http://www.sandia.gov/media/vcsel.htm |
| Name: | XM2 and PM10 |
| Purpose: | Particulate air samplers that can be mounted in high-mobility, multi-purpose wheeled vehicles. |
| Description: | XM2 is a military biological air sampler and PM10 is its commercial biological air sampler prototype. Both collect airborne particles on a collection plate and alert the operator if the air passing through the system has more particles than the normal calibrated value. During the Gulf War, after a XM2 or PM10 alert, soldiers took a sample from the collection plate and processed it with a Sensitive Membrane Antigen Rapid Test (SMART™) kit (see section 3.2.1.3). |
| Sensitivity/specificity: | No information available. Sensitivity and specificity will also be affected by the identification system. |
| Biothreat agents: | Determined by the identification system used. |
| IT or DSS: | Neither when operated as stand-alone units. |
| Source of information: | No peer-reviewed evaluations. Government reports,36, 74 other Web site,39 and other.10 |
| Contact information: | DOD |
| Name: | Advanced Nucleic Acid Analyzer (ANAA)/Handheld Advanced Nucleic Acid Analyzer (HANAA) (also called mini-PCR) |
| Purpose: | Portable, rapid, rugged system for field detection of biothreat agents. |
| Description: | The low-power (for battery operation) ANAA consists of an array of 10 reaction modules and a laptop computer. It uses silicon chip-based spectrofluorometric thermal cyclers with no moving optical components, custom plastic sample tubes with caps, and software tailored for first responders such as emergency medical technicians. When the analysis algorithm determines that the signal is positive, the software automatically informs the user via an audible alert and a green-to-red indicator. |
| Sensitivity/specificity: | Belgrader et al. report the results of an evaluation in which ANAA was able to detect 500 CFUs of E. herbicola, a vegetative bacterium that they used as a surrogate for Y. pestis, in 15 minutes. In a second analysis, after modifications to the thermal cycling, ANAA was able to detect 500 CFUs of E. herbicola in 7 minutes. |
| Biothreat agents: | Limited only by the available probes. |
| IT or DSS: | IT |
| Source of information: | Peer-reviewed evaluation76 and other.92 |
| Contact information: | Lawrence Livermore National Laboratory Biological and Biotechnology Research Program P.O. Box 808, L-452 7000 East Avenue Livermore, CA 94551 |
| Name: | AK (Adenylate kinase) Phage Biosensor |
| Purpose: | Provides rapid, automated diagnosis of infectious diseases using the AK Phage technique. |
| Description: | The AK Phage Biosensor uses bacteriaphages (special viruses that infect particular bacteria) to identify an infectious agent within a few hours. When the correct bacteriaphage is applied to individual bacteria from a patient sample, the bacteria will lyse. This event produces the release of adenylate kinase, which can be detected by measuring the light from the light-emitting enzyme luciferase. In addition to identifying the bacteria, the biosensor can also be used to determine which antibiotics should be used to treat a particular patient. |
| Sensitivity/specificity: | No information available. |
| Biothreat agents: | No specific information available. |
| IT or DSS: | Neither when operated as a stand-alone unit. |
| Source of information: | No peer-reviewed information. Other.99, 100 |
| Contact information: | Available through a joint effort between the UK's DERA and Acolyte Biomedica Ltd. |
| Name: | Anthrax Sensor |
| Purpose: | Highly sensitive, portable detection of biological agents in mere seconds. Currently under development. |
| Description: | This anthrax sensor combines an optical fiber sensing device with technology designed to purify pharmaceuticals present in blood plasma at trace levels. Due to its high level of sensitivity, hundreds of biological warfare agents that were previously undetectable can now be identified. An additional advantage of the new sensor is that it is able to provide results in a matter of seconds. Currently, only a prototype of the device is available but researchers hope to develop a belt pack size, battery-generated portable device that could be taken into the battlefield or anywhere else this kind of monitoring is required. |
| Sensitivity/specificity: | Experiments have demonstrated that the prototype biosensor is capable of detecting endotoxins at a level that is 20 times lower than previously achieved by other similar devices. |
| Biothreat agents: | B. anthracis, other endotoxins. |
| IT or DSS: | Neither when operated as a stand-alone unit. |
| Source of information: | No peer-reviewed information. Other Web site.101 |
| Contact information: | Virginia Tech Pharmaceutical Engineering Institute Chemical Engineering 132 Randolph Hall Blacksburg, VA 24061 Ph: 540-231-7869 http://www.accessexcellence.com/WN/SUA12/anthrax298.html |
| Name: | Australian Membrane and Biotechnology Research Institute (AMBRI) Biosensor Technology |
| Purpose: | Highly sensitive and specific detection of a variety of biothreat agents using a cell-based model. |
| Description: | The AMBRI biosensor technology emulates the natural cell sensory system. It uses a synthetic lipid bilayer membrane containing ion channels that can be switched on and off in response to the presence of an analyte. Artificial membrane components include membrane-forming molecules chemically tethered to a gold coated surface, simple ion channels within the membrane that facilitate the transport of ions like sodium, a reservoir space between the surface and the membrane to store ions, and receptors, such as antibodies, attached to the membrane to recognize target molecules. When a target molecule is present and binds the antibodies, it alters the population of conduction ion channel pairs within the tethered membrane, resulting in a change in membrane conduction of electrical current. The synthesized lipid bilayer membrane is integrated into a microelectrode array, and this system converts the biological event into a digital signal enabling computer technology to analyze and define the biological event. AMBRI expects that by combining the precise structure of the ion channel sensor membrane with micron scale ultra-violet photolithography of membrane components and the reproducibility of silicon fabrication processes, they will suppress interfering signals by at least 2 orders of magnitude compared with any other biosensor technique and, through further modifications and new developments, enhance the stability of the biosensor making it suitable for field conditions. To date, a 32 (4x8) electrode flow cell biosensor has been tested. In addition, a 96-channel silicon chip has been fabricated with 96 onboard amplifiers and 8 A/D converters. Optical patterning of the biosensor binding sites has also been demonstrated. The biosensor technology has applications in bioterrorism detection, food testing, veterinary diagnostics, and environmental monitoring. |
| Sensitivity/specificity: | Currently the sensor response to bacteria is at 3000 CFU/mL, with further sensitivity enhancement strategies testing down to 100 femtomolars of thyroid stimulating hormone and with a response time of 2 minutes. |
| Biothreat agents: | Phage display antibodies libraries are available for Y. pestis, in addition to monoclonal and polyclonal antibodies for Y. pestis, F1 antigen, B. anthracis, and C. burnetti. |
| IT or DSS: | Neither when operated as a stand-alone unit. |
| Source of information: | No peer-reviewed information. Government Web site103 and other Web site.102 |
| Contact information: | Australian Membrane and Biotechnology Research Institute (AMBRI) 126 Greville Street Chatswood NSW Australia 2067 info@ambri.com.aum Ph: + 61 2 9422 3000; Fax: + 61 2 9422 3013 http://www.ambri.com.auhttp://www.darpa.mil/dso/thrust/bwd/advdiag/Programs/AMBRI.html |
| Name: | Biolog microbiologic identification system |
| Purpose: | A general identification system for microorganisms, with possible uses for B. anthracis. |
| Description: | Although B. anthracis is not included in Biolog's system database, the Chemical and Biological Defense Establishment (UK) and the Centre for Applied Microbiology and Research (UK) created an experimental in-house database for B. anthracis. Testing demonstrated a high level of sensitivity, as well as a high level of false-positive results. The system could be used as a preliminary test for B. anthracis, but further analysis would be necessary. |
| Sensitivity/specificity: | In-house database correctly identified all samples of B. anthracis with readable profiles (19 out of 20). However, it also falsely identified 5 out of 12 closely related Bacillus strains as B. anthracis. For both B. anthracis and related strains, roughly 20% of the samples gave false positive reactions, in which all reaction wells were positive. |
| Biothreat agents: | B. anthracis not included in Biolog's commercially available database; no information on other biothreat agents. |
| IT or DSS: | IT |
| Source of information: | Peer-reviewed article.88 |
| Contact information: | Biolog, Inc. Hayward, CA |
| Name: | Biosensor for E. coli |
| Purpose: | Rapid detection of E. coli, using a simple change in color to denote the presence of the bacteria. |
| Description: | Government documents report that researchers at Lawrence Berkeley National Laboratory are developing a biosensor for E. coli that uses a simple color change to indicate the presence of the bacteria. Made of a thin film of linked diacetylene molecules, the sensor reflects red light when the bacteria are absent. However, when E. coli molecules bind to receptor molecules on the detector, the molecules reorganize and emit a blue light. The report suggests that this sensor will provide a rapid method for detecting E. coli 0157:H7 with future applications for other biothreat agents including cholera toxin. |
| Sensitivity/specificity: | No information available. |
| Biothreat agents: | E. coli. |
| IT or DSS: | Neither when operated as a stand-alone unit. |
| Source of information: | Peer-reviewed report105 and other.104 |
| Contact information: | Lawrence Berkeley National Laboratory http://www.lbl.gov/http://www.pnl.gov/er_news/06_97/art1.htmhttp://www.sciam.com/0397issue/0397techbus4.html |
| Name: | BioThreat Alert (BTA™) Strips |
| Purpose: | Portable antigen-antibody test designed for field detection of biothreat agents. |
| Description: | The BTA™ test strip employs agent-specific antibodies to identify the potential threat. The suspect material, solid or liquid, is mixed with an aqueous solution, the BTA™ Sample Buffer. Five drops of the liquid mix are added to the sample port of the test strip. The sample interacts with the reagents and moves along the test material, inside the test strip's plastic case. Lateral-flow immunochromatography then provides the results. Screening results are produced in 15 minutes. Two solid bands, one in the control area and one in the sample area, indicate a positive result. One solid band (in the control area) indicates negative results. Any other combination of bands indicates an invalid result, indicating the test should be rerun. The results may be read visually, or, for greater accuracy, using the Guardian BTA™ Test Strip Reader. The Test Strip Reader prompts the user through the evaluation procedure (on the LCD display), provides a printout of the test results and time/date stamp. Embedded radio frequency identification technology can be used to document the chain of custody for each individual BTA™ strip test. A separate and specific test strip is required to screen for each biological threat. |
| Sensitivity/specificity: | Per the manufacturer, the B. anthracis strips do not cross-react with B. thuringensis. They indicate that current specifications for BTA™ test strips are available to emergency response or law enforcement officials upon request by calling 847-419-1507. |
| Biothreat agents: | Test strips are available for B. anthracis, Ricin toxin, S. enterotoxin B, Botulinum toxin and Y. pestis. |
| IT or DSS: | Neither when operated as a stand-alone unit. However, it is an IT when its data are transferred to a computer. |
| Source of information: | No peer-reviewed information. Other Web site.90 |
| Contact information: | Marketed by Alexeter Technologies 830 Seton Court, Suite #6 Wheeling, IL 60090 Ph: 847-419-1507; Toll free: 877-591-5571; http://www.alexeter.com |
| Name: | CellChip™ |
| Purpose: | Performs a high-throughput and high-content analysis of intact cells; includes the ability to detect anthrax. |
| Description: | The CellChip™ is a rapid toxin detection and characterization method. It contains 96 different assays within 0.1 cm2, one of which is the FRET-based anthrax sensor. High-resolution fluorescence imaging allows the simultaneous detection of 2 or more proteins or pathways in the sample. The data produced by this assay is linked to a domain knowledge base created from both knowledge mining and computational approaches. |
| Sensitivity/specificity: | No information available. |
| Biothreat agents: | B. anthracis. |
| IT or DSS: | IT |
| Source of information: | No peer-reviewed information. Other Web site107 and other.106 |
| Contact information: | Cellomics Inc. and DARPA http://www.darpa.mil/mto/compcad/workshops/biocad.pdfhttp://www.cellomics.com/html/technology/high_content.htm |
| Name: | DNA Biochip |
| Purpose: | Rapid identification of biothreat agents using microelectro-optical probes such as DNA. |
| Description: | Under development at Oak Ridge National Laboratory, the DNA biochip is a rapid gene probe-based biosensor. The matchbox-sized biochip offers the promise of high selectivity and sensitivity by mimicking the recognition system used by a living cell. In the future, it is hoped that the chip's ability will be expanded to detect hundreds of different genes, allowing it to identify both bacterial and viral agents. |
| Sensitivity/specificity: | No information available. |
| Biothreat agents: | No information available. |
| IT or DSS: | Neither when operated as a stand-alone unit. |
| Source of information: | No peer-reviewed information. Other.93 |
| Contact information: | Oak Ridge National Laboratory The University of Tennessee Department of Comparative Medicine F258 Veterinary Teaching Hospital Knoxville, TN 37996 Ph: 865-974-5576 |
| Name: | DOD Biological Sampling Kit (BSK) |
| Purpose: | Antigen-antibody test designed for screening of suspicious packages and munitions for biothreat agents. |
| Description: | The DOD BSK is a simple, pre-packaged kit that contains: a panel of 8 handheld assays, a bottle of buffer solution, 2 sterile cotton swabs, and an instruction card. Each kit is for one time use. |
| Sensitivity/specificity: | It should not be used with soil samples as they may cause false positives. |
| Biothreat agents: | Eight assays, not otherwise specified. |
| IT or DSS: | Neither when operated as a stand-alone unit. |
| Source of information: | Peer-reviewed article.91 |
| Contact information: | Currently the DOD BSK is available for military use from the Joint Program Office for Biological Detection. It is not clear if these kits will be made available to first responders. |
| Name: | Fiber Optic Wave Guide (FOWG) and Rapid Automatic and Portable Fluorometer Assay System (RAPTOR™) and Analyte 2000™ Biological Detection |
| Purpose: | Portable biothreat identification system using antibody probes. |
| Description: | These systems have antibody probes bound to glass optical fibers that are immersed in a capillary tube containing an aqueous solution of the sample. Other antibodies, tagged with florescent dye, are added to the sample, where they bind to the target antigen. The antigen-labeled antibody complex then binds to the immobilized antibody. Light from a laser travels through the optic fiber. The very small amount of light that escapes from the optic fiber, the evanescent wave, excites the fluorescent tags, whose emission is sent back up the fiber and detected via a photodiode. RAPTOR™, the commercially available version of FOWG, is designed for severe transportation and operating situations and weighs 14 pounds. The breadbox sized biosensor runs specific antibody-based assays in a disposable cartridge the size of a credit card. Up to 4 samples can be simultaneously run with a separate user-defined protocol for each multi-step assay. The Analyte 2000™ includes a fluidics unit that automatically introduces sample, buffer, and fluorescent reagent to the fibers as necessary. Remote identification of airborne bacteria is possible when the Analyte 2000™, its accompanying fluidics system, an air sampler, and a radio transceiver are integrated into a remotely piloted plane. Tests against airborne bacteria demonstrated that the system could effectively collect bacteria, identify it, and radio the data to the ground with a measurement rate of 1 sample per second (although there is a warm-up time of 15 minutes). The Analyte 2000™ is 20 cm length (L) by 8.5 cm H by 11.2 cm W; weighs 1.6 kg; and is controlled by a remote computer. However, the peripheral equipment, including a laptop computer required to operate the system and supplementary buffer and waste bottles contribute some bulk. This instrument was field tested by the U.S. military to identify toxins from Iraq and by first responders of the 1999 World Trade Organization conference in Seattle to screen for potential biothreat agents. |
| Sensitivity/specificity: | It has the following estimated detection levels (in water): B. anthracis (30-100 CFU/mL), Ricin (less than 10ng/mL), S. enterotoxin (1 ng/mL), F. tularensis (105 CFU/mL), V. cholerae (10 ng/mL) and Y. pestis at levels below 1 ppb from samples of a few hundred mL. Additional sensitivity data are available at: http://www.resrchintl.com/images/raptor_fs_report_010300.pdf. |
| Biothreat agents: | B. anthracis, Ricin, S. Enterotoxin B, F. tularensis, V. cholerae Y. pestis, E. coli O157:H7, Listeria, Salmonella, and Cryptosporidium. |
| IT or DSS: | FOWG and RAPTOR™ are neither when used as stand-alone units. However, the Analyte 2000™ is an IT. |
| Source of information: | Peer-reviewed article,77 government report,36 and other Web sites.46, 78-81 |
| Contact information: | Developed by the Naval Research Laboratory. Commercialized under a license to Research International, who markets the portable device as RAPTOR™. 18706 142nd Ave. N.E. Woodinville, WA 98072 Ph: 425-486-7831; Fax: 425-485-9137 http://www.resrchintl.com/raptor_source.htm |
| Name: | Field Kit for Rapid Detection of Anthrax |
| Purpose: | Rapid detection of B. anthracis in the field. |
| Description: | Berkeley Lab researchers have developed what they report to be a rapid and accurate genetic-based assay for the field detection of B. anthracis in environmental or clinical specimens. |
| Sensitivity/specificity: | Reported to have a low false positive rate even in specimens that contain closely-related Bacillus species and other microorganisms. |
| Biothreat agents: | B. anthracis. |
| IT or DSS: | Neither when operated as a stand-alone unit. |
| Source of information: | No peer-reviewed information. Government Web site.94 |
| Contact information: | Lawrence Berkeley National Laboratory is seeking an industrial partner to commercialize a field diagnostic kit. |
| Name: | Fluorescence-based array immunosensor |
| Purpose: | Simultaneous, antibody-based detection of bioactive analytes in clinical fluids such as whole blood or a nasal swab. |
| Description: | This planar waveguide immunosensor offers rapid, portable analysis of complex bodily samples. It is capable of detecting multiple analytes in a single sample using antibody-antigen interactions. It is easy to use, even by inexperienced operators, and has sensitivity levels comparable to ELISA methods. No pre-treatment of the sample is needed. Analytes are collected by a patterned array of recognition elements located on the waveguide and then quantified using fluorescent detector molecules. The analyte is identified through computer-aided analysis of fluorescent signals from the antigen-antibody complex. Detection of analytes occurs within 15-35 minutes. |
| Sensitivity/specificity: | Unable to detect physiologically relevant S. enterotoxin B levels (<125 ng/mL) in experimentally spiked urine, saliva, and blood products; sensitivity for F1 antigen from Y. pestis at 25 ng/mL in body fluids. |
| Biothreat agents: | Include S. enterotoxin B and F1 antigen from Y. pestis. |
| IT or DSS: | IT |
| Source of information: | Peer-reviewed article.89 |
| Contact information: | Center for Bio/Molecular Science and Engineering, Code 6900 Naval Research Laboratory 4555 Overlook Avenue, SW Washington, DC 20375-5348 Geo-Centers, Inc. Rockville, MD |
| Name: | GeneChip® (LifeChip High-Density Nucleic Acid Microarrays) |
| Purpose: | Rapid, simultaneous detection of numerous nucleic acids of biothreat or other pathogens. |
| Description: | GeneChip® is a dime-sized array of 100,000 or more fluorescence-tagged hybridization probes that are read optically to detect gene mutations. The instrumentation is expensive and the chips themselves have a shelf life of only a few months, but the speed and ability to search so many genes at one time would be highly desirable particularly in clinical settings where probes for common pathogens (e.g., influenza) and biothreat agents could be placed on the same chip. |
| Biothreat agents: | The number of identifiable agents is limited by the development of probes. |
| Sensitivity/specificity: | No information available. |
| IT or DSS: | Neither when operated as a stand-alone unit. |
| Source of information: | No peer-reviewed information. Other Web site95 and other.10 |
| Contact information: | These chips and probes are being developed in collaboration with Affymetrix, Lawrence Livermore National Laboratory, and USAMRIID. Affymetrix, Inc. 3380 Central Exwy Santa Clara, CA 95051 Ph: 888-DNA-CHIP (888-362-2447); Fax: 408-481-9442 |
| Name: | Handheld Immunochromatographic Assays (HHA) |
| Purpose: | Handheld antigen-antibody test for rapid detection of biothreat agents. |
| Description: | A small quantity of solution containing the suspected agent is placed in a well on the assay. A color change provides a positive or negative indication within 15 minutes. |
| Sensitivity/specificity: | The sensitivity of these assays varies from an order of magnitude below a fatal dose (ricin) to more than an order of magnitude above the infectious dose (anthrax). These devices are strictly screening assays, and the analyses are subject to error from the introduction of contaminants. Therefore, positive results need to be confirmed with standard microbiology assays, conventional immunoassays, or genome detection technology. |
| Biothreat agents: | Designed to identify 1 agent per assay and can currently identify 8 different threat agents (Y. pestis, F. tularensis, B. anthracis, V. cholerae, S. enterotoxin B, ricin, botulinum toxins, Brucella species) and 4 simulant agents (nonpathogenic agents used to evaluate detection systems for biothreat agents). |
| IT or DSS: | Neither when operated as a stand-alone unit. |
| Source of information: | No peer-reviewed information. Government report36 and other.10 |
| Contact information: | Currently produced by the Navy Medical Research Institute at Bethesda, Maryland. Similar devices have recently become commercially available through Environmental Technologies Corporation. |
| Name: | LightCycler™, Ruggedized Advanced Pathogen Identification Device (RAPID™) and Lightweight Epidemiology and Advanced Detection and Emergency Response System (LEADERS) |
| Purpose: | LightCycler™ is an ultra rapid PCR thermal cycler with built in fluorimetric detection system for real-time quantification of DNA samples. RAPID™ is a rugged, portable system that uses LightCycler™ technology for field detection of biothreat agents. LEADERS is a medical surveillance tool that provides real-time analysis of surveillance data coming from any number of sources (e.g., biosensor data from a system like RAPID™, syndromal surveillance data from emergency rooms, pharmacy sales data) to identify the presence of a biothreat agent. |
| Description: | LightCycler™ (available through Roche Diagnostics) can carry out 30 cycles in 6 minutes by using tiny glass capillary tubes for the sample and high-velocity hot and cold air. RAPID™, developed as a collaboration between Idaho Technologies and the U.S. Air Force, integrates LightCycler™ technology into a portable, rugged package that is capable of real-time on-line automated analysis of any nucleic acid. Per the manufacture, field personnel with minimal training can prepare the sample, place them in the instrument, and push one button for a result. RAPID™ also provides real-time monitoring via a Web browser so that experts at any remote location can monitor reactions in the field. LEADERS integrates data from any number of surveillance systems (including detectors) for analysis to detect outbreaks and displays these data in a variety of ways for the purposes of outbreak investigation and command and control. |
| Sensitivity/specificity: | This will be largely a function of the probes use. Per the manufacturer, RAPID™ is reported to be 99.9% specific. The sensitivity is set for each assay for half the infective dose (for example, the infectious dose of Foot and Mouth Disease is 10 virus particles; RAPID™'s sensitivity is set to detect 5 virus particles). |
| Biothreat agents: | Limited only by the available probes. Can assay for 10 unknown organisms per run. |
| IT or DSS: | LightCycler™ and RAPID™ are neither when operated as stand-alone units. LEADERS is an IT. |
| Source of information: | No peer-reviewed information. Other Web site.85 |
| Contact information: | Idaho Technology and Roche Diagnostics Idaho Technology 390 Wakwara Way Salt Lake City, UT 84108 Ph: 801-736-6354 http://www.idahotech.comhttp://biochem.roche.com/lightcycler/ |
| Name: | Luminometer Rapid Detector |
| Purpose: | Rapid, portable method of detecting live bacteria on animal carcasses. |
| Description: | The Luminometer is a handheld device currently being used on poultry and beef carcasses to measure the amount of live bacteria present. It provides an estimate of the total number of bacteria on the carcass by monitoring the amount of light emitted by the sample. Live bacteria release adenosine triphosphate, which leads to the production of light through its effects on luciferin and luciferase. Consequently, the intensity of the light measured by the Luminometer is directly proportional to the amount of live bacteria. The Luminometer is also proposed as a screening tool to detect bacteria in animal feed, although it has not been validated for this use. |
| Sensitivity/specificity: | Instrument sensitive to low levels of bacteria (1000 organisms) |
| Biothreat agents: | Not specified. |
| IT or DSS: | Neither when operated as a stand-alone unit. |
| Source of information: | Peer-reviewed article.82 |
| Contact information: | New Horizons Diagnostics Corporation 9110 Red Branch Road Columbia, MD 21045 Ph: 410-992-9357, ext. 235 or 232; Fax: 410-992-0328 |
| Name: | MicroArray of Gel Immobilized Compounds on a Chip (MAGIChip™) |
| Purpose: | Provides rapid screening of drug resistant mutations in Mycobacterium tuberculosis. |
| Description: | This system uses a microarray of gel-immobilized, fluorescence-labeled nucleic acid. Another application by the same manufacturer (unclear if it is currently available) includes a "Bacillus microchip" to detect B. anthracis, indicate whether it is alive or dead (DNA matches, but no RNA matches), and distinguish it from other related bacteria, such as B. thuringiensis, B. subtilis, and B. cereus. Another array (unclear if it is currently available) uses RNA probes to take advantage of the highly conserved 16S ribosome to provide a tentative taxonomic assignment of unknown bacterial pathogens, including novel or bioengineered organisms. |
| Sensitivity/specificity: | No information available. |
| Biothreat agents: | Limited by the probes used on the array. |
| IT or DSS: | IT |
| Source of information: | No peer-reviewed information. Government report36 and other.10, 96 |
| Contact information: | Developed by Argonne National Laboratory and the Russian Academy of Sciences. |
| Name: | MiniFlo |
| Purpose: | Rapid, portable detection of multiple biological agents using an innovative approach to flow cytometry. |
| Description: | Lawrence Livermore National Laboratory's (LLNL) MiniFlo portable flow cytometer is capable of rapid, simultaneous detection of several biological agents, including viruses, bacteria and proteins. Conventional flow cytometers detect variations in biological cells or their DNA by analyzing the scattered light pattern created when laser beams are directed at a single-file stream of cells in solution. In contrast, MiniFlo uses a unique approach in which the flow stream itself is used as the waveguide for the laser beams, thereby allowing faster, more accurate analysis. |
| Sensitivity/specificity: | According to the June 1998 volume of Lawrence Livermore National Laboratory's "Science and Technology Review," the rugged cytometer detected 87% of unknown biological agent simulants, including ones for anthrax and plague, with a false positive rate of 0.4% at the Dugway, Utah Field Trials in 199698. |
| Biothreat agents: | Y. pestis and B. anthracis, as well as other viruses, bacteria and proteins. |
| IT or DSS: | Neither when operated as a stand-alone unit. |
| Source of information: | No peer-reviewed information. Other.55, 98 |
| Contact information: | Lawrence Livermore National Laboratory http://www.llnl.gov/ |
| Name: | Nitric Oxide (NO) Sensor |
| Purpose: | Sensitive, rapid detection of biothreat agents. Currently under development. |
| Description: | The University of Maine and the Sensor Research and Development Corporation, under sponsorship from DARPA, are currently collaborating on the development of a "biowarfare breathalyzer" which would monitor NO levels as an early sign of exposure to biological pathogens. According to Stephen Morse of DARPA, exhaled NO levels are higher in symptomatic subjects, sometimes even before changes in self-reported symptoms. A compact, portable NO sensor may be a future option for detection of biological warfare agents. |
| Sensitivity/specificity: | No information available. |
| Biothreat agents: | No information available. |
| IT or DSS: | Neither when operated as a stand-alone unit. |
| Source of information: | No peer-reviewed information. Government Web sites.108, 109 |
| Contact information: | DARPA http://www.darpa.mil/darpatech99/presentations/dsopdf/dsobwd1.pdfhttp://www.darpa.mil/dso/thrust/bwd/advdiag/programs/maine.html |
| Name: | Optical fluorescence biosensor technique |
| Purpose: | A reagent-free technique for the rapid detection of biological toxins and pathogens. |
| Description: | Los Alamos National Laboratory has developed a rapid and highly sensitive optical technique based on the unique relationship between cell surface receptors and the biological toxins or pathogens that target them. The presence of the toxin is signified by a 2-color optical fluorescence change that occurs only after the aggregation of receptors within a bilayer membrane, which takes place after the binding of a toxin to such receptors. Changes in fluorescence are easily monitored in a flow cytometer. |
| Sensitivity/specificity: | Sensitivities down to less than 50 parts per trillion have been demonstrated. |
| Biothreat agents: | Capable of identifying specific protein toxins, such as the cholera toxin. |
| IT or DSS: | Neither when operated as a stand-alone unit. |
| Source of information: | Peer-reviewed report110 and other.111 |
| Contact information: | Los Alamos National Laboratory http://www.newswise.com/articles/1998/11/CHOLERA.LAL.html |
| Name: | Pyrolysis-gas Chromatography-ion Mobility Spectrometer (PY-GC-IMS) |
| Purpose: | Portable system for detection and identification of biological aerosols. |
| Description: | PY-GC-IMS is a rugged, fully self-contained system performing trigger, detection, and classification functions for biological aerosols. Samples are first combusted, then separated by gas chromatography, and finally analyzed using an ion mobility spectrometer; minimal post data processing is required. In recent testing at the Joint Field Trials-6 in DRES, Suffield, Alberta, Canada, the PY-GC-IMS was linked to the XM-2 aerosol concentrator and produced promising results in regards to samples containing simulants for protein toxins, bacteria, and sporulated bacteria. Biodetection takes approximately 3 minutes. It is also capable of chemical warfare agent detection. |
| Sensitivity/specificity: | No information available. Partially determined by the type of collection system used. |
| Biothreat agents: | Protein toxins, bacteria, and sporulated bacteria. |
| IT or DSS: | Neither when operated as a stand-alone unit. |
| Source of information: | No peer-reviewed information. Government report43 and other.97 |
| Contact information: | PY-GC-IMS was recently developed in a joint partnership between ECBC and the University of Utah. |
| Name: | RealTime BioSensor |
| Purpose: | Automated, rapid detection of a wide variety of biological pathogens. |
| Description: | The RealTime BioSensor is a portable detector currently being developed by a joint partnership of MesoSystems Technology Inc and DARPA. Combining Mesosystem's cell capture technology and a sensitive immuno-electrochemical sensor system, this technology offers the possibility of automated, real-time detection of biowarfare agents. The sensor can be linked to either fluidic or aerosol samplers. There is a digital output of results. |
| Sensitivity/specificity: | The current prototype has a high sensitivity, demonstrated by its ability to detect low numbers (100) of bio-contaminants in samples ranging from millimeters to liters. |
| Biothreat agents: | Includes airborne pathogens, E. coli 0157:H7 and Salmonella. |
| IT or DSS: | Neither when operated as a stand-alone unit. |
| Source of information: | No peer-reviewed information. Other Web site.112 |
| Contact information: | MesoSystems Technology, Inc. http://www.mesosystems.com/bs.htm |
| Name: | Sensitive Membrane Antigen Rapid Test (SMART™) and the Antibody-based Lateral flow Economical Recognition Ticket (ALERT) system |
| Purpose: | Rapid, portable antigen-antibody test for the detection of biothreat agents. |
| Description: | SMART™ is a self-contained, colorimetric, solid-phase immuno-filtration assay designed to be used in conjunction with an aqueous sample. Two types of SMART™ devices have been developed: one for detecting endospore-forming bacteria, and another for proteinaceous toxins or soluble antigens. Antibodies specific to the agent of interest are conjugated to colloidal gold particles. Two steps are involved in the process: first, the sample and reagent must be mixed manually, and, second, the sample must be applied to the test kit, which "tells" if antigen is present in the sample. The mixing step can be automated by using the ALERT system, which uses capillary action to move the sample up a membrane strip over embedded reagent. Both systems require antibody reagent development work before being ready for use. The presence or absence of the target antigen is detected by a small red dot that the user compares to a color chart. |
| Sensitivity/specificity: | When field tested during the Gulf War, the SMART™ system had an "alarmingly" high false positive rate thought secondary to contamination of sample fluid from fragments of filter fiber from the SMART™ kit and environmental silt particles.74 Apparently, after addressing these issues, no additional false positives were detected.74 Other diagnostic devices also manufactured by New Horizons include: Profile®-1, Total ATP, Bengal SMART™, Cholera SMART™, BengalScreen, and CholeraScreen. Profile®-1 is specifically designed for detection of bacterial contaminants of pork, beef, and poultry and is used by the United States Department of Agriculture. Bengal SMART™ is a dot-on-a-membrane test for V. cholerae O139 and, per the manufacturer, has a 99% sensitivity and 99% specificity. Cholera SMART™ is reported to be capable of detecting as few as 2x103 organisms of V. cholerae O1. BengalScreen and CholeraScreen are coagglutination tests for V. cholerae O139 and O1, respectively. They both have a sensitivity of 96% and a specificity of 94%. The tests take less than 10 minutes to run. |
| Biothreat agents: | One report said that the kits can detect B. anthracis or botulinum toxin in a sample in approximately 45 minutes. Another source reported that SMART™ kits were available for B. anthracis,S. enterotoxin B, Y. pestis, botulinum toxins, ricin, Venezuelan Equine Encephalitis, and Brucella species and that the response time was 5 to 15 minutes.84 |
| IT or DSS: | Neither when operated as a stand-alone unit. |
| Source of information: | Peer-reviewed article,82 government reports,36, 74 and other Web sites.83, 84 |
| Contact information: | New Horizons Diagnostics Corporation 9110 Red Branch Road Columbia, MD 21045 Ph: 410-992-9357, ext. 235 or 232; Fax: 410-992-0328 http://www.nhdiag.com/index6.html |
| Name: | SmartCycler® and GeneXpert™ |
| Purpose: | Real-time nucleic acid-based detection in laboratory and field situations. |
| Description: | The micro-fluidic circuits automatically process fluid samples, mix reagents, and purify DNA, which then undergoes PCR amplification, providing detection capabilities within an hour. Testable samples include blood, swabs, urine, cell cultures, food and industrial water. The dimensions of the processing block are: 12"W x 12"H x 10" L; weight 22 lbs (the entire kit with processing block, computer and accessories is 65lbs). Sixteen independently programmable channels can run tests without batching. The same manufacturer has developed GeneXpert™, which is designed to integrate automated sample preparation with SmartCycler® amplification and detection technology in a disposable cartridge format (it is unclear when GeneXpert™ will become commercially available). |
| Sensitivity/specificity: | The system is reported to be specific to 12 B. anthracis strains tested and able to detect 5 genome copies. |
| Biothreat agents: | Depends on available probes. |
| IT or DSS: | IT |
| Source of information: | No peer-reviewed information. Other Web site87 and other.86 |
| Contact information: | Developed by a partnership of USAMRIID and Lawrence Livermore National Laboratory and commercialized by Cepheid, Inc. Cepheid, Inc. 1190 Borregas Ave Sunnyvale, CA 94089-1302 http://www.cepheid.com/pages/contact.html |
| Name: | Tissue-Based Biological Sensor (TBBS) |
| Purpose: | Detection of biological pathogens using a technique that mimics the response of the body's own immune response. Capable of detecting new organisms, which have not been identified at the molecular level. |
| Description: | The CANARY project is developing a TBBS that uses B-cells to detect harmful agents in combination with high-density microarray technology. B-cells are part of the body's immune system and produce exquisitely sensitive antibodies in response to foreign antigens. In the CANARY project, B-cells are engineered to light up when they bind to a specific antigen, allowing the user to detect the event optically. A panel of B-cells, engineered to detect a variety of pathogens, is integrated into a microfluidic chip. When a potential pathogenic sample is introduced into the system it can be rapidly screened for the presence of any of the antigens specific to the engineered B-cells. Preliminary results using several prototype chips have demonstrated high specificity in addition to rapid response. In addition, this technology has been incorporated into prototype portable, handheld devices to determine the feasibility of their use for on-site support. Continued development is also focusing on increasing screening capability for a wider range of chemical and biological threats as well as determining the limits of sensitivity, false alarm rates, and the effects of interferents. |
| Sensitivity/specificity: | No information available. |
| Biothreat agents: | No information available. |
| IT or DSS: | Neither when operated as a stand-alone unit. |
| Source of information: | No peer-reviewed information. Other.113, 114 |
| Contact information: | DARPA/DSO 3701 North Fairfax Drive Arlington, VA 22203-1714 |
| Name: | Upconverting Phosphor Technology (UPT) |
| Purpose: | Rapid detection and identification of pathogens in the field while maintaining a high sensitivity and specificity. |
| Description: | This technology uses a number of rare earth compounds that, in crystal form, have the unique property of emitting a photon of visible light in response to absorbing 2 or 3 photons of lower-energy infrared light of the proper wavelength. Coating the crystals with antibody provides a highly identifiable signal, since no naturally occurring substances upconvert. Ten spectrally unique phosphors have been synthesized to date, making it possible to simultaneously probe with as many as 9 antibodies. Reportedly, a handheld sensor incorporating this technology allows portable, highly sensitive and rapid detection of multiple pathogens (e.g. bacteria, viruses, and toxins) simultaneously. After application of the liquid sample to a disposable test strip, the strip is inserted into the device and read. Results are ready in less than 5 minutes and samples can be archived at room temperature. Reading of the strip takes less than 30 seconds. A compact UPT-based flow cytometer, which is capable of simultaneously detecting and identifying up to 8 antigens, is also available. The system package weighs less than 30 pounds and occupies less than 1.6 cubic feet; however, future envisioned configurations will weigh less than 20 pounds and take-up less than 1 cubic foot while providing at least an order of magnitude more multiplexing capability. |
| Sensitivity/specificity: | The device can detect picogram levels of small (e.g., virus or toxin) target antigens in a sample of less than 1 ml. The goal for detection of spores and bacteria is sensitivity below 100 organisms/ml. Because no other materials in nature upconvert, there is no optical background and materials can be detected with high sensitivity even in dirty environmental samples. |
| Biothreat agents: | Based on the antibody probes used. |
| IT or DSS: | Neither when operated as a stand-alone unit. |
| Source of information: | No peer-reviewed information. Other Web site.115 |
| Contact information: | SRI International 333 Ravenswood Avenue Menlo Park, CA 94025-3493 http://www.sri.com/structure/chembio.html |
| Name: | Biological Aerosol Sentry and Information System (BASIS) |
| Purpose: | Early warning of airborne biological incidents for special events such as major sporting events and political meetings through a network of distributed sampling units deployed around the target area. |
| Description: | Aerosol samples are regularly retrieved from Distributed Sampling Units placed around the field, which collect, store and time-register the samples. The samples are periodically transported to a field laboratory for PCR analysis. If a biothreat agent is detected, authorities are immediately notified through a linked communication system. In addition to information regarding the location and identity of the biothreat agent, BASIS provides estimates on the level and duration of exposure. |
| Sensitivity/specificity: | No information available. |
| Biothreat agents: | Agents identifiable via PCR techniques (i.e., limited by availability of reagents). |
| IT or DSS: | IT |
| Source of information: | No peer-reviewed information. Government Web site.117 |
| Contact information: | Los Alamos National Laboratory PO Box 1663, MS F607 Los Alamos, NM 87545 |
| Name: | Biological Agent Warning Sensor (BAWS) and Joint Biological Point Detection System (JBPDS) |
| Purpose: | Detection of biological agents in aerosol samples. |
| Description: | BAWS uses a laser beam to illuminate a sample air stream and a detector records reflected photons. The current output is compared with that of the past 10 minutes. The sensing algorithm classifies the change as interferent (i.e. dust), unknown, or potential bio-aerosol (which triggers an alarm). The system is intended to detect biological agents in less than 1 minute and identifies the agents in less than 15 minutes. JBPDS is an integrated collection and identification system that uses BAWS as its detector. The next version, scheduled for fielding during FY06, will integrate advances in technologies to decrease size, weight, and power requirements, as well as to detect 26 agents. The systems have numerous sharp edges that can injure users and puncture protective gear. The man-portable units are so heavy that the 4-man crews experienced difficulties in transporting them. |
| Sensitivity/specificity: | During field-testing, the system experienced "many false positives" and "significant human factors deficiencies: operators in protective gear experienced difficulties, particularly in assembling and disassembling the system." |
| Biothreat agents: | 10 agents (not otherwise specified). |
| IT or DSS: | Neither when operated as stand-alone units. |
| Source of information: | Peer-reviewed article,91 government report,36 government Web site,118 other Web sites,39, 118, 119 and other.10 |
| Contact information: | BAWS was developed at the Massachusetts Institute of Technology Lincoln Laboratory. Lockheed Martin currently produces both BAWS and JBPDS. |
| Name: | Biological Integrated Detection System (BIDS) |
| Purpose: | Vehicle mounted continuous air sampler to determine the background distribution of aerosol particles. |
| Description: | Particles with diameters in the 2 to 10 micron range are concentrated and analyzed for biological activity, as indicated by the presence of adenosine 5'-triphosphate. Flow cytometry then separates and concentrates bacterial cells, and antibody-based tests are conducted for specific agents. Currently in use in the Washington, D.C. area (October 2001). |
| Sensitivity/specificity: | No information available. |
| Biothreat agents: | B. anthracis, Y. pestis, botulinum toxin A, and S. enterotoxin B. |
| IT or DSS: | Neither when operated as a stand-alone unit. |
| Source of information: | No peer-reviewed information. Government reports,36, 66 other Web site,120 and other.10 |
| Contact information: | Battelle 505 King Avenue Columbus, OH 43201 http://www.battelle.org |
| Name: | Canadian Integrated Biochemical Agent Detection System (CIBADS II) and 4WARN |
| Purpose: | Networked system designed to detect a broad spectrum of chemical or biological agents. |
| Description: | CIBADS/4WARN consists of a group of instruments integrated into a portable unit capable of remote, unattended operation. For biological agent detection, a positive response from FLAPS® (Fluorescence Aerodynamic Particle Sizer®) (see above under Particle Counters and Biomass Indicators) triggers the collection of a liquid sample for identification by an antigen/antibody system with an automated reader that can identify the agent. The system is radio-linked to a command and control unit which accepts incoming data from numerous sentries, plots the position of positive detection events on field maps, and couples the information to hazard assessment tools which can predict the downwind hazard and source of agent. 4WARN has been integrated into a commercial SUV (GMC Suburban) to reduce recognition by the public and was deployed in Washington, D.C. The goal was to provide a demonstration platform for the 4WARN detection system that would highlight the flexibility of configuration and the performance of the system in a mobile platform. |
| Sensitivity/specificity: | The system was operated in mobile modes (up to 50 mph) without significant degradation of performance. The impact of weather patterns on performance was also determined to be low. The exception was immediately after a thunderstorm, when the number of particles in the air rose dramatically and reduced the sensitivity of the system. |
| Biothreat agents: | Determined by the identification system used. |
| IT or DSS: | IT |
| Source of information: | No peer-reviewed information. Government Web site57 and other Web site.116 |
| Contact information: | Computing Devices Canada 1020 - 68th Avenue N.E. Calgary, Alberta Canada T2E 8P2 Ph: 403-295-6772 |
| Name: | Joint Biological Remote Early Warning System (JBREWS) |
| Purpose: | Network of sensors with communication links to a command post. |
| Description: | JBREWS consists of a network of sensors and communication links. Initially equipped with commercially available sensors, JBREWS is being configured so that improved biodetectors can be incorporated into the system as they become available. Currently, the system includes sample identification units (for continuous sampling of air and checking for biological agent using antibody tickets), SR-BSDS units (Short-Range Biological Standoff Detection System, see above under Particulate Counters and Biomass Indicators), a radio network for transmitting sensor data that is capable of rerouting transmissions that are blocked, and a sensor network command post that collects and processes data to determine if an outbreak has occurred. |
| Sensitivity/specificity: | No information available. Sensitivity and specificity will also be affected by the identification system. |
| Biothreat agents: | Determined by the identification system used. |
| IT or DSS: | IT |
| Source of information: | No peer-reviewed information. Other.121 |
| Contact information: | Developed in collaboration among Lawrence Livermore Laboratory, Johns Hopkins Applied Physics Laboratory and Los Alamos National Laboratory http://www.llnl.gov/str/Imbro.html |
| Name: | Joint Service Warning and Reporting Network (JWARN) |
| Purpose: | An automated nuclear, biological and chemical information system that is designed to integrate the data from detectors and sensors into the Joint Service command. |
| Description: | JWARN transfers data automatically and provides commanders with analyzed data for decisions. It is currently being developed by the DOD, and will be deployed with nuclear-biologic-chemical sensors in a wide variety of defense situations. |
| Sensitivity/specificity: | No information available. |
| Biothreat agents: | No information available. |
| IT or DSS: | IT |
| Source of information: | No peer-reviewed information. Government reports.36, 66 |
| Contact information: | DOD |
| Name: | Mobile Atmospheric And Sampling Identification Facility (MASIF) |
| Purpose: | Collection of aerosol samples for evidence of biothreat agents and transmission of findings to a central command location. |
| Description: | This system consists of a central command location and a network of self-contained aerosol detectors that continually monitor the particulate content of the air for evidence that a biowarfare agent may be present. After an alarm is sent to the central location indicating that such an event has occurred, samples are collected and analyzed using rapid assay methods in the Mobile Agent Identification Unit. MASIF was used during the Gulf War. |
| Sensitivity/specificity: | No information available. Sensitivity and specificity will also be affected by the identification system. |
| Biothreat agents: | Determined by the assay used for identification. |
| IT or DSS: | IT |
| Source of information: | No peer-reviewed information. Government Web site.122 |
| Contact information: | DRES http://www.dres.dnd.ca |
| Name: | Multi-Purpose Integrated Chemical Agent Alarm (MICAD) |
| Purpose: | Lightweight, automated nuclear-biologic-chemical (NBC) detection, warning and reporting system. |
| Description: | MICAD requires minimal operator input and is easily mounted in vehicles due to its small size and flexibility. Individual alerts can be transmitted from MICAD via soldier-worn pagers, while NBC reports are automatically prepared and transmitted over local and tactical Internets. Telemetry link radios integrate remote detection and area warning into MICAD's capabilities. Finally, Universal Interface Units allow the connection of 2 peripheral devices, such as detectors or alarms, making MICAD easily adaptable to other systems. |
| Sensitivity/specificity: | No information available. |
| Biothreat agents: | No information available. |
| IT or DSS: | IT |
| Source of information: | No peer-reviewed information. Other.123 |
| Contact information: | Developed by Lockheed Martin. http://www.lockheedmartin.com/manassas/nbc.html |
| Name: | Nuclear-biologic-chemical (NBC) Field Laboratory |
| Purpose: | To detect and identify any kind of biological warfare agent or any other agent of biological origin representing a health risk to soldiers on the battlefield. |
| Description: | The NBC Field Laboratory contains 4 subsystems, a Chemical Analysis Shelter, a Radiation and HazMat Shelter, 2 Biological Analysis Shelters and a Sampling Vehicle. These shelters are capable of several rapid identification methods including PCR-assisted fluorescence, high performance liquid chromatography, enzyme-linked immunosorbent assay, and gas chromatography/mass spectrometry. The lab has numerous safety features and is reported to be "safety level 3" compliant.124, 125 |
| Sensitivity/specificity: | No information available. |
| Biothreat agents: | No information available. |
| IT or DSS: | Neither when operated as a stand-alone unit. |
| Source of information: | No peer-reviewed information. Government report,36 other Web sites,39, 124 and other.10, 125 |
| Contact information: | Rheinmetall Landsysteme GmbH Henschelplatz 1 D-34127 Kassel, Germany Ph: ++49 561 801-5189 http://www.rheinmetall-detec.com/html/ravele/h-abc.htm |
| Name: | Portal Shield Air Base/Port Biological Detection System |
| Purpose: | Rapid, automated system that integrates data from multiple sites for outbreak detection. |
| Description: | The system is fully automated and composed of 6 or more sensor systems (typically 12-20 sensors per site) linked to a centralized command post computer that monitors the sensors, evaluates networked sensor data to determine if a bioterrorist attack has occurred, and alerts the operator in the event of biothreat detection. The command computer incorporates algorithms that use both aerosol count and meteorological data to determine the presence of a suspicious aerosol cloud. In less than 25 minutes, it can simultaneously detect, identify and report 8 different biological agents. |
| Sensitivity/specificity: | The algorithm looks for a significant increase in at least 2 sensors before it will sound an alarm, giving the system a theoretical false positive rate of 0.25%. In practice, after having gone through over 10,000 assays, the Portal Shield system has not had any false positives. |
| Biothreat agents: | Eight agents, not otherwise specified. |
| IT or DSS: | IT |
| Source of information: | No peer-reviewed information. Government report.66 |
| Contact information: | Designed by the DOD. |
For each system discussed in the Other Diagnostic Systems section of Chapter 3, we present selected abstracted information including its name, the purpose, a description, and information about the system's diagnostic accuracy.
| Name: | BloodLink |
| Purpose: | To decrease diagnostic test-ordering by clinicians |
| Description: | BloodLink provides a computerized test-ordering form with a reduced number of tests (15 compared with 178 on the old form). To order a rarely indicated test, the clinician can type the name of the required test in a section at the bottom of the form. Clinicians can use the BloodLink to suggest diagnostic tests recommended by clinical practice guidelines. |
| Diagnostic accuracy: | No information available. |
| IT or DSS: | DSS |
| Source of information: | Peer-reviewed article.176 |
| Contact information: | Institute of Medical informatics Erasmus University Rotterdam The Netherlands |
| Name: | Clinical DSS for detection and respiratory isolation of tuberculosis (TB) patients |
| Purpose: | To automate the detection and respiratory isolation of patients with positive cultures and chest X-rays suspicious for tuberculosis. |
| Description: | Each time a new patient with an abnormal chest radiograph report is entered into the electronic medical record, a natural language processing system triggers a medical logic module to check the patient's hospital location (i.e., if they are assigned to an isolation bed), evaluate the patient's immune status (i.e., checks an algorithm of laboratory and pharmacy data that suggests HIV infection). An alert message is sent via e-mail to the hospital epidemiologist indicating the date of the abnormal chest X-ray, type of abnormality, level of suspicion of TB, presence of immunocompromised status, and isolation status. This system operates 24 hours per day and requires no additional data entry. |
| Diagnostic accuracy: | In a retrospective analysis, 171 adult culture positive TB inpatients were used to assess the accuracy of the system: without the DSS 51% (45 of 88) patients were appropriately isolated compared with 75% (62 of 83) patients with the DSS.174 The system would have erroneously recommended isolation of 27 of 171 patients (false positives). In a prospective analysis, clinicians adhering to the hospital's isolation policy correctly and promptly isolated 70% (30 of 43) of patients with TB. The DSSs did not identify 21 of these patients (false negatives). However the DSS identified 4 patients not identified by the clinicians.174 Note, since this DSS identifies patients with positive X-rays, it can never be 100% sensitive since it cannot identify cases of X-ray-negative TB. |
| IT or DSS: | DSS |
| Source of information: | Peer reviewed article.174 |
| Contact information: | 630 W. 168th Street New York, NY 1002 |
| Name: | Computer Program for Diagnosing and Teaching Geographic Medicine |
| Purpose: | To provide a differential diagnosis of infectious diseases matched to 22 clinical parameters for a patient; also to provide general information about infectious diseases, anti-infective agents and vaccines. |
| Description: | The database is limited to infectious diseases but does not include slow viral illnesses and a number of self-defined and "obvious" conditions such as otitis externa.168 The database includes all critical biological agents (Table 2.1). The user is first requested to indicate the country of disease origin and is then presented with a list of 22 clinical parameters which the user indicates with a + or - sign as being present or absent. The system uses a series of branching questions -- for example, if the user indicates that a rash is present, the system asks additional questions about the nature and location of the skin lesions. A Bayesian matrix processes user input and a list of possible diagnoses is presented in order of likely probability. Additional information about each disease and antimicrobial agents is also provided. For example, a user may request a list of all parasitic disease likely to be acquired in Togo from mosquitoes. |
| Diagnostic accuracy: | In a study to evaluate the diagnostic accuracy of this system, 6 infectious disease specialists (blinded to the patients' actual diagnoses) were asked to record all positive and negative clinical data for 295 consecutive patients with established diagnoses and 200 hypothetical cases. The computer program correctly identified 75% (222 of 295) actual cases and 64% (128 of 200) hypothetical cases. The clinical diagnosis was included in the computer differential diagnosis list in 94.7% of cases. Among the cases included in this evaluation, several were for biothreat-related organisms: anthrax, brucellosis, cholera, cryptosporidiosis, Hantavirus respiratory distress syndrome, Lassa fever, Plague, Q fever, Rocky Mountain spotted fever, shigellosis, and tularemia. However, this system was only tested on cases for which the diagnosis was known; therefore, there is no information on how it would perform for cases with unknown outcomes. |
| IT or DSS: | DSS |
| Source of information: | Peer reviewed article.168 |
| Contact information: | Dept. of Microbiology Tel-Aviv Medical Center 6 Weitzman Street Tel Aviv 64239, Israel |
| Name: | DERMIS |
| Purpose: | To provide a differential diagnosis of skin lesions |
| Description: | DERMIS is a computerized skin disease diagnosis assistance system designed for use by general practitioners. It matches the signs of a new case with a database of signs from 5203 cases with 221 separate diagnoses collected from a single dermatology clinic. The user is led through the process of describing a rash or lesion in a structured way. Once the signs are entered, the program produces a differential diagnosis list based on these signs and a Bayesian analysis. |
| Diagnostic accuracy: | In a 1992 evaluation of DERMIS using descriptions of lesions by a dermatologist, the system correctly diagnosed a lesion 76% of the time and included the correct diagnosis among its top 3 choices 95% of the time (out of a total of 5203 cases).177 In a subsequent evaluation, DERMIS gave the correct diagnosis 51% of the time when given a description of a skin lesion by general practitioners and 80% when given a description by dermatologists (out of 100 cases). It listed the correct diagnosis in the top 3 of its differential list 70% of the time when given a description by general practitioners and 93% of the time for dermatologists.178 |
| IT or DSS: | DSS |
| Source of information: | Peer reviewed articles.177, 178 |
| Contact information: | Firm 3, St. John's Institute of Dermatology St. Thomas' Hospital Lambeth Palace Road London, SE1 7EH |
| Name: | Fuzzy logic program to predict source of bacterial infection |
| Purpose: | To use age, blood type, gender and race to predict the etiology bacterial infections. |
| Description: | The authors developed and evaluated a fuzzy logic program that predicts class of bacterial organism responsible for infection based on age, blood type, gender, and race. A dataset of demographic variables for 187 patients with a known bacterial infection was randomly divided into training data (155 patients) and test data (32 patients). 159 rules were kept from the training set and applied to the test set. Using the training data, a set of fuzzy rules was generated to model the system. The system generated 4 classifications of infections: "staphylococci" (S. aureus and S. epidermidis), "streptococci" (S. pneumoniae, groups B and D streptococci), "E. coli", and "non-E. coli gram negative rods" (Klebsiella, Serratia, Proteus, Morganella, Prevotella, Pseudomonas, and Bateroides species). |
| Diagnostic accuracy: | The program was able to correctly classify 27 of 32 patients into 1 of these 4 groups based on demographic data alone. Note, because the program was established from the data of patients with infections and then tested on patients with known infections, it is difficult to know how this system performs in detecting those patients without 1 of the 4 disease categories. |
| IT or DSS: | DSS |
| Source of information: | Peer-reviewed article.171 |
| Contact information: | School of Science and Health Philadelphia University Philadelphia, PA 19144 Ph: 215-951-2664 |
| Name: | Global Infectious Disease and Epidemiology Network (GIDEON) |
| Purpose: | Provide differential diagnoses for patients with diseases of infectious etiology. |
| Description: | GIDEON is a computer program with an extensive infectious disease database that uses Bayes' theorem to generate a list of possible diseases for a given case. No information is available regarding which biothreat agents are in the GIDEON database. |
| Diagnostic accuracy: | The diagnostic accuracy of GIDEON was compared with that of medical house officers admitting 86 febrile adults to the Boston Medical Center. The house officers listed the correct diagnosis first in their admission note 87% (75/86) of the time compared with 33% (28/86) for GIDEON. |
| IT or DSS: | DSS |
| Source of information: | Peer reviewed article.169 |
| Contact information: | C.Y. Informatics; Ramat Hasharon; Israel |
| Name: | Neural Network for Diagnosing Tuberculosis (TB) |
| Purpose: | To use an artificial neural network that incorporates clinical and radiographic information to predict active pulmonary TB at the time of presentation at a health care facility so that patients may be appropriately isolated. |
| Description: | Several episodes of nosocomial transmission of Mycobacterium tuberculosis have been reported as a result of missed or delayed diagnosis, usually as a result of failure to consider the diagnosis, atypical radiographic presentation, delayed recognition of drug resistance, and lack of adequate respiratory protection.175 Neural networks are computation systems that process information in parallel and excel in tasks involving pattern recognition. Authors from the State University of New York at Buffalo used the medical record information of 563 patients isolated for suspicion of TB to create a general regression neural network which they tested on 119 patients. The data input included demographic variables (e.g., age and purified protein derivative (PPD) test status), symptoms (e.g., weight loss, fever, night sweats and cough), and radiographic findings (e.g., presence and location of infiltrates, cavities, and pleural effusions). |
| Diagnostic accuracy: | For the derivation set of 563 patients, the neural network achieved a sensitivity of 100% (95% CI: 91% to 100%) and specificity of 72% (95% CI: 65% to 77%). Clinicians correctly diagnosed active pulmonary TB in 22 of 47 patients (sensitivity of 47% (95% CI: 32% to 62%) and specificity of 75% (95% CI: 71% to 79%). For the validation set of 119 patients, the neural network correctly identified 11 of 11 patients with active TB (sensitivity of 100% (95% CI: 72% to 100%) and specificity of 69% (95% CI: 61% to 78%). Clinicians correctly diagnosed 7 of 11 patients with active TB (sensitivity of 64% (95% CI: 31% to 89%) and specificity of 79% (95% CI: 72% to 87%). The interpretation of these results are limited by the initial use of only those patients who were isolated for suspected active pulmonary TB -- therefore, the neural network does not include patients whom clinicians incorrectly chose not to isolate. |
| IT or DSS: | DSS |
| Source of information: | Peer reviewed article.175 |
| Contact information: | The artificial neural network described here is available at: http://bgrant.med.buffalo.edu/activetb/ |
| Name: | PNEUMON-IA |
| Purpose: | To diagnose community-acquired pneumonia from clinical, radiologic and laboratory data. |
| Description: | Information about cases is input into the DSS with a knowledge base that includes 22 pneumonia etiologies (including Q fever). For each of the 22 possible diagnoses, the system assigns a probability to each of the following 8 labels for each case: impossible, almost impossible, slightly possible, somewhat possible, possible, quite possible, very possible, and sure. |
| Diagnostic accuracy: | Reports of 76 cases of adult community-acquired pneumonia were analyzed by PNEUMON-IA and by 5 clinician experts. Ten of these 76 cases had confirmed diagnoses from microbiology data. The DSS correctly identified the diagnosis in 4 of these 10 cases compared with between 3 and 6 cases for the clinician experts. |
| IT or DSS: | DSS |
| Source of information: | Peer-reviewed article.179 |
| Contact information: | Departament d'Informática Biomèdica Institut Municipal d' Investigació Mèdica Passeig Marítim 25-29 08003 Barcelona, Spain Fax: 343-485-49-52 |
| Name: | Texas Infectious Disease Diagnostic DSS |
| Purpose: | To provide a weighted differential diagnosis based on manually entered patient information. |
| Description: | Users manually enter case-specific information, which is compared with a knowledge base containing 223 infectious and parasitic diseases with associated symptoms, treatment protocols, geographic associations, and occupational associations, among other features. The user sets a "closeness-of-fit" parameter that determines how tight the relationship between cases signs/findings and diseases must be before a disease can become a candidate for the differential diagnosis list. On average, it took 3 minutes to enter relevant patient information into the DSS. |
| Diagnostic accuracy: | Records of 342 cases of brucellosis were obtained from the Texas Department of Health. Ninety-eight patients had been diagnosed more than 11 days after presentation and were considered missed diagnoses. In 86 of the 98 patients defined as missed diagnoses, the DSS listed brucellosis in the top 5 diagnoses on the differential diagnosis list, and in 69 of these 98 patients, brucellosis was the only disease suggested. The DSS missed the diagnosis in 12 of 98 patients. The mean number of days to suspect the correct diagnosis without the DSS was 17.9 days and with the DSS was 4.5 days (an improvement of 12.9 days; p=0.0001). |
| IT or DSS: | DSS |
| Source of information: | Peer-reviewed article.173 |
| Contact information: | Texas A&M University |
| Name: | ABIX |
| Purpose: | To advise doctors with suggested management plans for various infectious diseases. |
| Description: | ABIX is an antibiotic information database designed to help non-specialist physicians treat patients with infectious diseases. The system runs on a PC that communicates with a server either remotely via modem or locally via a network connection. As of 2000, the database contained 136 different drugs, listed by generic name. However, the database is continually updated. Dosage, drug administration, and risk information is provided along with the recommended antibiotics. |
| Evaluation data: | A survey of 50 doctors from a pilot evaluation of ABIX reported that the system is easy to understand and use. In addition, 85% indicated that the information included and the system's classifications were satisfactory.217 |
| IT or DSS: | DSS |
| Source of information: | Peer-reviewed article.217 |
| Contact information: | Institute of Social and Preventive Medicine, Athens Department of Hygiene and Epidemiology, Medical School, University of Athens Evagelismos Hospital, Athens |
| Name: | Antibiotic Assistant™ |
| Purpose: | To provide physicians with patient- and disease-specific decision support on antibiotic treatments. |
| Description: | An easy-to-use system, Antibiotic Assistant™ uses an inference engine and syndrome-specific evidence-based knowledge bases to help physicians determine the appropriate dose, duration, and choice of antibiotics. Only a limited number of syndromal knowledge bases are currently integrated into Antibiotic Assistant™, including one on meningitis. TheraDoc plans to release 2 to 3 new modules each year. |
| Evaluation data: | Not available. |
| IT or DSS: | DSS |
| Source of information: | No peer-reviewed information. Other Web site.218 |
| Contact information: | TheraDoc 127 South 500 East, Suite 600 Salt Lake City, UT 84102 Ph: 801-415-4400; Fax: 801-415-4444 http://www.theradoc.com/ |
| Name: | Pneumonia Therapy Advisor (PTA) |
| Purpose: | To advise ICU physicians on the diagnosis and initial treatment of ventilator-associated pneumonia. |
| Description: | This expert system is designed to support clinicians in prescribing empirical therapy to mechanically ventilated patients with pneumonia in the ICU. Using the C2000-Eclipsys commercial clinical information system as its foundation, it uses both decision analytic and Bayesian models. Diagnostic recommendations are made based on variables such as the patient's symptoms and signs, duration of hospitalization and ventilation, and relevant laboratory data. Cost, side effects, and expected efficacy are taken into consideration for therapy recommendations. A total of 32 different therapy regimes are offered. |
| Evaluation data: | A comparison of PTA's treatment recommendations for 12 ICU patients with those of an infectious disease specialist demonstrated that 100% (12 out of 12) of the model's choices were considered "acceptable" or "second-best choice." However, in 66% (8 out of 12) of the cases, PTA recommended a therapy that covered more pathogens than the expert-recommended therapy, due to the model's lack of knowledge regarding the broadness of the antimicrobial spectrum.220 Preliminary evaluations suggest that this issue may be partially alleviated with the addition of a function that only includes pathogens with a posterior marginal probability of 31% or greater, as well as a utility function that discourages the prescription of broad-spectrum antibiotics.219 |
| IT or DSS: | DSS |
| Source of information: | Peer-reviewed articles.219, 220 |
| Contact information: | Department of Computer Science, Utrecht University, The Netherlands Department of Internal Medicine, University Medical Centre Utrecht, The Netherlands |
| Name: | QID |
| Purpose: | To provide physicians with patient- and disease-specific decision support on antibiotic treatments. Uses an inference engine and syndrome-specific evidence-based knowledge bases. |
| Description: | The QID program runs on Iliad's inference engine and is able to calculate a differential diagnosis from an infectious disease knowledge base before culture results are available. Once it has determined the differential diagnosis, QID then calculates the maximum "Good days of life saved" (GDS) for each of the most likely disease/organisms assuming optimal antibiotic coverage. QID uses local ravelersid epidemiology from the last 15 months to determine the effectiveness of each antibiotic for each of the most likely disease/organisms. The basic QID algorithm formula is: Antibiotic GDS = Sum across most likely diseases/organisms x Optimal GDS score x Antibiotic Susceptibility. The algorithm generates a list of antibiotics ordered by GDS and displayed along with the toxicity and cost per 24 hours of treatment for each drug. |
| Evaluation data: | To test whether physicians' would make more appropriate antibiotic choices with the aid of QID, University of Utah physicians were asked to evaluate 4 infectious disease cases abstracted from existing medical records of infectious disease cases. Immediately after their initial review and selection of antibiotic therapy for each case, participants were presented with QID's antibiotic recommendations for the same case to determine if this information would change the physician's initial drug choice. QID had a greater impact on the most difficult cases but statistically improved scores overall by a mean increase of 6.8% correct (p < 0.001). |
| IT or DSS: | DSS |
| Source of information: | Peer-reviewed article.221 |
| Contact information: | Sunquest Information Systems, Inc. Salt Lake City, UT |
| Name: | Rabin Medical Center Antibiotic DSS |
| Purpose: | To assist in the selection of empiric antibiotics in suspected moderate to severe bacterial infections. |
| Description: | This DSS combines site-specific information regarding pathogen prevalence and susceptibility to antibiotics, with prediction models derived from large pools of data and validated in other sites. It is targeted at cases involving inappropriate empirical antibiotic treatment, notably ones in which there is a high risk for infections caused by 7 pathogens (S. aureus, coagulase-negative staphylococci, Pseudomonas aeruginosa, Acinetobacter sp., enterococci, anaerobes and Candida sp.) Patient-specific data used by the DSS include demographic information, underlying disorders, presentation of the infectious episode, and laboratory test results. |
| Evaluation data: | A study comparing the recommendations of the system with those of a physician for 219 patients with positive cultures or serological tests demonstrated that the system recommended treatment to which the pathogen was shown to be susceptible in 77% of patients, compared with 58% for physicians. The DSS made inappropriate drug recommendations for 23% of the patients (compared with 42% for the physicians) and recommendations for antibiotics that were either unneeded or too broad-spectrum in 11% of the patients (compared to 15% for the physicians). Use of the system would have reduced the rate of inappropriate treatments in patients with a known pathogen by 19%.222 |
| IT or DSS: | DSS |
| Source of information: | Peer-reviewed article.222 |
| Contact information: | Rabin Medical Centre, Israel Tel-Aviv University, Israel |
| Name: | Eindhoven Automated Knowledge Acquisition Tool |
| Purpose: | To provide decision support to health care workers in clinical care and emergency care environments. |
| Description: | The developers designed and implemented a knowledge acquisition tool for the ICU consisting of: 1) a graphical knowledge acquisition tool, 2) tools that perform logical and semantic tests on proposed guidelines, 3) an electronic medical record of patient information, 4) an expert system that reminds ICU health care workers of inconsistencies between a treatment plan and implemented guidelines. Physicians enter the guidelines using the knowledge acquisition tool. The guidelines are checked for accuracy and exported to the reminder system used in daily practice. |
| Evaluation data: | ICU physicians used the knowledge acquisition tool to enter 58 guidelines into the reminder system's knowledge base. These guidelines were tested on a database of 803 previously admitted patients. During this test, 27 of the 58 guidelines generated at least 1 reminder; a total of 406 reminders were generated. Of the 406 reminders, 356 (88%) were issued correctly and 50 (12%) were considered false alarms.223 The false alarms were attributed to lack of specificity in the underlying guideline. This realization led to improvements in the guidelines. |
| IT or DSS: | DSS |
| Source of information: | Peer reviewed article.223 |
| Contact information: | Eindhoven University of Technology, Room EH 3.08 P.O. Box 513 5600 MB Eindhoven The Netherlands |
| Name: | ICONS |
| Purpose: | To provide rapid antibiotic recommendations for ICU patients with hospital-acquired infections. |
| Description: | ICONS is a case-based reasoning system that provides therapy advice to physicians before microbiological data are available. It uses the patient's medical background and known resistances to determine a spectrum of possible agents. The system retrieves the case histories of former patients with a similar presentation to the current patient. These similar former cases and prototypes are retrieved from the system's hierarchically generalized storage base, and the effective treatment strategies from these cases are adapted to the current patient's medical situation. Additional medical knowledge is represented with a context-sensitive background knowledge base, which can be easily revised through a programmed knowledge acquisition tool. For each of the therapy options presented, information on side effects and costs may be obtained; dosage is provided once a particular therapeutic agent is prescribed. ICONS was developed using a Macintosh computer and requires operating system 7.0 or higher, 4 Mb RAM and hard disk occupation of less than 10 Mb. According to available literature, it cannot yet be transferred to other operating systems. |
| Evaluation data: | Not available. |
| IT or DSS: | DSS |
| Source of information: | Peer-reviewed articles.224, 225 |
| Contact information: | Computer Centre of the Medical Facility, Ludwig-Maximilians University of Munich Institute of Anaesthesiology, Ludwig-Maximilians University of Munich, Germany |
| Name: | Intelligent Decision Aid System (IDEAS) for ICU and IDEAS for NICU |
| Purpose: | To make treatment recommendations for new patients admitted to the ICU/NICU based on similarity to former ICU/NICU patients. |
| Description: | For each new patient admitted to the ICU or NICU, this system provides a list of the 10 closest matching patients from a database of thousands of adult or neonatal intensive care patients. Each entry in the database contains 98 fields of clinical and administrative information, with up to 7 medical diagnoses and multiple lines for procedural information. Certain fields are weighted more heavily in the matching process. The expert-system reasoning shells used by the developers of this system include ART-IM, Version 2.5 (Inference Corporation, El Segundo, CA) and The Easy Reasoner™ (The Haley Enterprise, Sewickley, PA). |
| Evaluation data: | After a 3-week study involving a prototype version of IDEAS for ICUs and 27 patients, 5 evaluation forms submitted by physicians indicated that the system would be beneficial to clinicians, while 22 said that no benefit was currently foreseeable. Comments regarding improvements were incorporated into a newer version of the system. In a separate preliminary study, IDEAS for NICUs was rated highly in terms of usability by 5 neonatologists but not considered very clinically useful in its current form. |
| IT or DSS: | DSS |
| Source of information: | Peer-reviewed article.226 |
| Contact information: | School of Information Technology and Engineering, University of Ottawa, Canada Department of Systems and Computer Engineering, Carleton University, Canada Division of Neonatology, 401 Smyth Rd, Ottawa, ON, Canada |
| Name: | Severity Scores for Community-acquired Pneumonia |
| Purpose: | To classify patients using a published prediction rule by calculating a severity score based on laboratory, radiographic, and historical data (from electronic medical records) to make severity-based management recommendations for patients with community-acquired pneumonia. |
| Description: | This system automatically determines a severity score for patients with community-acquired pneumonia using a published prediction rule for the prognosis of community-acquired pneumonia. It uses the MedLEE medical language processor for extraction and encoding of electronic medical records. Severity score calculation is based on a total of 18 variables, including laboratory, radiographic and historical data collected from electronic discharge summaries. The rule assigns patients to 1 of 5 risk categories and makes treatment recommendations based on risk class. |
| Evaluation data: | In comparison with a reference standard obtained manually by an independent expert for 79 patients with community-acquired pneumonia cases, the system achieved an accuracy of 93%, a sensitivity of 92%, and a specificity of 93% for processing discharge summaries. For chest x-rays, it demonstrated an accuracy of 96%, a sensitivity of 87%, and a specificity of 98%. It was 80% accurate in assigning the exact risk class, with the remaining 20% differing by only 1 class, and 85% accurate with vital sign values.216 |
| IT or DSS: | DSS |
| Source of information: | Peer-reviewed article.216 |
| Contact information: | Department of Computer Science Queens College CUNY |
| Name: | ABDX |
| Purpose: | To aid the Naval Independent Duty Corpsman in the diagnosis and treatment of patients with acute abdominal pain aboard Navy ships. |
| Description: | ABDX is designed to help health care personnel at sea make appropriate decisions regarding patients with abdominal pain. It was originally designed for use with the Tektronix 4051 machine, but was modified to run on an IBM PC or compatible computer. Gynecological data was incorporated into the program in FY89, thereby leading to ABDX Version 3.0. |
| Evaluation data: | None available. |
| IT or DSS: | DSS |
| Source of information: | No peer-reviewed information. Other.227 |
| Contact information: | Naval Submarine Medical Research Laboratory |
| Name: | Columbia-Presbyterian Clinical Event Monitor |
| Purpose: | To monitor a patient database in order to generate alerts, interpretations, and screening messages for clinicians throughout the Columbia-Presbyterian Medical Center. |
| Description: | This automated DSS, which has been in clinical use since March 1992, is triggered by clinical events throughout the medical center by reading a centralized patient database of administrative data, laboratory results, radiology findings (via natural language processing - See Section 3.4.3 for additional information on this feature), medication orders, and text reports from most ancillary departments. Based on the events and data, the system generates emergent alerts (about 50 per day), informational interpretations (about 2000 per day), and screening messages for clinical research, quality assurance, and administration (e.g., billing rules). The system runs for all the medical center's patients, and all clinicians have access to the generated messages. There are about 100 rules at present, which concentrate on laboratory alerts, lab-drug interactions, health maintenance protocols, tuberculosis follow-up, administrative rules, and screening messages for research and quality assurance. |
| Diagnostic accuracy: | Formal studies are in progress. No information is currently available. |
| IT or DSS: | DSS |
| Source of information: | Peer-reviewed article.157 |
| Contact information: | Department of Medical Informatics Columbia-Presbyterian Medical Center 161 Fort Washington Avenue, AP-1310 New York, NY 10032 |
| Name: | Déjà Vu |
| Purpose: | The automatic recognition and monitoring of time-dependent medical scenarios. |
| Description: | The Déjà Vu system is a conceptual model for representing physiological processes. It uses temporal reasoning to model medical scenarios such as the management of mechanical ventilation and pulmonary edema. The system takes information from a number of sources (e.g. sensor, laboratory, and clinical staff data) to generate recognition of the patient's clinical scenario. On the basis of the scenario, an action plan is generated. The system is iterative and changes with incoming data. |
| Evaluation data: | None available. |
| IT or DSS: | DSS |
| Source of information: | Peer-reviewed article.228 |
| Contact information: | Institut National de la Santé et de la Recherche Médicale, France Université de Technologie de Compiégne, France |
| Name: | Emergency Medical Alert Network (EMAN) |
| Purpose: | To provide early detection of adverse health events and dissemination of health information regarding disease treatment and personal protection. |
| Description: | EMAN is a Web-based program developed by San Diego County, California to aid in surveillance and detection of adverse health events and to disseminate appropriate health information to public health and safety agencies. The system was developed using the CDC's HAN architectural standards; however, state and local resources funded it. Participants in the San Diego County EMAN include: directors of hospital emergency departments, directors of clinical microbiology laboratories, physicians specializing in infectious disease, the Medical Examiner's Office, Emergency Medical Services (EMS) Office and 9-1-1 Center, San Diego County Department of Environmental Health Services, and the San Diego County Veterinarian's office. EMAN is also linked to the San Diego Quality Assurance Network, a live, real-time Wide-Area-Network hospital resource status indicator that provides information regarding emergency department and ICU saturation and hospital bed availability. Standard alerts are sent via e-mail; whereas, more urgent matters are sent via fax. Recent alerts will also be posted on the Web site. In a major emergency, the network would include more rapid communication methods such as mass paging (it is unclear whether or not this is currently under development). |
| Evaluation data: | None available. |
| IT or DSS: | DSS |
| Source of information: | Other Web site229 and other.230 |
| Contact information: | Emergency Medical Alert Network (EMAN) County of San Diego, Health and Human Services Agency Community Epidemiology Division MS: P511C-A P.O. Box 85222 San Diego, CA 92186-5222 Ph: 619-515-6620; Urgent ph: 858-565-5255; Fax: 619-515-6644 E-mail: epidivhe@co.san-diego.ca.us |
| Name: | Health Evaluation through Logical Processing (HELP) |
| Purpose: | To monitor a patient database in order to generate alerts, interpretations, and screening messages for clinicians. |
| Description: | Please see text (Chapter 3, Management and Prevention Systems). |
| Evaluation data: | Please see text (Chapter 3, Management and Prevention Systems). |
| IT or DSS: | DSS |
| Source of information: | Peer-reviewed articles.156, 158, 180-215 |
| Contact information: | LDS Hospital 8th Avenue and C Street Salt Lake City, Utah 84143-0001 Ph: 801-408-1100 |
| Name: | MEDTRAK |
| Purpose: | To track the location o |
