Display Settings:

Format

Send to:

Choose Destination

    PLoS Comput Biol. 2006 Sep 8;2(9):e118. Epub 2006 Jul 27.

    Imitating manual curation of text-mined facts in biomedicine.

    Rodriguez-Esteban R, Iossifov I, Rzhetsky A.

    Department of Electrical Engineering, Columbia University, New York, New York, United States of America.

    Text-mining algorithms make mistakes in extracting facts from natural-language texts. In biomedical applications, which rely on use of text-mined data, it is critical to assess the quality (the probability that the message is correctly extracted) of individual facts--to resolve data conflicts and inconsistencies. Using a large set of almost 100,000 manually produced evaluations (most facts were independently reviewed more than once, producing independent evaluations), we implemented and tested a collection of algorithms that mimic human evaluation of facts provided by an automated information-extraction system. The performance of our best automated classifiers closely approached that of our human evaluators (ROC score close to 0.95). Our hypothesis is that, were we to use a larger number of human experts to evaluate any given sentence, we could implement an artificial-intelligence curator that would perform the classification job at least as accurately as an average individual human evaluator. We illustrated our analysis by visualizing the predicted accuracy of the text-mined relations involving the term cocaine.

    PMID: 16965176 [PubMed - indexed for MEDLINE]

    PMCID: PMC1560402

    Supplemental Content

    Click here to read Click here to read