Universität Karlsruhe (TH), Institut für Programmstrukturen und Datenorganisation (IPD), Am Fasanengarten 5, 76128, Karlsruhe, Germany. witte@ipd.uni-karlsruhe.de
The development of text analysis systems targeting the extraction of information about mutations from research publications is an emergent topic in biomedical research. Current systems differ in both scope and approach, thus preventing a meaningful comparison of their performance and therefore possible synergies. To overcome this evaluation bottleneck, we developed a comprehensive framework for the systematic analysis of mutation extraction systems, precisely defining tasks and corresponding evaluation metrics, that will allow a comparison of existing and future applications.