Format

Send to

Choose Destination
See comment in PubMed Commons below
Biomed Res Int. 2016;2016:4248026. doi: 10.1155/2016/4248026. Epub 2016 Jan 28.

ChemTok: A New Rule Based Tokenizer for Chemical Named Entity Recognition.

Author information

1
Computer Engineering Department, Eastern Mediterranean University, Famagusta, Northern Cyprus, Mersin 10, Turkey.
2
Information Technology Department, Eastern Mediterranean University, Famagusta, Northern Cyprus, Mersin 10, Turkey.

Abstract

Named Entity Recognition (NER) from text constitutes the first step in many text mining applications. The most important preliminary step for NER systems using machine learning approaches is tokenization where raw text is segmented into tokens. This study proposes an enhanced rule based tokenizer, ChemTok, which utilizes rules extracted mainly from the train data set. The main novelty of ChemTok is the use of the extracted rules in order to merge the tokens split in the previous steps, thus producing longer and more discriminative tokens. ChemTok is compared to the tokenization methods utilized by ChemSpot and tmChem. Support Vector Machines and Conditional Random Fields are employed as the learning algorithms. The experimental results show that the classifiers trained on the output of ChemTok outperforms all classifiers trained on the output of the other two tokenizers in terms of classification performance, and the number of incorrectly segmented entities.

PMID:
26942193
PMCID:
PMC4749772
DOI:
10.1155/2016/4248026
[Indexed for MEDLINE]
Free PMC Article
PubMed Commons home

PubMed Commons

0 comments
How to join PubMed Commons

    Supplemental Content

    Full text links

    Icon for Hindawi Publishing Corporation Icon for PubMed Central
    Loading ...
    Support Center