Developing a disease outbreak event corpus

Mike Conway, Ai Kawazoe, Hutchatai Chanlekha, Nigel Collier

Research output: Contribution to journalArticlepeer-review

10 Scopus citations


Background: In recent years, there has been a growth in work on the use of information extraction technologies for tracking disease outbreaks from online news texts, yet publicly available evaluation standards (and associated resources) for this new area of research havebeen noticeably lacking. Objective: This study seeks to create a "gold standard" data set against which to test how accurately disease outbreak information extraction systems can identify the semantics of disease outbreak events. Additionally, we hope that the provision of an annotation scheme (and associated corpus) to the communitywill encourage open evaluation in this new and growing application area. Methods: We developed an annotation scheme for identifying infectious disease outbreak events in news texts. An event-in the context of our annotation scheme-consists minimally of geographical (eg, country and province) and disease name information. However, the scheme also allows for the rich encoding of other domain salient concepts (eg,international travel, species, and food contamination). Results: The work resulted in a 200-document corpus of event-annotated disease outbreak reports that can be used to evaluate the accuracy of event detection algorithms (in this case, for the BioCaster biosurveillance online news information extraction system). In the 200 documents, 394 distinct events were identified (mean 1.97 events per document, range 0-25 events per document). We also provide a download script and graphical user interface (GUI)-based event browsing software to facilitatecorpus exploration. Conclusion: In summary, we present an annotation scheme and corpus that can be used in the evaluation of disease outbreak event extraction algorithms. The annotation scheme and corpus were designed both with the particular evaluation requirements of the BioCaster system in mind as well as the wider need for further evaluation resources in this growing research area.

Original languageEnglish
Pages (from-to)e43
JournalJournal of Medical Internet Research
Issue number3
StatePublished - 2010
Externally publishedYes


  • Biosurveillance
  • Corpora
  • Disease outbreaks
  • Information extraction
  • Natural language processing
  • Public health informatics
  • Text mining


Dive into the research topics of 'Developing a disease outbreak event corpus'. Together they form a unique fingerprint.

Cite this