Auto-CORPus: a natural language processing tool for standardising and reusing biomedical literature
File(s)Frontiers_Auto-CORPus_revisionDec21_Supplementary_Material_final.pdf (1.84 MB) fdgth-04-788124.pdf (3.13 MB)
Supporting information
Published version
Author(s)
Type
Journal Article
Abstract
To analyse large corpora using machine learning and other Natural Language Processing (NLP) algorithms, the corpora need to be standardised. The BioC format is a community-driven simple data structure for sharing text and annotations, however there is limited access to biomedical literature in BioC format and a lack of bioinformatics tools to convert online publication HTML formats to BioC. We present Auto-CORPus (Automated pipeline for Consistent Outputs from Research Publications), a novel NLP tool for the standardisation and conversion of publication HTML and table image files to three convenient machine-interpretable outputs to support biomedical text analytics. Firstly, Auto-CORPus can be configured to convert HTML from various publication sources to BioC. To standardise the description of heterogenous publication sections, the Information Artifact Ontology is used to annotate each section within the BioC output. Secondly, Auto-CORPus transforms publication tables to a JSON format to store, exchange and annotate table data between text analytics systems. The BioC specification does not include a data structure for representing publication table data, so we present a JSON format for sharing table content and metadata. Inline tables within full-text HTML files and linked tables within separate HTML files are processed and converted to machine-interpretable table JSON format. Finally, Auto-CORPus extracts abbreviations declared within publication text and provides an abbreviations JSON output that relates an abbreviation with the full definition. This abbreviation collection supports text mining tasks such as named entity recognition by including abbreviations unique to individual publications that are not contained within standard bio-ontologies and dictionaries. The Auto-CORPus package is freely available with detailed instructions from GitHub at https://github.com/omicsNLP/Auto-CORPus.
Editor(s)
Ruch, Patrick
Date Issued
2022-02-15
Date Acceptance
2022-01-21
Citation
Frontiers in Digital Health, 2022, Healthcare Text Analytics: Unlocking the Evidence from Free Text, Volume II, 4
ISSN
2673-253X
Publisher
Frontiers Media
Journal / Book Title
Frontiers in Digital Health
Volume
4
Copyright Statement
© 2022 Beck, Shorter, Hu, Li, Sun, Popovici, McQuibban, Makraduli,
Yeung, Rowlands and Posma. This is an open-access article distributed under the
terms of the Creative Commons Attribution License (CC BY). The use, distribution
or reproduction in other forums is permitted, provided the original author(s) and
the copyright owner(s) are credited and that the original publication in this journal
is cited, in accordance with accepted academic practice. No use, distribution or
reproduction is permitted which does not comply with these terms.
Yeung, Rowlands and Posma. This is an open-access article distributed under the
terms of the Creative Commons Attribution License (CC BY). The use, distribution
or reproduction in other forums is permitted, provided the original author(s) and
the copyright owner(s) are credited and that the original publication in this journal
is cited, in accordance with accepted academic practice. No use, distribution or
reproduction is permitted which does not comply with these terms.
License URL
Sponsor
Medical Research Council (MRC)
Medical Research Council
Identifier
https://www.frontiersin.org/articles/10.3389/fdgth.2022.788124/abstract
Grant Number
MR/S004033/1
MR/S004033/1
Subjects
Natural Language Processing
text mining
health data
biomedical literature
Semantics
Notes
Published in the Health Informatics section of the journal
Edition
Healthcare Text Analytics: Unlocking the Evidence from Free Text, Volume II
Publication Status
Published
Article Number
ARTN 788124