An expanded evaluation of protein function prediction methods shows an improvement in accuracy

File Description SizeFormat 
art%3A10.1186%2Fs13059-016-1037-6.pdfPublished version3.06 MBAdobe PDFView/Open
Title: An expanded evaluation of protein function prediction methods shows an improvement in accuracy
Authors: Jiang, Y
Oron, TR
Clark, WT
Bankapur, AR
D'Andrea, D
Lepore, R
Funk, CS
Kahanda, I
Verspoor, KM
Ben-Hur, A
Koo, DCE
Penfold-Brown, D
Shasha, D
Youngs, N
Bonneau, R
Lin, A
Sahraeian, SME
Martelli, PL
Profiti, G
Casadio, R
Cao, R
Zhong, Z
Cheng, J
Altenhoff, A
Skunca, N
Dessimoz, C
Dogan, T
Hakala, K
Kaewphan, S
Mehryary, F
Salakoski, T
Ginter, F
Fang, H
Smithers, B
Oates, M
Gough, J
Toronen, P
Koskinen, P
Holm, L
Chen, C-T
Hsu, W-L
Bryson, K
Cozzetto, D
Minneci, F
Jones, DT
Chapman, S
Dukka, BKC
Khan, IK
Kihara, D
Ofer, D
Rappoport, N
Stern, A
Cibrian-Uhalte, E
Denny, P
Foulger, RE
Hieta, R
Legge, D
Lovering, RC
Magrane, M
Melidoni, AN
Mutowo-Meullenet, P
Pichler, K
Shypitsyna, A
Li, B
Zakeri, P
ElShal, S
Tranchevent, L-C
Das, S
Dawson, NL
Lee, D
Lees, JG
Sillitoe, I
Bhat, P
Nepusz, T
Romero, AE
Sasidharan, R
Yang, H
Paccanaro, A
Gillis, J
Sedeno-Cortes, AE
Pavlidis, P
Feng, S
Cejuela, JM
Goldberg, T
Hamp, T
Richter, L
Salamov, A
Gabaldon, T
Marcet-Houben, M
Supek, F
Gong, Q
Ning, W
Zhou, Y
Tian, W
Falda, M
Fontana, P
Lavezzo, E
Toppo, S
Ferrari, C
Giollo, M
Piovesan, D
Tosatto, SCE
Del Pozo, A
Fernandez, JM
Maietta, P
Valencia, A
Tress, ML
Benso, A
Di Carlo, S
Politano, G
Savino, A
Rehman, HU
Re, M
Mesiti, M
Valentini, G
Bargsten, JW
Van Dijk, ADJ
Gemovic, B
Glisic, S
Perovic, V
Veljkovic, V
Veljkovic, N
Almeida-e-Silva, DC
Vencio, RZN
Sharan, M
Vogel, J
Kansakar, L
Zhang, S
Vucetic, S
Wang, Z
Sternberg, MJE
Wass, MN
Huntley, RP
Martin, MJ
O'Donovan, C
Robinson, PN
Moreau, Y
Tramontano, A
Babbitt, PC
Brenner, SE
Linial, M
Orengo, CA
Rost, B
Greene, CS
Mooney, SD
Friedberg, I
Radivojac, P
Item Type: Journal Article
Abstract: Background A major bottleneck in our understanding of the molecular underpinnings of life is the assignment of function to proteins. While molecular experiments provide the most reliable annotation of proteins, their relatively low throughput and restricted purview have led to an increasing role for computational function prediction. However, assessing methods for protein function prediction and tracking progress in the field remain challenging. Results We conducted the second critical assessment of functional annotation (CAFA), a timed challenge to assess computational methods that automatically assign protein function. We evaluated 126 methods from 56 research groups for their ability to predict biological functions using Gene Ontology and gene-disease associations using Human Phenotype Ontology on a set of 3681 proteins from 18 species. CAFA2 featured expanded analysis compared with CAFA1, with regards to data set size, variety, and assessment metrics. To review progress in the field, the analysis compared the best methods from CAFA1 to those of CAFA2. Conclusions The top-performing methods in CAFA2 outperformed those from CAFA1. This increased accuracy can be attributed to a combination of the growing number of experimental annotations and improved methods for function prediction. The assessment also revealed that the definition of top-performing algorithms is ontology specific, that different performance metrics can be used to probe the nature of accurate predictions, and the relative diversity of predictions in the biological process and human phenotype ontologies. While there was methodological improvement between CAFA1 and CAFA2, the interpretation of results and usefulness of individual methods remain context-dependent.
Issue Date: 7-Sep-2016
Date of Acceptance: 4-Aug-2016
ISSN: 1474-760X
Publisher: BioMed Central
Journal / Book Title: Genome Biology
Volume: 17
Copyright Statement: © 2016 The Author(s). Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.
Sponsor/Funder: Biotechnology and Biological Sciences Research Council (BBSRC)
Biotechnology and Biological Sciences Research Council (BBSRC)
Biotechnology and Biological Sciences Research Council (BBSRC)
Biotechnology and Biological Sciences Research Council (BBSRC)
Funder's Grant Number: BB/F020481/1
Keywords: Science & Technology
Life Sciences & Biomedicine
Biotechnology & Applied Microbiology
Genetics & Heredity
Protein function prediction
Disease gene prioritization
Disease gene prioritization
Protein function prediction
05 Environmental Sciences
06 Biological Sciences
08 Information And Computing Sciences
Publication Status: Published
Open Access location:
Article Number: ARTN 184
Appears in Collections:Faculty of Natural Sciences

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.

Creative Commonsx