Epistasis Blog

From the Artificial Intelligence Innovation Lab at Cedars-Sinai Medical Center (www.epistasis.org)

Thursday, January 30, 2014

Reconciling clinical importance and statistical significance in GWAS

Genome-wide association studies (GWAS) have identified many risk-associated SNPs with very small effects. The mantra for identifying more associations is to greatly increase the sample size to be able to detect smaller and smaller effects. This wonderful letter in the European Journal of Human Genetics points out that at some point the effect size goes below the measurement error calling into question the clinical significance of these GWAS hits. If I were funding a big GWAS study I would first want to know whether increasing the sample size is justified given the effects sizes to be detected and the error of the phenotype measures.

Shriner D, Adeyemo A, Rotimi CN. Reconciling clinical importance and statistical significance. Eur J Hum Genet. 2014 Feb;22(2):158-9. [EJHG]

Saturday, January 25, 2014

My take of the FDA's decision to regulate 23andMe

In 2013 the FDA ordered 23andMe to stop selling it's genetic testing services for health-related purposes. This was a very controversial ruling that generated lots of discussion in the media. A collection of links to media coverage and opinions put together by writer David Dobbs can be found here. My take on the issue can be found in this Dartmouth Medicine Magazine piece.

Saturday, January 11, 2014

Percentile Ranking and Citation Impact of a Large Cohort of NHLBI-Funded Cardiovascular R01 Grants

I just ran across this interesting new study that evaluated the relationship between the score that an NIH R01 grant receives during peer-review and the future impact of the grant as measured by number and quality of publications. The bottom line is that a grant that receives a top score in the 10th percentile does not produce publications with impact above and beyond a grant in the 30th percentile that would not be funded by 2014 criteria.

Danthi N, Wu CO, Shi P, Lauer MS. Percentile Ranking and Citation Impact of a Large Cohort of NHLBI-Funded Cardiovascular R01 Grants. Circ Res. 2014 Jan 9. [PubMed]

Abstract

Rationale: Funding decisions for cardiovascular R01 grant applications at NHLBI largely hinge on percentile rankings. It is not known whether this approach enables the highest impact science.

Objective: To conduct an observational analysis of percentile rankings and bibliometric outcomes for a contemporary set of funded NHLBI cardiovascular R01 grants.

Methods and Results: We identified 1492 investigator-initiated de novo R01 grant applications that were funded between 2001 and 2008, and followed their progress for linked publications and citations to those publications. Our co-primary endpoints were citations received per million dollars of funding, citations obtained within 2-years of publication, and 2-year citations for each grant's maximally cited paper. In 7654 grant-years of funding that generated $3004 million of total NIH awards, the portfolio yielded 16,793 publications that appeared between 2001 and 2012 (median per grant 8, 25th and 75th percentiles 4 and 14, range 0 - 123), which received 2,224,255 citations (median per grant 1048, 25th and 75th percentiles 492 and 1,932, range 0 - 16,295). We found no association between percentile ranking and citation metrics; the absence of association persisted even after accounting for calendar time, grant duration, number of grants acknowledged per paper, number of authors per paper, early investigator status, human versus non-human focus, and institutional funding. An exploratory machine-learning analysis suggested that grants with the very best percentile rankings did yield more maximally cited papers.

Conclusions: In a large cohort of NHLBI-funded cardiovascular grants, we were unable to find a monotonic association between better percentile ranking and higher scientific impact as assessed by citation metrics.