Tuesday, November 29, 2016

Event Recap: Anonymous Peer Review & PubPeer

by Ian McLaughlin 

On the 24th of October, the Penn Science Policy Group met to discuss the implications of a new mechanism by which individuals can essentially take part in the peer review process.  The group discussion focused on a particular platform, PubPeer.com, which emerged in 2012 and has since become a topic of interest and controversy among the scientific community.  In essence, PubPeer is an online forum that focuses on enabling post-publication commentary, which ranges from small concerns by motivated article readers, to deeper dives into the legitimacy of figures, data, and statistics in the publication.  Given the current state of the widely criticized peer-review process, we considered the advantages and disadvantages of democratizing the process with the added layer of anonymity applied to reviewers.

PubPeer has been involved in fostering investigations of several scandals in science.  Some examples include a critical evaluation of papers published in Nature 2014 entitled Stimulus-triggered fate conversion of somatic cells into pluripotency [1].  The paper described a novel mechanism by which pluripotency might be induced by manipulating the pH environments of somatic cells.  However, following publication, concerns regarding the scientific integrity of published experiments were raised, resulting in the retraction of both papers and an institutional investigation.
Subsequently, the publications of a prolific cancer researcher received attention on PubPeer, ultimately resulting in the rescission of a prestigious position at a new institution eleven days before the start date due, at least in part, to PubPeer commenters contacting faculty at the institution.  When trying to return the professor’s former position, it was no longer available.  The professor then sued PubPeer commenters, arguing that the site must identify the commenters that have prevented a continued career in science.  PubPeer, advised by lawyers from the ACLU working pro-bono, is refusing to comply – and enjoy the support of both Google and Twitter, both of which have filed a court brief in defense of the website [2]. 
Arguably at its best, PubPeer ostensibly fulfills an unmet, or poorly-met, need in the science publication process.  Our discussion group felt that the goal of PubPeer is one that the peer review process is meant to pursue, but occasionally falls short of accomplishing. While increased vigilance is welcome, and bad science – or intentionally misleading figures – should certainly not be published, perhaps the popularity and activity on PubPeer reveals a correctable problem in the review process rather than a fundamental flaw. While the discussion group didn’t focus specifically on problems with the current peer review process – a topic deserving its own discussion [3] – the group felt that there were opportunities to improve the process, and was ambivalent that a platform like PubPeer is sufficiently moderated, vetted, and transparent in the right ways to be an optimal means to this end.
Some ideas proposed by discussion participants were to make the peer-review process more transparent, with increased visibility applied to the reasons a manuscript is or is not published.  Additionally, peer-review often relies upon the input of just a handful of volunteer experts, all of whom are frequently under time constraints that can jeopardize their abilities to thoroughly evaluate manuscripts – occasionally resulting in the assignment of peer review to members of related, though not optimally relevant, fields [4].  Some discussion participants highlighted that a democratized review process, similar to that of PubPeer, may indeed alleviate some of these problems with the requirement that commenters be moderated to ensure they have relevant expertise.  Alternatively, some discussion participants argued, given the role of gate-keeper played by journals, often determining the career trajectories of aspiring scientists, the onus is on Journals’ editorial staffs to render peer review more effective.  Finally, another concept discussed was to layer a 3rd party moderation mechanism on top of a platform like PubPeer, ensuring comments are objective, constructive, and unbiased.
The concept of a more open peer review is one that many scientists are beginning to seriously consider.  In Nature News, Ewen Callaway reported that 60% of the authors in Nature Communications agreed to have publication reviews published [7].  However, while a majority of responders to a survey funded by the European Commission believed that open peer review ought to become more routine, not all strategies of open peer review received equivalent support.


Ultimately, the group unanimously felt that the popularity of PubPeer ought to be a signal to the scientific community that something is wrong with the publication process that requires our attention with potentially destructive ramifications [5].  Every time a significantly flawed article is published, damage is done to the perception of science and the scientific community, and at a time when the scientific community still enjoys broadly positive public perception [6], now is likely an opportune time to reconsider the peer-review process – and perhaps learn some lessons that an anonymous post-publication website like PubPeer might teach us.


1) PubPeer - Stimulus-triggered fate conversion of somatic cells into pluripotency. (n.d.). Retrieved November 25, 2016, from https://pubpeer.com/publications/8B755710BADFE6FB0A848A44B70F7D 

2) Brief of Amici Curiae Google Inc. and Twitter Inc. in Support of PubPeer, LLC. (Michigan Court of Appeals). https://pubpeer.com/Google_Twitter_Brief.pdf

3) Balietti, S. (2016). Science Is Suffering Because of Peer Review’s Big Problems. Retrieved November 25, 2016, from https://newrepublic.com/article/135921/science-suffering-peer-reviews-big-problems

4)Arns M. Open access is tiring out peer reviewers. Nature. 2014 Nov 27;515(7528):467. doi: 10.1038/515467a. PubMed PMID: 25428463.

5) Jha, Alok. (2012). False positives: Fraud and misconduct are threatening scientific research. Retrieved November 25, 2016, from https://www.theguardian.com/science/2012/sep/13/scientific-research-fraud-bad-practice

6) Hayden, E. C. (2015, January 29). Survey finds US public still supports science. Retrieved November 25, 2016, from http://www.nature.com/news/survey-finds-us-public-still-supports-science-1.16818 

7) Callaway E. Open peer review finds more takers. Nature. 2016 Nov 10;539(7629):343. doi: 10.1038/nature.2016.20969. PubMed PMID: 27853233

Sunday, November 6, 2016

Tracing the ancestry and migration of HIV/AIDS in America

by Arpita Myles
Acquired immunodeficiency syndrome or AIDS is a global health problem that has terrified and intrigued scientists and laypeople alike for decades. AIDS is caused by the Human Immunodeficiency Virus, or HIV, which is transmitted through blood, semen, vaginal fluid, and from an infected mother to her child [1]. Infection leads to failure of the immune system, increasing susceptibility to secondary infections and cancer, which are mostly fatal. Considerable efforts are being put into developing prophylactic and therapeutic approaches to tackle HIV-AIDS, but there is also interest in understanding how the disease became so wide-spread. With the advent of the Ebola and Zika viruses in the last couple of years, there is a renewed urgency in understanding the emergence and spread of viruses in the past in order to prevent those in the future. The narrative surrounding the spread of HIV has been somewhat convoluted, but a new paper in Nature by Worobey et. al, hopes to set the record straight [2].
Humans are supposed to have acquired HIV from African chimpanzees- presumably as a result of hunters coming in contact with infected blood, containing a variant of the virus that had adapted to infect humans. The earliest known case of HIV in humans was detected in 1959 in Kinshasa, Democratic Republic of the Congo, but the specific mode of transmission was never ascertained [3].
There has been little or no information about how HIV spread to United States, until now. HIV incidences were first reported in the US in 1981, leading to the recognition of AIDS [4]. Since the virus can persist for a decade or more prior to manifestation of symptoms, it is possible that it arrived in the region long before 1981. However, since most samples from AIDS patients were collected after this date, efforts to establish a timeline for HIV’s entry into the states met with little success. Now, researchers have attempted to trace the spread of HIV by comparing genetic sequences of contemporary HIV strains with blood samples from HIV patients dating back to the late 1970’s [2]. These samples were initially collected for a study pertaining to Hepatitis B, but some were found to be HIV seropositive. This is the first comprehensive genetic study of the HIV virus in samples collected prior to 1981.
The technical accomplishment of this work is significant as well. In order to circumvent the problems of low amounts and extensive degradation of the viral RNA from the patient samples, they developed a technique they call “RNA jackhammering.”  In essence, a patient’s genome is broken down into small bits and overlapping sequences of viral RNA are amplified. This enables them to “piece together” the viral genome, which they can then subject to phylogenetic analysis.
Using novel statistical analysis methods, Worobey et al. reveal that the virus had probably entered New York from Africa (Haiti) during the 1970s, whereupon it spread to San Francisco and other regions. Upon analyzing the older samples, the researchers found that despite bearing similarities with the Caribbean strain, the strains from San Francisco and New York samples differed amongst themselves. This suggests that the virus had entered the US multiple, discreet times and then began circulating and mutating. Questions still remain regarding the route of transmission of the virus from Haiti to New York.
The relevance of this study is manifold. Based on the data, one can attempt to understand how pathogens spread from one population to another and how viruses mutate and evolve to escape natural immunity and engineered therapeutics. Their molecular and analytical techniques can be applied to other diseases and provide valuable information for clinicians and epidemiologists alike. Perhaps the most startling revelation of this study is that contemporary HIV strains are more closely related to their ancestors than to each other. This implies that information derived from ancestral strains could lead to development of successful vaccine strategies.
Beyond the clinic and research labs, there are societal lessons to be learned as well. Published in 1984, a study by CDC (Center for Disease Control) researcher William Darrow and colleagues traced the initial spread of HIV in the US to GaĆ©tan Dugas- a French Canadian air steward. Examination of Dugas’s case provided evidence linking HIV transmission with sexual activity. Researchers labeled Dugas as “Patient O”, as in “Out of California” [5]. This was misinterpreted as “Patient Zero” by the media- a term still used in the context of other epidemics like flu and Ebola. The dark side of this story is that Dugas was demonized in the public domain as the one who brought HIV to the US. As our understanding of the disease and its spread broadened, scientists and historians began to discredit the notion that Dugas played a significant role. However, scientific facts were buried beneath layers of sensationalism and hearsay and the stigma remained.
Now, with the new information brought to light by Worobey’s group, Dugas’s name has been cleared. Phylogenetic analysis of Dugas’s strain of HIV was sufficiently different from the ancestral ones, negating the possibility that he initiated the epidemic.
The saga in its entirety highlights the moral dilemma of epidemiological studies and the extent to which the findings should be made public. Biological systems are complicated, and while narrowing down origin of a disease has significance clinical relevance, we often fail to consider collateral damage. The tale of tracking the spread of HIV is a cautionary one; scientific and social efforts should be focused more on resolution and management, rather than on vilifying unsuspecting individuals for “causing” an outbreak.

1. Maartens G, Celum C, Lewin SR. HIV infection: epidemiology, pathogenesis, treatment, and prevention. Lancet. 2014 Jul 19;384(9939):258-71.
2. Worobey M, Watts TD, McKay RA et al., 1970s and 'Patient 0' HIV-1 genomes illuminate early HIV/AIDS history in North America. Nature. 2016 Oct 26. doi: 10.1038/nature19827.
3. Faria NR, Rambaut A et al., HIV epidemiology. The early spread and epidemic ignition of HIV-1 in human populations. Science. 2014 Oct 3;346(6205):56-61.
4. Centers for Disease Control (CDC). Pneumocystis pneumonia--Los Angeles. MMWR Morb Mortal Wkly Rep. 1981 Jun 5;30(21):250-2.
5. McKay RA. “Patient Zero”: The Absence of a Patient’s View of the Early North American AIDS Epidemic. Bull Hist Med. 2014 Spring: 161-194.