How Peer Review at Top-tier Journals is Flawed
October 13th, 2011
How peer review at top-tier journals is flawed.
Problems with the Reviewing Process
Articles are usually reviewed by between two and four people who are chosen by an editor to recommend whether an article is fit for publication. These peers are generally—but not always—experts on the article’s subject and are asked to read for consistency, mistakes, misinformation and evidence of fraud. Some journals prefer for these peers to be anonymous and some have a fully transparent review process but either way, the review process usually does not accomplish its goals.
Peers miss an amazing amount of mistakes, inconsistencies and outright fraud. Generally unpaid, peers have little motivation to give articles the thorough dissection they need. Peer reviewers are often asked to review far more articles than their schedules allow, a situation documented by Daniel Myers in a recent essay about his experiences as a peer reviewer. As a result, peers tend to breeze through an article, decide whether it sort of makes sense or not and pass it along to the editor with a recommendation based more on what they had for breakfast than the integrity and quality of the research.
The recently discredited work of Woo-Suk Hwang at Seoul National University is an excellent example. Woo-Suk Hwang’s work went through a rigorous peer review process that entirely failed to catch the researcher’s massive amounts of fraud. Peers caught up in today’s publish-or-perish academic environment are often too overwhelmed with their own department’s writing demands to devote the correct amount of time and attention to the articles of their fellow scientists.
Submission Increases and Stylistic Changes
The editor-in-chief of Science, Donald Kennedy, states that his publication rejects at least 6,000 papers every year and submissions are increasing steadily. With so many rejections, editorial staff members are required to use more of their already over-stuffed workdays addressing author complaints and sorting through claims of impartiality and plagiarism.
Authors are going to increasingly desperate measures to get editors to publish their work. In recent years, there has been a serious uptick in the amount of time scientists spend networking with editors in attempt to make a personal connection that will make rejection less likely. Another tactic is to exaggerate the results of a study, creating a flashier article at the expense of good science. Articles that have any link to human diseases are always more popular and writers have been known to stress very tenuous links between their research and human health just to appeal to editors.
The peer review system has never been standardized and that would be an excellent next step toward saving it. Creating a basic training program for reviewers, a universal form for reviews and a system to deal with accusations of favoritism and plagiarism in the review process would be useful. Deciding whether reviews for individual publications would be anonymous or transparent would also be a smart move as would limiting the number of reviews a peer can do per year.
The peer review system is deeply flawed but salvageable. Authors and publishers are losing confidence in the whole system for good reason, and many authors are now pushing for its abandonment. Peer reviews is an essential control for maintaining the integrity and quality of research and those interested in saving it need to make changes sooner than later to prevent the breakdown it’s headed toward.