Is Groupthink Ruining Peer Review?

6842682139_0b9cf529fe_ophotograph: Luc Melanson

Groupthink could be the crack in the corner stone of science, peer-review, and open access may be a possible solution, suggests one expert.

Groupthink is a psychological phenomenon where more emphasis is placed on consensus than free thought when making a decision. The result is often poor decisions as everyone is so focused on conformity they forget to consider the caveats of their choice. Irving Janis, the father of groupthink theory, outlines that there are eight symptoms of groupthink, which make it more likely for the phenomenon to happen, such as the group is small and defined with unified decision-making powers.

Generally, groupthink is applied to poor political or corporate decisions, such as the Vietnam War. But one area that has been overlooked is academia, specifically the peer review process.

The principal of science is built on the trial and error of certain ideas. Peer review is an integral part of the process of determining if a scientific theory is valid. All new discoveries are supposed to be sent through a panel of experts in the field and reviewed to catch any possible errors. But when beliefs about a certain theory are so strongly held the peer review process may become inherently flawed.

For example doctors believed that stomach ulcers were caused by stress. But in 1982, Australian scientists, Robbie Warren and Barry J. Marshal, discovered that it was Helicobacter pylori, a bacterium, that was the dominant cause of stomach ulcers.  Treatments could be a simple antibiotic. However, since antacid tablets were a great source of revenue for pharmaceutical companies and there was huge pressure to ignore the new evidence. It took many years and a fierce campaign to finally convince health care providers that stress wasn’t the main cause of stomach ulcers.

klein

I spoke with Dr Daniel Klein, an economist at George Mason University, who wrote a paper on how groupthink is affecting academia and asked what his thoughts were on the subject.

Why do you think groupthink is a problem in academia?

“Groupthink” is an insulting term that presupposes defectiveness in the beliefs of the group. Since judgments cannot be separated from the analysts’ own beliefs and principles we – academics – should confess where we are coming from.

Academics do not, cannot, and should not leave all of their commitments and sensibilities at the door. Beliefs or ideals in academia are bound to be ideological.

I think the beliefs that one holds influence judgments about the most important things in academic study, such as which interpretations to embrace, questions to address, issues to consider, positions on the issue, arguments for the position, authorities to heed, and so on.

For example, I am a classical liberal and the vast majority of professors in my field are social democratic. I favour limited government involvement in social affairs, while social democrats generally favour either the status quo or greater governmentalization of social affairs.

In my field I have noticed that professors are often supporting or yielding to status quo policies, or even promoting greater governmentalization of social affairs in their work. Further they frequently neglect the strong and profound arguments against the governmentalization of social affairs.

How does the phenomenon of groupthink influence the peer-review process?

The editor’s personal beliefs affects his attitude toward a submission, which referees he assigns, how he evaluates the referee reports, etc.

Do you not think that because when one is peer-reviewed the reviewers are all in a very specialized field and thus tend towards certain ideas and opinions that reviewers may miss some inherent flaws or be less accepting of research that goes against previous theories?

For the big problems I am concerned with, I see ideological sensibilities as more important than myopia from specialization.

What are some possible solutions to prevent groupthink in academia?

I don’t think there is much that any one individual can do. I think there needs to be more of a culture of debate and criticism, including at the highest echelons, at the prestigious journals.

I think it is good to talk about the problem, even if we don’t expect to see much change, because at least people will put less faith in academia.

Also, I think that scholars in the social science should be more ideologically open.

130905_burke_a

However, Professor Timothy Burke, from Swarthmore College, has a different view on the reasons behind groupthink in academia.

As opposed to political affiliations influencing the peer review process Prof Burke believes it is embedded in the academic institution. “The same forces that help academics to produce knowledge and scholarship are the forces that produce unwholesome close-mindedness and inbred self-satisfied attitudes,” he writes in a blog post.  In another piece for The Minnesota Review he comments how groupthink manifests in “a thousand tiny cuts”, peer-review just being one of the cuts. I caught up with him and asked about his views on the subject.

Why do you think groupthink is a problem in academia?

Groupthink is a problem in two very different ways. The first is when it prevents any group of scholars from seeing ideas, interpretations, or evidence that ought to be interesting or important to them – when it keeps their work from achieving its full potential. This is largely unconscious, some sort of social mechanism, often an academic discipline that is making it difficult to express or consider an idea or look at a phenomena.

The second way that groupthink is a problem is more conscious and more troubling: when it leads to excluding, bullying or mocking dissident scholars whose work deserves inclusion or respect within a particular discipline or intellectual tradition. That can sometimes be a fine line, and it is sometimes used to defend unethical or scurrilous practices, or approaches that genuinely don’t belong within a particular group.

“Creation scientists”, for example, argue that they belong inside of science but most scientists want them to be labelled as religion. The creationists might complain that this is a case of groupthink, but if all groups or disciplines in academia are obliged to include on equal footing anyone who thinks they belong in the group, then groups would lose any ability to do focused, clearly defined work using specific methods and approaches.

How do you think groupthink may influence the peer-review process?

Either kind of groupthink, conscious or unconscious, can enter into the peer review process. When it is more unconscious, scholars are often unable to understand or accurately review arguments or evidence that they’ve never imagined to be part of a discipline or specialization up to that point.

When it is conscious, peer review can be one of the most insidious ways that a particular interpretation or viewpoint enshrines itself in the discipline, by deliberately squashing any rivalrous or opposing approach in a very safely confidential and invisible manner.

Do you think groupthink may be a reason why peer-review can fail to catch fraud and errors in scientific work? 

Yes, groupthink is at least partially responsible for why fraud is hard to detect for several reasons.

First off, there is a pervasive institutional form of ‘groupthink’ that discriminates against the publication of negative findings. Often reviewers don’t even have to consciously think about that rejection, they just know that there is something “not interesting” or “not important” about a negative finding. But having a publication norm that is always against negative findings means that there is enormous incentive to claim a significant finding. This can lead to fudging of data, at least unconsciously.

Also if you’re reviewing something that fits your own paradigm or privileged approach, you are inclined to see as incontestably true or obvious findings that might actually be highly debatable or ambiguous

Finally someone who is defending a particular idea very much needs findings that align with that idea to be true. Therefore, they are much more instrumentally motivated to give a positive review to work that they might otherwise really see as dubious, and vice-versa, to claim that work which challenges that orthodoxy is dubious when it’s not.

What are some possible solutions to prevent groupthink in academia and during the peer-review process? 

Some of the less conscious kinds of groupthink are probably not preventable, and you might just generally hope that over time, a flawed or untrue approach is going to collapse inevitably of its own weight.
But there are some ways to at least break up groupthink, such as, practices of open or semi-open peer review; either all the time or at least on a frequent enough periodic basis to keep reviewers honest.

In the sciences, a commitment to publishing negative results, also strong standards for disclosure of conflicts of interest that include membership in professional associations or groups that are associated with well-defined orthodoxies.

Further there is a need for conscious, committed editorial practices that insist on intellectual heterogeneity in reviewers and that are willing to override reviewers who are trying to act as gatekeepers for a given orthodoxy. This should include in some cases the modest or partial use of interdisciplinary reviewing–that someone from a very different scholarly community may more quickly spot manipulated or flawed methods and data. Of course, they may also equally be really pointlessly annoying by questioning practices that are legitimately fundamental to a particular discipline or specialization.

Finally, I can’t stress how important open access publication is since for-profit publishers have both conscious and unconscious reasons to aid and abet in groupthink and to downplay fraud.

While there seems to be an indication that groupthink could be tainting the peer review process these are just two academics opinions. We would love to hear what you think on the topic. Tweet us @peerrevwatch or the author @mishagajewski or leave a comment below.

Advertisements

One comment

  1. Reblogged this on Orogeny and commented:
    I try to always sign my reviews. It helps keeping the review fair. Also, I often meet the authors, a year or so later, at a conference and ask about any hard feelings. Honesty, fairness and openness should lead to better, more helpfull reviews.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s