[Previous message][Next message][Back to index]
[ecrea] Peer Review Survey 2009: Preliminary Findings
Sun Nov 29 09:29:23 GMT 2009
http://www.senseaboutscience.org.uk/index.php/site/project/395
Peer Review Survey 2009: Preliminary Findings
Should peer review detect fraud and misconduct?
What does it do for science and what does the
scientific community want it to do? Will it
illuminate good ideas or shut them down? Should reviewers remain anonymous?
On 8th September 2009 the preliminary findings of
one of the largest ever international surveys of
authors and reviewers, the Peer Review Survey
2009 1, were released. The findings were
presented in the session "Science Fact or Science
Fiction: Should Peer Review Stop Plagiarism, Bias
or Fraud?" at the British Science Festival, where
Tracey Brown of Science About Science, David Adam
of The Guardian and Peter Hayward of Lancet
Infectious Diseases debated the challenges of publishing research.
Peer review is fundamental to integration of new
research findings. It allows other researchers to
analyse findings and society at large to weigh up
research claims. It results in 1.3 million3
learned articles published every year, and it is
growing rapidly with the expansion of the global
research community. With that growth come new
concerns - about getting the next generation of
researchers to review in sufficient numbers,
about maintaining the system's integrity and
whether it can be truly globalised; and also new
ideas - about alternative quality measures,
technologies to prevent plagiarism, rewarding reviewers and training them.
Sense About Science has promoted understanding of
peer review to help people to work out whether
research claims have been independently
scrutinised. But with all the proposed changes
and expansion in research publication, what do
researchers think about peer review and its
future? To find out, Sense About Science
developed the Peer Review Survey 2009, in
consultation with editors and publishers and
administered with a grant from Elsevier; the
survey included some questions from the Peer
Review Survey 20074 for comparison, and new
questions about future improvements, public
awareness and pressures on the system.
Tracey Brown, Managing Director: "The 2007 survey
had raised some of the issues. We sought to
broaden that, particularly to find out whether
the demand for all this free, independent
scrutiny from the research community is
sustainable, and what the future of quality
control is likely to be. It's a matter of public
as well as scientific interest."
Preliminary findings include:
1. Playing an active role in the community is
top of reasons to review: 90% say they review
because they believe they are playing an active
role in the community; only 16% say that
increasing their chances of having future papers
accepted is a reason to review.
2. Researchers want to improve, not replace peer review:
* 84% believe that without peer review
there would be no control in scientific
communication, but only a third (32%) think it is
the best that can be achieved; 20% of researchers
believe that peer review is unsustainable because of too few willing reviewers.
* 91% say that their last paper was
improved through peer review; the discussion was
the biggest area of improvement.
* 73% of reviewers (a sub-group) say
that technological advances have made it easier
to do a thorough job than 5 years ago. Whilst 86%
enjoy reviewing, 56% say there is a lack of
guidance on how to review; 68% think formal
training would help. On average, reviewers turn down two papers a year.
* Just 15% of respondents felt that
'formal' peer review could be replaced by usage statistics.
* 61% of reviewers have rejected an
invitation to review an article in the last year,
citing lack of expertise as the main reason -
this suggests that journals could better identify suitable reviewers.
3. High expectations:
* 79% or more of researchers think
that peer review should identify the best papers,
determine their originality and importance,
improve those papers and, though lower scoring,
also determine whether research is plagiarised or fraudulent.
* While 43% of respondents thought
peer review was too slow, 65% of authors (a
further sub-group) reported that they had
received a decision on their most recent paper within 3 months.
4. Reviewers want anonymity: 58% would be
less likely to review if their signed report was
published. 76% favour the double blind system
where just the editor knows who the reviewers are.
5. Understanding of peer review: Researchers
agree that peer review is well understood by the
scientific community but just 30% believe the public understands the term.
6. Papers aren't recognising previous work:
81% think peer review should ensure previous
research is acknowledged; 54% think it currently
does. This reflects current concerns in the research community5.
7. Detecting plagiarism and fraud might be a
noble aim but is not practical: A majority think
peer review should detect plagiarism (81%) or
fraud (79%) but fewer (38% and 33%) think it is capable of this.
8. Reviewers divided over incentives: Just
over half of reviewers think receiving a payment
in kind (e.g. subscription) would make them more
likely to review; 41% wanted payment for
reviewing, but this drops to just 2.5% if the
author had to cover the cost. Acknowledgement in
the journal is the most popular option.
Comment from Editors:
Robert Campbell, Wiley-Blackwell & Chairman,
Publishing Research Consortium: "Most researchers
give up time to review papers for no charge. The
whole scholarly communication system is dependent
on this. Why do they do it? This study helps us
to answer the question. And I take heart in the
finding that 85% of the around 4000 respondents
quite simply do it because they enjoy being able to improve papers. "
Sir Brian Heap, Vice Chairman, European Academies
Science Advisory Council: "These latest insights
from researchers show why the peer review process
continues to be so important - and enjoyable!"
Adrian Mulligan, Associate Director of Research
and Academic Relations, Elsevier: "I'm impressed
by the vast amount of responses to the study - it
tells us just how engaged the scientific
community is with peer review. Not only do they
greatly value it, but they have a strong desire
to enhance it. Elsevier in partnership with
editors, plays an active role in developing peer
review, and we are moving forward with
initiatives such as Crosscheck, a pan-publisher
plagiarism detection tool. We are keen to look at
other ways to respond to the views raised in this survey."
Irene Hames, Managing Editor The Plant Journal,
Author 'Peer Review and Manuscript Management in
Scientific Journals: guidelines for good
practice': "It's very important to know what
authors and reviewers actually think about the
current status of peer review. Too many
commentators make broad generalizations that are
not evidence based. So I welcome this large-scale
survey from Sense About Science. Once again, the
importance with which peer review is viewed comes
through, with the great majority of researchers
believing that without peer review there would be
no control in scientific communication. That is
not to say there aren't problems - there clearly
are, and improvements and innovative solutions
are needed. Crucial in this is the need to
professionalize this area of activity, which too
often is put in the hands of people who may have
great academic reputations and research
expertise, but no experience of running a peer-review system."
Notes
1. The Peer Review Survey was an electronic
survey conducted between 28th July 2009 and 11th
August 2009; 40,000 researchers were randomly
selected from the ISI author database, which
contains published researchers from over 10,000
journals. Altogether 4,037 researchers completed
our survey. The error margin was ± 1.5% at 95%
confidence levels; reviewers answered a subset of
questions aimed specifically at reviewers (3,597
- a subset of the base) the error margin for this
group was ± 1.6% at 95% confidence levels.
2. The full findings and report are due to be published in November 2009.
3. Bjork et al (2008) 'Global annual volume
of peer reviewed scholarly articles and the share
available via different Open Access options'
Proceedings ELPUB2008 Conference on Electronic
Publishing - Toronto, Canada - June 2008
4. Publishing Research Consortium (2007)
'Peer Review in Scholarly Journals: perspective
of the scholarly community. An international study'
5. Chalmers & Glasziou (2009) 'Avoidable
waste in the production and reporting of research
evidence' The Lancet; 374: 86-89.
Media Coverage
The Times Online A review of peer review
Times Higher Education Peer Reviewers satisfied with the system
BBC World Service Europe Today(about 20 minutes into clip)
Nature blogs Peer review reviewed
Chemistry World Peer review under the microscope
Information World Review Researchers show
loyalty to peer review but want a lot more from it
Physics World blog And the survey say...
-----------------------------------------------------------------------
Nico Carpentier (Phd)
-----------------------------------------------------------------------
Vrije Universiteit Brussel - Free University of Brussels
Centre for Studies on Media and Culture (CeMeSO)
Pleinlaan 2 - B-1050 Brussels - Belgium
T: ++ 32 (0)2-629.18.56
F: ++ 32 (0)2-629.36.84
Office: 5B.401a
-----------------------------------------------------------------------
European Communication Research and Education Association
Web: http://www.ecrea.eu
----------------------------
E-mail: (Nico.Carpentier /at/ vub.ac.be)
Web: http://homepages.vub.ac.be/~ncarpent/
-----------------------------------------------------------------------
----------------
ECREA-Mailing list
----------------
This mailing list is a free service from ECREA.
---
To unsubscribe, please visit http://www.ecrea.eu/mailinglist
---
ECREA - European Communication Research and Education Association
Postal address:
ECREA
Université Libre de Bruxelles
c/o Dept. of Information and Communication Sciences
CP123, avenue F.D. Roosevelt 50, b-1050 Bruxelles, Belgium
Email: (info /at/ ecrea.eu)
URL: http://www.ecrea.eu
----------------
[Previous message][Next message][Back to index]
