[OAI-general] Alternative Peer Review Models for Scholarship
Toth Ben
ben.toth@nhsia.nhs.uk
Sun, 27 Apr 2003 21:24:01 +0100
Gerry
William Arms has done some work on this subject. See
http://www.cs.cornell.edu/wya/home/presentations.html for a link to a
presentation. But I know he has also written on the subject and this should
be available on the Web.
There are some papers on peer-review at
http://bmj.com/cgi/collection/peer_review including reference to an
important Cochrane Review, available from the National electronic Library
for Health (www.nelh.nhs.uk) or from
http://www.update-software.com/Cochrane/MR000016.pdf . This link may
disappear, so the abstract is below.
regards
Ben Toth
--------------------------------------------
Dr Ben Toth
Head of Knowledge Management
NHS Information Authority
Aqueous II, Aston Cross, Birmingham, B6 5RQ
T 0121 333 0252
M 07775 993 168
E ben.toth@nhsia.nhs.uk
--------------------------------------------
Background
Editorial peer-review is a globally used tool to select submissions to
biomedical journals and improve their content. As such it
has a central role in biomedical science. The knowledge base of peer-review
has traditionally lagged far behind its acceptability
and use as a quality assessment tool. In the last decade a growing body of
research has been available, making synthesis and
conceptual analysis possible. We reviewed the evidence of the effects of
processes undertaken as part of editorial peer-review of
original research studies submitted for paper or electronic publication in
biomedical journals.
Objectives
To estimate the effects of processes in editorial peer-review.
Search strategy
The following electronic sources were searched:
The Cochrane Library (Issue 2, 2000), MEDLINE 1966 to February 2000,
Australasian Medical Index 1980 to 2000, Best Evidence
(4, 2000), Bioethicsline 1973 to 2000, CINAHL (Combined Index of Nursing &
Allied Health Literature) 1997 to 1999, Current
Contents 1999 to 2000, Dissertation Abstracts 1861 to 2000, EMBASE January
1989 to January 2000, HEALTHSTAR 1975 to
1999, National Research Register, PsycLIT 1887 to 2000, PubScience 1998 to
February 2000, SIGLE 1980 to 6/1999. We also
searched reference lists of relevant articles, posted requests on the EASE
and WAME web sites, the Locknet site and hand-searched
books and JAMA theme issues and relevant bulletins. The WAME (World
Association of Medical Editors), EASE (European
Association of Science Editors), and CSE (Council of Science Editors)
memberships were canvassed in an effort to locate further
published and unpublished studies.
Selection criteria
We included prospective or retrospective comparative studies, with two or
more comparison groups, generated by random or
other methods, reporting original research regardless of publication status.
We hoped to find studies identifying good submissions
on the basis of: importance of the topic dealt with, relevance of the topic
to the journal, usefulness of the topic, soundness of
methods, soundness of ethics, completeness and accuracy of reporting.
Data collection and analysis
We identified 135 reports of studies, which could possibly fulfil our
inclusion criteria. Twenty-one of these fulfilled our criteria
and we excluded the remaining 114 studies from our review. Because of the
diversity of study questions, viewpoints, methods
and outcomes we carried out a descriptive review of included studies
grouping them by broad study questions.
Main results
The well-researched practice of concealing the identities of peer-reviewers
or authors, while laborious and expensive, appears to
have little effect on the outcome of the quality assessment process (9
studies). Checklists and other standardisation media have
little reliable evidence to support their use (2 studies). There is no
evidence that referees' training has any effect on the quality of
the outcome (2 studies). Electronic communication media do not appear to
have an effect on quality (2 studies). On the basis of
one study little can be said about the ability of the peer-review process to
detect bias against unconventional drugs. Validity of
peer-review was tested by only one small study in a specialist area.
Editorial peer-review appears to make papers more readable
and improve the general quality of reporting (2 studies), but the evidence
for this may be of limited generalisability.
Reviewers' conclusions
At present there is little empirical evidence to support the use of
editorial peer-review as a mechanism to ensure quality of
biomedical research, despite its widespread use and costs. A large,
well-funded programme of research on the effects of editorial
peer-review is needed.
This Review should be cited as:
Jefferson TO, Alderson P, Davidoff F, Wager E. Editorial peer-review for
improving the quality of reports of biomedical studies
(Cochrane Methodology Review) In: The Cochrane Library, Issue 1, 2003.
Oxford: Update Software.
-----Original Message-----
From: Gerry Mckiernan [mailto:gerrymck@iastate.edu]
Sent: 25 April 2003 21:26
To: oai-general@oaisrv.nsdl.cornell.edu
Subject: [OAI-general] Alternative Peer Review Models for Scholarship
Alternative Peer Review Models for Scholarship
For a forthcoming presentation [and, of course, the obligatory associated
Web registry and article {:-)], I am greatly interested in the key
articles/report/documents about Alternative Peer Review for Scholarship, as
well as examples of exemplar (Web-based) publishing initiatives that have
implemented one or more of these alternatives.
The Alternative Models that come to mind are:
Open Peer Review
Formal Peer Review with Subsequent Commentary
Commentary Only (No Prior Formal Peer Review)
Pre and/or Post Commentary
Citation-based Peer Review
Reader-based Peer Review
Computer-Assisted Peer Review
Collaborative Filtering
Others?
and, of course
No Peer Review (Yes, No Review to Me can be reviewed as an Alternative).
I am *particularly* interested in any current research or implementations
that are Computer-Assisted such as
A SOFTWARE PROGRAM TO AID IN PEER REVIEW
By Alvar Loria (1) Gladys Faba (2) for the ARTEMISA Selection Committee
(1)Instituto Nacional de Nutrición Salvador Zubiran, 14000 Mexico City,
Mexico
(2)Centro Nacional de Informacion y Documentacion en Salud, Mexico City,
Mexico
[ http://www.ama-assn.org/public/peer/arev.htm ]
As Always Any and All contributions, suggestions, recommendations,
comments, DNA samples, etc. etc. etc. are Most Welcome.
/Gerry
Gerry McKiernan
Alternative Librarian
Iowa State University
Ames IA 50011
gerrymck@iastate.edu
"The Best Way to Predict the Future is to Invent It!"
Alan Kay
_______________________________________________
OAI-general mailing list
List information, archives, preferences and to unsubscribe:
http://oaisrv.nsdl.cornell.edu/mailman/listinfo/oai-general
This e-mail is confidential and privileged. If you are not the intended
recipient please accept our apologies; please do not disclose, copy or
distribute information in this e-mail or take any action in reliance on its
contents: to do so is strictly prohibited and may be unlawful. Please
inform us that this message has gone astray before deleting it. Thank you
for your co-operation.