THE ROLE OF PEER REVIEW, AN ALTERNATIVE VIEW
Eric Sandewall
Linkoping University

An 'open' system of peer review, where all papers submitted are mounted on a  web server for wide access and comments from readers are posted, has been proposed by many including Zoltan Nadasdy in his preprint for this meeting.  This seems to me to be throwing the baby out with the bathwater.  Informed peer review by chosen experts is replaced by a free-for-all where the comments posted on a paper by readers could be from experts or by 'reviewers' who are entirely uninformed.  How are comments ranked in importance or weighted?  How is personal bias eliminated?  What happens to those papers that receive no comments from readers?  How does one stop an avid commentator from exerting undue influence on the process?  How does one encourage comments from the experts?  Is there any revision of the paper built in on acceptance?

The traditional review and the "free-for-all" are not the only alternatives though. In the ETAI, articles go through two successive steps:

1)  An *open review* period of three months, where the article is open for questions and comments from peers in the same specialized area.

2)  A *refereeing* phase where two or (usually) three anonymous referees evaluate whether the article is to be accepted or not. It is recommended that the referees should only consider "pass" or "fail"; detailed feedback to authors is supposed to occur in the reviewing phase. (However this does not always work as intended). The authors have a chance of revising the paper between phase 1 and phase 2, based on the feedback. Often, we invite one or two colleagues to write open reviews for the beginning of the open review period. The exchange of opinions on articles is moderated by the area editor for the research area in question, which answers the question about avid commentators. Questions or comments from incompetent participants are to be answered correctly by the authors. It is then up to the referees to evaluate the weight of the criticisms and the strength of the answers of the authors. For those cases where no contributions or insufficient contributions are obtained in the above procedure, we return to the classical procedure and expect referees to provide feedback to the authors. Their anonymity is preserved but their comments are posted on the web page.

In order to protect the anonymity of the referees, it is our policy that the referees will usually *not* be chosen among those who contributed actively to the discussion about an article.This arrangement changes the game considerably, compared with both traditional review and simplistic "open review". In particular, critical comments in phase 1 are welcomed because they are perceived as helping the author to pass phase 2. Also, they give the author a chance to prove his or her worth by answering tough questions.Vicious review comments (which occur from time to time in confidential reviews) are eliminated in favor of civilized behavior between colleagues.

The author is protected against biased and unfair reviews leading to rejection, because she (or he) has a realistic chance of defending her point of view. Incorrect rejection of an article can be corrected later on since the article counts as published (unrefereed publication) from the date of first appearance in the discussion.

Failure to cite relevant work can be corrected in a relaxed manner simply through contributions to the open review phase. They do not delay the publication of the article and are therefore not perceived as threatening. Since they are made openly they do not lead to speculation about who was the confidential referee. Failure by the author to accept suggestions for additional citations are observed by the peer community, if they should occur.

Also: the fact that articles are exposed to open reviewing and that it is publicly known, in the peer community, when an article is rejected, provides strong reasons for authors (and their advisors, in case of graduate students) to exercise self-control and only submit good papers. This tends to reduce the number of contributions of insufficient quality. By comparison, the work that is put into reviewing and rejecting "bad" papers represents a considerable effort with no visible result, in journals using the conventional reviewing procedure.
 
We advertised the ETAI in May, 1997 and received the first submission in July. We now have a thriving discussion environment for submitted articles, especially in one of the participating sub-areas, and I consider that it is possible to make a number of observations about how this scheme works in practice.

The ETAI is produced in both an electronic edition and a paper edition. The first issue of the ETAI contains an editorial note explaining these practices in more detail. Please refer to http://www.ida.liu.se/ext/etai/, click "The Journal", and proceed to "Electronic Edition".