The Back Page

Peer Review at the Physical Review

By Reinhardt Schuhmann

Courtesy David Tomanek Reinhardt Schuhmann at March Meeting 2008 in New Orleans, LA.

We editors often say, and also often hear, that the great strength of Physical Review and Physical Review Letters lies in the extensive peer review that submitted manuscripts receive. This widely held view is a natural topic for our discussion surrounding the 50th anniversary of PRL. The issue is particularly relevant because recently both Physical Review and PRL have taken steps that have increased the number of papers that are returned to authors without external review. These steps were a response to the relentless increase in submissions and to a generally held view that average manuscript quality has decreased. Another important goal was to address the ever increasing burden on reviewers, the source of our journals’ strength. Thus in recent years more papers are reviewed only by the editors, and it is of interest to examine these issues in historical context.

The Physical Review was conceived in 1893 as a more egalitarian publication than was usual for the time. One can guess as to why. Perhaps it was a reflection of the 19th century American inclination to redefine class. Perhaps it came about because the journal was initially located at Cornell, a relatively new, and quite progressive, institution that admitted women as well as men. (Incidentally, Ezra Cornell, a self-made millionaire of humble beginnings, was an embodiment of the changing social strata of the time.) More prosaically, The Physical Review may have embraced an egalitarian model for the simple reason that it was natural to do so amongst late 19th century US physicists because there were few of wide fame. In any case, we know that at least a few submissions were sent by the editors to external reviewers as early as 1901. Other papers were reviewed by the editors, eventually with assistance from an Editorial Board that was in place by 1913, when APS assumed responsibility for The Physical Review. As was common practice at scientific journals around the turn of the last century, most papers were published or rejected without extensive review. Decisions about what to publish and what not to publish were to a large extent made solely by the editors.

We know that by the 1930’s peer review at the journal was more established. Ledger pages from the time contain the same basic information that we now store in our computer: date of receipt, date sent to a reviewer, date returned, date published or rejected. From these we see that many papers were sent out for expert evaluation, but also that in many cases the expert assigned to a manuscript was the Editor, John Tate. We also learn that many papers had no referee assigned to them, and that some of these were accepted and some were rejected. Thus during this period peer review was growing, and making a larger contribution, but decisions were still often made by the editors alone. 

This situation apparently continued for many years. In the early 1960’s, when the APS journals were located at Brookhaven National Laboratory, Editor Simon Pasternack would obtain local input from physicists who worked at the lab. A memo in use at the time clearly allows for rejection of a paper without formal peer review. One of its options was “It should be refereed,” indicating that the contrary possibility was also viable. 

So far this discussion has been about The Physical Review. The history at Physical Review Letters is somewhat different. First, one must consider the basis for PRL, the Letters to the Editor section of The Physical Review, which first appeared in 1929. From 1929 through the first half of 1958 it included a disclaimer from the editors: “The board of editors does not hold itself responsible for the opinions expressed by the correspondents.” This statement carries the implication that Letters to the Editor were not reviewed. This is supported by the fact that no Letters to the Editor appear in the ledger entries mentioned above. It is also clear that not all submissions to the Letters to the Editor section were published: By the 1950’s, the section had grown to such an extent that the Editor, now Sam Goudsmit, took steps to reduce their numbers, which of course means that he turned away some Letters to the Editor without external review. So, manuscripts that appeared as Letters to the Editor were chosen entirely by the Editor.

It is interesting to note that the primary motivation for the Letters to the Editor section was speed of publication. The description of the section offered “prompt publication ... of important discoveries in physics.” The absence of review was a strategy aimed at minimizing delay. The growth in Letters to the Editor mentioned above likely occurred because of an increasing association of “prompt” and “important” in the community. The idea that important work was published quickly began to be turned around: work that was published quickly was important, apparently, by definition. Goudsmit indicated his understanding of this logical reversal by noting in the 1 May 1958 announcement of the upcoming birth of PRL, “Such a fast-publishing journal may become very popular with authors and could  soon grow beyond reasonable bounds.”  

When PRL began, it initially followed the practice established for Letters to the Editor. In an editorial in the first issue of PRL, Editor Goudsmit states that “most of the decisions for acceptance...will have to be made in the Editor’s office.” Naturally this means that decisions against acceptance were to be made there also. This original intent quickly shifted, however, as PRL grew, and a few months later, 1 August 1958, Editor Goudsmit wrote that the journal was “obliged to send to referees many of the submitted Letters to ascertain whether their contents require rapid publication.” Consultation with single referees grew through the sixties, and in the early seventies a shift to simultaneous consultation of two referees took place.

Once again, the motivation for these changes was speed of publication. The editors found themselves overwhelmed by the number of submissions, and in order to move things along found it necessary to distribute the decision-making process among some referees, initially one at a time. Consulting a single referee did not always work, however, because sometimes referees did not respond. Thus the use of two referees was initially a means to insure that at least one report appeared. Because Letters were seen by authors as important and prestigious, however, a tendency arose to conclude that all papers must have two referees. This point of view is often expressed by authors today, in particular if there is only one, and he or she opposes publication. It is not, in fact, a policy requirement of PRL today. While in practice most Letters are reviewed by more than a single referee, it has always been well within standard practice for an editor to make a decision on the basis of a single report.

There is a somewhat ironic aspect to the increased inclusion of anonymous review as a means to insure prompt publication. Our statistics consistently show that time with referees represents about half of the time required to review a manuscript, with the other half divided roughly equally between authors and editors. Publication today is somewhat different than it was many decades ago, because it is now possible, e.g., via the ArXiv, to distribute results immediately, without peer review. The steady increase in submissions to PRL shows that promptness is no longer as large a consideration for contributors, although naturally authors still want a quick decision, especially a favorable one. For better or worse, some journals are viewed as more important than others, and publication at the highest possible level is preferred. It seems that journal importance is thought to prove individual manuscript importance, although in general the range published by all journals is very broad.

Three other points are of interest in the context of level of review and speed of publication. Beginning in July 1964 and continuing into the 1970’s, PRL submissions that covered high-energy physics experiments were accepted without review, if they met certain simple criteria, including a cover letter from a senior administrator at the home institution. The basis for this was the fact that high energy experimental groups had many members, and were few in number, which led to many (unwarranted) accusations of unfair referee behavior. Authors felt that any potential referees who were not coauthors were fierce competitors, so unbiased review was not possible, and this policy was an attempt to reduce acrimonious exchanges by immediate publication. It was always applied at the discretion of the Editors, and it has been decades since any paper was accepted in this way.

On the other hand, in March 1969, faced with continued growth and with financial pressure, Editors Goudsmit and George Trigg wrote that while “in the past, most borderline cases, when referees’ opinions differed, were decided in favor of the author,” they “could no longer afford that luxury.” Finally, in the late 1980’s, in response to a flood of submissions relating to high temperature superconductors, the journals established a temporary advisory board, to make quick decisions about submittals on this topic. This board acted similarly to the 1913 Editorial Board mentioned above, making quick recommendations to the editors either for or against publication. The three events demonstrate that in some cases during these years decisions were made largely by the editors, sometimes without extensive review.

So, what can we conclude from this? Certainly we can say that throughout its history, the editors of The Physical Review and Physical Review Letters have made decisions about publication using some referee advice. We may also state that referee input has grown over the years. Further, we see that the editors have adjusted their reaction to, and usage of, input from referees for cause, e.g., to control growth of published pages. We find it reassuring to revisit the considerable precedent for our recent efforts to turn away some submittals without external review.  

The proof of the pudding, however, is in the eating, and it is also reassuring to note the considerable evidence that these efforts have been successful. Certainly the roughly 20% of submittals to PRL that now do not go out for review has reduced the burden on our pool of referees. In addition, our early decisions have allowed manuscripts to find homes in more appropriate journals without undue delay. They have also sometimes inspired authors to take another look at their manuscript, and improve it, occasionally to the extent that the manuscript becomes appropriate for one of our journals. Finally, early decisions have had no obvious impact on the quality of the published journals, and have not diminished interest among authors in publishing in them. Overall, early decisions have proven themselves to be beneficial, will continue, and should probably increase.

This does not mean that we intend to abandon peer review, which we still believe to be essential to the success of the APS journals. Most submitted manuscripts will continue to be reviewed. Input from referees is invaluable to us as editors, especially when it provides a substantive basis for any assessment, because it provides essential information to help us choose what to print. We also will keep in mind the fact that referee effort is a finite resource, and will work to apply peer review judiciously, so as not to overburden referees.  

Referee reports also help authors, because they lead to improved manuscripts. It is not possible to keep precise statistics for “manuscript improvement,” since it is inherently subjective. What one person views as significant change another might view as a minor clarification. It is fair to say, though, that most submitted manuscripts undergo revision in response to referee reports, and that a substantial number of those are changed significantly for the better, both in terms of presentation and in their substance. Many papers submitted to PRL eventually appear in another journal, either one of the sections of The Physical Review or elsewhere, and the revisions are valuable there as well. 

Finally, peer review is useful to the entire community, because manuscripts that are both more readable and more rigorous do a better job of communicating results to readers. Peer review is and will remain an essential element of the APS mission “to advance and diffuse the knowledge of physics.” 

Reinhardt Schuhmann received his Ph.D. in experimental condensed matter physics from Clark University in 1988. He joined the APS journals in 1990, worked for Physical Review A for a year, and moved to PRL in 1991, where he is now Managing Editor.