On 18 November 2020, “The American Sociologist” has published an interesting study by Ben Merriman, a sociologist at the University of Kansas, who examined 351 annual reports from editors of the official journals of The American Sociological Association (ASA), the most prestigious association of sociologists worldwide. These materials allowed him to reconstruct the history of peer review at sociological journals from the 1950s to 2018, including the main flagship journal of this association, i.e., the American Sociological Review (ASR).

peer review: handle with careMerriman identified certain important historical changes in the organization of reviewing. The first was the shift from reviewing by editorial boards to peer review as we know it today, occurred rapidly between the late 1960s and mid-1970s. In the 1960s, about 90% of reviews at ASR were produced by editorial board members, who routinely delivered 30 or more reviews a year! This stable group of reviewers were instrumental to help editors avoid erroneous judgment. Exclusive submission and editorial timeliness were originally part of kind of a reciprocal obligation between authors and editors.

Merriman showed that the management of the revise & resubmit decisions were critical to the shift from a board-centred to an author-focused, developmental model of peer review by the late 1970s. This shift was an organizational adaptation to a very sharp increase in the number of submissions. Note that these changing practices reflected the collective sharing of responsibility on criteria for publication in the larger community of experts while reducing editors’ ability to shape the overall direction of a journal. At the same time, these changes did not affect the canon of double-blind review at sociological journals, which was established since the mid-1950s.

Double-blind peer review evolved as a social norm serving an intellectual aim rather than an organisational necessity, such as the practices mentioned above. Merriman suggests that the greatest virtue of the developmental model of double-blind peer review has always been “social equality”, i.e., the fact that the access to expert opinion is open to all authors and refers only to their work, although at a significant time costs for editors, reviewers, and authors.

While double-blind review served a deliberate purpose, later developments reflected more an effort to balance two organizational constraints. The first was to control the amount of work to be carried out by the editor-in-chief, i.e., an academic appointed by the association to edit a journal on a voluntary base. The second was to speed up editorial decisions while preserving the idea of peer review as a constructive process aimed to increase the knowledge value of manuscripts by collaboration between previously unrelated experts, such as authors and reviewers. Merriman argues that peer review was a procedural safeguard for authors: on the one hand, reviews were increasingly used by editors to support their decisions; on the other, reviews were a service for rejected authors to help them improving their work. Note that the practice of desk-rejections to mitigate workload problems associated with the developmental peer review, especially the publication delay, began only in late 2000s, as well as scholars’ diminished willingness to accept invitations to review. In my understanding, these were adaptive practices reflecting the rise of academic hyper-competition and the proliferation of academic duties intrinsic to the university beaurocratization, which made minimizing publication delays and using time for more rewarding activities, the top priorities of academics.

Note that these multifaceted functions of peer review and its sensitivity to the changing contexts of the academic system were also found in a historical reconstruction of the evolution of editorial processes at the Royal Society journals, published in Science, Technology and Human Valuesby a team I have part of (see Fyfe et al. 2020 in the reference). This study of ours had a historical coverage of a century from 1865 to 1965, so only minimally overlapping with Merriman’s coverage and included journals from other fields than sociology. The history is the same: certain norms persist over time, others adapt to the changing circumstances, including increasing pressures due to the number of submissions.

What is the message here? The message is that peer review did not arise from deliberate plans or social engineering, but it has been more the outcome of a mix of deliberate norms and organisational practices, the latter reflecting the changing circumstances of the academic system. Whenever we want to judge peer review also today, we always must remember that this important pillar of science is not isolated from the rest of the institutional system that permeates the academic community. Peer review reflects a constitutive evolution process, including different interests, constraints and priorities. If we want to take this interesting study seriously, we must continue to handle peer review with care!

Flaminio Squazzoni, ReviewerCredits Peer Review Scientific AdvisorFlaminio Squazzoni  [email protected]
ReviewerCredits Scientific Advisor
BEHAVE Lab, Department of Social and Political Sciences, University of Milan, Italy

 

 

References:

  • Fyfe A, Squazzoni F, Torny D, Dondio P. (2020) Managing the Growth of Peer Review at the Royal Society Journals, 1865-1965. Science, Technology, & Human Values, 45(3):405-429: doi
  • Merriman, B. (2020) Peer Review as an Evolving Response to Organisational Constraint: Evidence from Sociology Journals, 1952–2018. The American Sociologist