Nearly every academic will encounter scholarly peer review during their career, either as graduate students, research assistants and postdocs, adjuncts, or tenure-track or tenured faculty. Peer review takes many forms and is used in many contexts, from conferences and grants to academic publishing.
It’s something that is the subject of countless “reviewer two” memes and the source of sometimes great frustration as well as yet another “service” obligation that is rarely rewarded and recognized by one’s institution.
“Reviewer 2”-focused memes such as these are commonly seen in online academic communities.
Peer review is also, according to a recent article in The Chronicle of Higher Education, a skill that “almost no one is teaching.”
Perils and Problems
Our own experiences with peer review have varied depending on our education, work experience in various countries, and the conventions that exist within the sub-disciplines of communication that we explore. This section shares some vignettes from our experiences with the peer-review process that speak to how scholars-in-training are socialized into the peer-reviewing process and some of the associated problems and issues we’ve encountered along the way. After presenting these vignettes, we provide some evidence-based recommendations from studying a year’s worth of peer reviews from a U.S. communication journal.
The 32-Month Wait: T.J.’s Experiences
I didn’t have a dedicated class in peer review but did have some informal learning opportunities with the process while a graduate student at the University of Missouri. One such example was when a senior faculty member offered to complete a peer review together with me. We both independently read the piece, wrote our feedback, compared notes, and then he let me watch over his shoulder while he typed up the final review. A few professors also shared in various seminars examples of peer reviews they had written or received to try and help us learn more about the process and how best to approach it.
Even with the benefit of this extra attention to and insight into the topic, I still encountered my fair share of problems with the process as my academic journey progressed. While still a graduate student, for example, I submitted an article to a journal and waited a staggering 32 months to get the first reviewer feedback on the piece. As a graduate student trying to differentiate oneself in a competitive job market, the pressure to publish is particularly intense. But delays such as these, caused in part by a lack of experience in knowing where to try to publish one’s work and in part by editorial neglect, can have a real effect on a graduate student’s job chances and future academic career.
There’s Always a Person at the Other End: Lesley’s Experience
Most of my academic career has been in teaching. When I was asked to review some articles for a conference, I thought it would be a perfect way to explore the idea of peer review. I spent a lot of time reading and responding to the work. However, after I had sent off my comments, I realized that I had graded the work rather than reviewed it. On reflection, I understand why I took this approach. I did what I do best and what I have done for a long time. I think there was an assumption that because I worked in academia, I would know what to do. My career as a researcher is recent. After talking with colleagues, I realize that my experience with reviews is not uncommon. I have received very lengthy reports and much shorter ones. It can be confusing. For example, should you respond to all reviewer feedback or accept that sometimes they are merely providing an alternative point of view or approach to the study? I have heard both ideas. The one piece of advice that I hold on to comes from a colleague who said: “Just remember there is always a person at the other end of your words.” I think this is a good mantra before giving any type of feedback.
Reviews Are Not the Last Word: Glen’s Experience
Peer (and editorial) review can be such a frustrating process. My first peer review, long ago now, was the classic Reviewer1/Reviewer 2 situation, where Reviewer 1 recommended some changes, additions, and new ideas, but Reviewer 2 basically suggested re-writing the entire article. I showed the reviews to my good friend and mentor, who told me to ignore Reviewer 2 and just focus on the Reviewer 1 feedback. His sage advice was the right approach: the article was accepted, and offered an insightful lesson of not trying to please all reviewers all the time. Reviews can also be unintentionally comic. My wife, who also writes academic papers, once received a review that scolded her for not citing [wife’s name] in her research as a glaring oversight. Perhaps modesty is not an academic’s best friend?
Vague advice is just as hair-pullingly frustrating. Recent editorial advice to me has suggested “polishing up this section”, which is exactly the situation Lesley described as grading the submission, not offering peer review or insight. Polish where? In what way?
The advice I always pass on to colleagues, students, and anyone who will listen is the same advice that was given to me as a Ph.D. student. Your work won’t be perfect; not everyone will like it, but that’s fine. This is not the last word you will have on the topic, or the last time you will engage with the ideas.
Keep in mind, when the reviews come back, that this is not the end: you’ll write again.
Other Problems
Other problems with peer review include: the power (im)balance between submitters and reviewers and editors; the potential for acerbic or unduly critical feedback; and the general lack of consistent standards and expectations for what “good” peer review looks like, how it is accomplished, and what various stakeholders’ roles are in the process.
Peer review, as has already been acknowledged, takes myriad forms and can look and feel quite different across various contexts. It is a complex and opaque process, but one that we’ve been trying to better understand within the academic discipline of communication. We want to help improve the process by getting more academics to talk about and advocate for better peer review practices.
Data-Driven Recommendations
When we sought out data to better understand the process, we realized that very few peer-reviewed publications examine the process itself. As a result, we worked with a journal editor in our field who gave us access to an entire year’s worth of reviews for articles submitted to the journal. With these data, we systematically examined reviews, and the data provided us with insight into peer review that would not otherwise be available. We use this exploratory approach to present some evidence-based recommendations.
Recommendations
Reviewer Training
It’s important to have dedicated training opportunities to help scholars understand what good peer review is and how to perform it. While some scholars are fortunate to have mentors who provide informal guidance, dedicated training would ensure more consistent understanding and application of peer review standards.
Clear Guidelines
Journals should provide clear guidelines and expectations for reviewers. This includes not only the technical aspects of writing a review but also how to provide constructive and respectful feedback. Consistency in guidelines can help in standardizing the quality of peer reviews.
Recognition and Reward
Institutions should recognize and reward the important service that peer review represents. This can be in the form of formal acknowledgment, incentives, or even professional development opportunities for those who regularly participate in the peer review process.
Transparency
Greater transparency in the peer review process can help demystify it for scholars. This might include sharing anonymized examples of reviews, providing clear communication about the process, and explaining editorial decisions in a detailed manner.
Improving the peer review process requires collective effort and open dialogue within the academic community. By sharing experiences and advocating for better practices, we can contribute to a more effective and equitable system for scholarly communication.
From left to right: Glen, Lesley, and T.J.