As an editorial collective member of ACME: An International Journal for Critical Geographies and as someone who once managed WSQ: Women’s Studies Quarterly for three years, I know how difficult it is to find appropriate and available peer reviewers. I often seek out graduate candidates (ABD students) who would offer that strong expertise but may not have the have reviewed journal articles or many journal articles before. I remember how awkward and nervous I was–and how many, many hours I devoted (oy)–when I wrote my first peer reviews.
Thanks to various search engines, I’ve read quite a few posts on how to write peer reviews. Many of them are written by publishers, peer review corporations (yeeghads!), or from other academics. These are all helpful in that they structure the work of peer review, but I found the former to be too detailed and formal, and then more anxiety-producing than clarifying. If they were brief, like the academic perspective, I found myself unclear about how they expected to accomplish each step. I’ve cobbled together my own thoughts about how to do a peer review that comes from my own experience as a gesture of support and solidarity for graduate students, postdocs, and early career researchers engaged in critical and radical research who wish to be part of the project of producing knowledge through peer review. My own take as a social scientist offers an organized response through the parts of–surprise!!–a social science paper, which I have not found mention of as of yet.
Before I go on, my first tip is that each peer review should take no more than two to six hours. If you spend the maximum number (6) on your early peer reviews, then that number should significantly decrease over time as you perfect your own approach to reviews. A six-hour review imagines you read the paper three times and then type up your notes. My second tip is that your entire review can be a page, preferably, or two pages long. You’re wondering if I’m serious but how would you feel if you wrote a 20-page paper over months and someone gave you five pages of single-spaced feedback? Exactly. One or two pages is a lot to chew on. Finally, as you read think about making summary comments and identifying trends (in style a la too many commas or overciting, in writing a la a vast absence of methods, etc.) rather than line edits.
I’ve been asked to do a peer review. What should I do first? First, make sure this article is a fit. Can you speak to the topic because it close to your own research and interests? Is this a journal you have or may one day publish in? If not, say no, because you don’t have the time, energy, or sanity to read into a new topic. Do you have time to review the article? See above. If not, say no. Have you done more than two or even four of these already in a year? It’s up to you, but I do about three reviews a year if I’m not on any editorial boards. (If you are on a editorial board, you are expected to that many per journal.) Key takeaway: it’s always okay to say no. Studies show that women tend to most of the labor for peer review, just as they handle most of the social reproduction. Then there are the politics of working with journals. I know some people who don’t review for Elsevier journals and I think this is a great politics as Elsevier touts this academic labor as their own “value added” labor, and the basis of why they charge libraries thousands and thousands for access to their materials. If it’s an open access (OA) journal or a feminist, black, queer, disabilities, radical, critical, anti-colonialism, Latinx, Indigenous, Asian studies, etc.-focused journal, there’s even greater reason to share your labor there.
I’ve said yes, and I’m totally unsure what to do second. Read the article but quickly. My best advice is not to ruminate on every word but to read through once quickly and take in what the article is telling you, scribbling a few words or phrases per page in the margins (max!). Now read it once more, and don’t let yourself stop, again, except to had maybe five more notes. You’re noticing patterns in your notes, aren’t you? Most excellent! Because indeed there are always trends and it’s these systemic patterns you’ll deliver back to the author that will help them most with this paper and their writing overall.
What the heck do I do now that I read it? Are you sure I read it enough??? STOP READING as I am sure you know what it said. Before you go on, ACME‘s first question it asks its reviewers is about suitability: does this article fit this journal? If not, you needn’t do a review and can say why. If you feel like being generous with your time, offer a longer review. For example, I reject at least two papers per year as an ed board member of Gender, Place and Culture as that journal has a high impact factor and folks submit anything they have on gender that notes a place and/or culture. I usually check if they cite any feminist geographers. As that is the very basis of GPC, if they do not, I reject them. (Sigh, and I usually write an encouraging paragraph or two for the author for their rewrites to send elsewhere.)
But I think this paper didn’t make sense! I need to re-read! If it didn’t make sense, it may not make sense and that is not your fault. The first paper I rejected as a peer reviewer I read five times before writing the most polite rejection I could concoct. The editor told me I should be a bit less angsty next time and save myself, may the goddess bless her.
Okay, so how do I write this? For me, I find it easiest to take notes through each section and take maybe a few sentences MAXIMUM notes on each. My practice is to jot down handwritten notes (because I would type WAY too much) before I summarize them in typed form. I always follow the following, always conscious of the kind of scholarship published in a journal and the field generally:
- Introduction/Conclusion: Does the paper have a clear argument? Based on your reading, write down in a sentence or two what that is. Don’t write it for them–that’s their job! But it’s great to say what you hear from the paper so that the author can edit to meet that argument or adjust to make their argument clearer. Again, if they don’t have an argument, it’s not your fault to make one. The utterly fabulous Barbara Katz-Rothman once said to a class I took with her that if she didn’t get to an argument by page 3, she rejected a paper. Since she’s BKR and a pretty big sociological deal, that’s fair. You may like that approach too; I’m still working on it. At the same time, there are some lazy scholars who write partial papers to get feedback and I have rejected a couple of those in my time too. This is where ACME‘s second and third queries for its reviewers fit: is there originality (a unique argument) and clarity (that argument makes sense, hopefully with just one argument at that) in this piece? If there is originality, it cannot be published; if so, note what sticks out to you as new. If there is no clarity, it needs significant revisions.
- Theory: Who does the paper connect with and why? Just write a few words down (performativity, everyday, social reproduction, homonationalism, etc.). Of course, more applied pieces may lack this element! At the same time, the ability to connect our work through theory makes for great conversations. If you think another or a different theorist would be of help, note that for the author.
- Lit review: Who is left out? Who is written of too much? Articles by junior scholars tend to cite everyone and then some; perhaps you can help them by pointing out a key author or two (or five if there is a significant hole in the lit) you know of, or recommend cutting just as many. Are they citing people and issues relevant to the journal? Is it just another paper about David Harvey (who was a solid teacher of mine by the way)? Are they citing work from the journal and from key folks in the field / on the topic? If not, rethink if this is a fit.
- Methods & Analysis: Do they describe methods and analysis well enough? Are you convinced they have done enough work to make the claims they are offering? I once reviewed a piece on mental mapping decrying that all personal spaces could be discerned from a certain way of drawing maps. There was no mention of the race, age, gender, sexuality, or drawing skills of participants, let alone how they collected the data. I wasn’t convinced that the size of the argument (all mental maps reveal this one thing) fit their data; I was convinced these maps said a lot about the region they studied and said that, and I added they needed to clarify their methods further–bam, onward with my own work (or sleep) (or friendships) (or Netflix). You should be convinced. ACME‘s fourth reviewer query: does the paper possess rigor, as in: is this paper convincing based on the current literature? If not, it should not be published. You may want to say, well mayyyyybeeeee this author didn’t…just relent. What’s our mantra by now? Don’t do the work for the author! Yeah!
- Findings: Are they any? Again, are they original? May be combined with the next section.
- Discussion: Does the author actually address the literature (already discussed! clearly!) and use their findings to take the literature / theory further? Do they clearly say which literature / theories they are speaking to? Are their arguments convincing? Do they match the intro and conclusion? Here is ACME‘s fifth and final ask of reviewers: does the paper have merit, i.e. does it meets the goal (treatise, evidence, theoretical contribution) it sets out?
- Citations: Are there too many, too few, or any glaring omissions (both in not citing a key source or leaving out citations they mention)? Just skim at most for this.
- Abstract (yes, leave for last): Does it fit the article’s argument? I find that 80% of the time it does not in a draft form; just make a note for them to update it.
- Note any wonderful and/or painful writing elements you experienced in your reading.
How the heck do I actually write a review?
- I begin by writing three sentences about what I think the paper is about. I draw directly from the intro/conclusion/arguments jotted down above.
- I look over my notes and choose which areas you think the author really needs to focus on. Write two to five VERY BRIEF paragraphs (MAX) of two to four sentences each (MAX) saying where they should focus and why.
- Add three to twelve bullet points of other things they need to consider, if any. Egregious typos are kindly shared here.
But what should it sound like?
- Be helpful, kind, patient, and encouraging. Imagine you took years or months to do what was summed up in the paper before you and what you would want to hear. Of course, if you get a paper by someone who did absolutely no work, no need to be terse but no need to give much of your time either.
- Be balanced to help them balance. Usually an author will have no mention of methods but a great argument, or superb methods and two major ideas in a paper that only has room for one. Tell them that.
- Be on time-ish. A couple of weeks late is fine. Even ask for extensions. But don’t hide for months on these. Editors can always find another reviewer if an emergency appears BUT RARELY DO THIS. Your reputation is the most important thing you have. So, better yet, you can write this review, gasp, quickly, and not hate yourself for Getting It Done.
I’m but, wait: really, Jack–no line edits?? I know this to be true because of this experience: I met someone a few years ahead of me at a conference. They were making fun of a “ridiculous” review of line edits to their article manuscript they had just gotten back, and I laughed along truly and said haha yeah, I’ve seen those! These edits were helpful, this author said, but bordered on absurd. She then described the edits, and how most were line edits and on and on–stating what I had written a few weeks before. I’m sure I went pale as I gulped and said, Um, I wrote that. And they froze and I froze and they apologized and I said no, no, that’s fine, I actually do have better things to do with my life. I also know this to be true because of my work at ACME and my three years managing WSQ: no one writes line edits. Oops. Of note: we are friends now. Funny, that.
That’s it? One more thing: accept or reject. Here are the ACME six options below (and some journals give you a narrower three or four). I would say that 4 out of 5 times, a paper is “accepted subject to the major revisions outlined in the report” so let that be your default and go from there. What’s the difference? I’ve only seen no revisions once since I started doing peer reviews in 2004 so it’s best to imagine it’s not an option. Minor revisions would just be some typos and clarifying their methods and you need to cite this piece from Gerry Pratt or Heather Love too. Major revisions / declined with encourage to substantially revise and resubmit is something like I can’t tell what the argument is and it doesn’t cite x, y, and z authors, and is missing half its citations but I know there is something in there; which way you go is up to you. The options are as follows:
- Accepted in its current form
- Accepted subject to changes as outlined in the report
- Accepted subject to the major revisions outlined in the report
- Declined with encouragement to substanially revise and resubmit
- Declined with advice to submit elsewhere
- Rejected without qualification
I submitted it! That’s it? That’s it. Do note on your resume/CV that you reviewed for this journal. Whatever work you go into, it indicates the kind of conversations you are in and where you see your contributions.
Thanks to Jess Linz and Rachel Goffe who gave me the idea to write this. Go Wildcats. And go CUNY.