Another entry in my series of posts on “things they didn’t teach us in graduate school, because it wasn’t the wonderful art of crafting an argument” is: how to make everyone’s life better when asking people for things. The tl:dr of this post is that you should start from what you want to receive from people, and then ask them for it in a way that makes it easy for them to give that to you.
The point of this post is to help us all be more efficient and less burdensome when we harangue each other for favors. We are a discipline built on imposing on each other, more than we like to admit, so I think this has the potential to help out a lot. I am going to focus on one case, the most familiar case for most of us. But the point generalizes. Sadly, with the case I am going to focus on, the haranguers are a very small class of people: the editors of journals. So while I think it will be instructive as a case study, the only people who can take direct action on this recommendation are people who formulate referee requests (there is a tangent in the middle with more general applicability).
Suppose you get asked to referee for a journal. I’d say many journals ask you for your report in the following fashion (I’ve included a lot of the subtext and/or context, to be helpful):
- Thanks so much for agreeing to read this manuscript! We truly value your advice (though we will frequently opt to follow the other referee’s advice in ways that will not make sense to you, and probably won’t notify you of what happens anyway). Please recommend one of the four official options below, and provide comments for the author. If you have separate comments for the editors’ eyes only, indicate those in the separate area provided. We get a lot of submissions (i.e. when in doubt, recommend rejection). We try to avoid Qualified Rejections that won’t lead to publication, so…(please don’t recommend 3 just because you can’t make up your mind). We appreciate if you can do this within a month (but we are so desperate for people to agree to do this, that as long as you can provide *some* date by which you will do it, we will give you any amount of time you ask for):
- Unconditional Acceptance
- Conditional Acceptance
- Rejection with the Possibility of Resubmission
- Total Rejection
And: this is the sum total of the guidance you are given on how to advise them on whether to accept the paper, because, I guess, we all “know” what a “good” paper “looks like”. Pause for a moment. Reflect on how many people in the discipline complain about receiving bad referee reports. Just really pause.
In addition to all the other obvious reasons we have for why this system misfires, it is hard for me to see these two facts juxtaposed, and not draw some sort of conclusion.
Perhaps we can also reflect on how widely we teach people in the discipline how to write referee reports. This here is the tangent promised above with more wide applicability: Ask yourself whether, even setting aside that people are trying to write such reports as a volunteer task when they are overworked and busy and hurried and such, it would, perhaps, help if the thing they were trying to smush into their busy schedule as a favor to journals was a skill for which we had widely agreed upon norms, and whose norms were routinely taught to students across the discipline, and they were given feedback on their performance, so that, at least, everyone was doing the same thing which they were well trained to do, albeit in a hurried and sometimes haphazard matter? Maybe that would help? I’m just spitballing here.
Anyway, back to what this post is actually about: this request is awful (no offense to journal editors intended, though I am sure that some will likely accrue), because unless the job of an editor is just to tally votes (it is not), this process is designed very poorly to get useful feedback. The only thing it uniformly tells people to provide is the 1-4 verdict, and nothing about what kind of information is desired in the comments! Suppose a referee report is supposed to tell you:
- What is the paper’s main thesis?
- How does it seek to establish this? Is it successful relative to its aims?
- Does it engage sufficiently with existing literature?
- Is there something novel/intriguing about its approach?
Well, then the editor should ask you those four questions instead of asking “for some comments” (or maybe five, with the last one being “any other comments?”). Maybe those aren’t the exact things the editors want to know from you. The thing is: they should figure that out. Because it’s both easier for you to write a report, and the report is more useful to them, if it’s divided up that way (it’s also more useful to the author). Note that in this case—referee reports—doing it this way, literally helps prevent/block off, some of the cardinal referee sins. You can’t sneak your response paper into these questions as easily as you can into a request for an unstructured block of comments.
No most people aren’t journal editors. But this isn’t just for journals. I know “rubrics are exciting, actually” is a hard sell, but, it is absolutely wild how much we don’t think about mutually making each others lives easier/better when we are trying to ask each other to help out.
Does your department have a form that distills all of the department specific requirements and timeline for degree to a single checklist? (if yes, and you aren’t in my department: go thank the person who made it, if not: I promise you it would be useful unless you have very few requirements). When you ask someone for feedback on a paper, do you tell them what kind of feedback you want? I know that this is a hard topic to get jazzed about (unless you are me), reducing cognitive and administrative burdens for requests we make on each other, making your own and each other’s lives easier by structuring your requests so they know what you actually want and you get what you actually want is a cause I can get behind (you can tell because I am currently behind it right this minute)!
In general, when you are asking people to give you feedback, to do something for you, when you are creating a task for someone: Think about what you want the end result to look like. Then think about what you can do to so that your request facilitates them getting you to that end result.
I almost titled this post “the art of asking” but there is a book by that name written by someone whose approach, as I understand it was “get rich and famous, have a super successful kickstarter and raise a ton of money, but then get people to volunteer their labor for you for free anyway, and offer to pay them in hugs/beer” which is not the approach I recommend; my approach is to constantly remind people that service work and undercompensated administrative work is a crucial component of how we are able to keep things functioning, and every part of the job that we enjoy depends on it. It is inordinately valuable to us, and so, we ought, therefore, to value it accordingly.
as long as you are still paying attention, I also recommend reading this piece by Rachel Anne McKinney
Horseless Telegraph 