How to improve the review process when there are counter-proposals?

When SE-270 was accepted, the review manager invited a discussion about the review process:

From the full text of the acceptance announcement, it's clear that the core team is dissatisfied with something about how the review went. I interpret what I read there as follows—which may be inaccurate; please feel free to correct me:

  • The core team thinks the author's response to counter-proposals could have been more complete.
  • The core team is concerned that when reviews get very involved, potential reviewers tune out.
  • The core team is concerned about the effort it takes from authors to get a proposal accepted.

I'd like to point out first that this was not actually a drawn-out review. There were just 15 posts over the nine days between the announcement of the review starting and its resolution. If the process of getting the proposal accepted was exhausting it's because it actually got three back-to-back reviews, in quick succession. Given that review feedback resulted in substantial revisions each time, it seems clear to me that the proposal should have gone back to the pitch phase for more collaborative design work, but instead we returned to find yet another review had started, with a new revision of a large proposal.

Having worked with most of members of the core team for many years, I know and respect them, and draw no conclusion about why the proposal was handled this way. That said, to someone less connected it could easily appear that core team had decided the proposal needed to pass, and if there were objections, it was just going to keep running reviews until everyone—including core team members—got so sick of the arguing that they decided to “just accept it already.” Therefore, I think handling feedback this way is bad PR, discouraging to reviewers, and so tiring for everyone that it can't help leading to worse results for the language.

In the end, I didn't say anything about counter-proposals because I don't think they're at the core of a problem—I just put that in the title because it's what we were invited to discuss—and therefore I don't know if any of what I've written actually addresses the core team's concerns. But that's my take on what could have gone better, as a reviewer.

11 Likes

2 posts were merged into an existing topic: Evolution process discussion

Also, for what it's worth, Nevin did start a thread on this already. Would you mind if I moved these posts over there?

Not a bit; please go ahead!