How to review papers?

Each TPC member will have to read a list of papers. Thus, we can here provide a list of pointers on several documents which already discuss the methodlogy to apply:

However, reading and reviewing a paper represent two different objective. A review should:

  • provide a constructive feedback to the authors. Please remind that your explanations must be self-sufficient. The authors cannot initiate a discussion, thus they should be able to understand your comments / critics just by reading your review;
  • be positive: as author, you always prefer a review which criticizes your work in a friendly way. Thus, any feedback such as “this paper presents a stupid solution for a badly identified problem” should be clearly discarded. The reviewer will probably provide a more constructive feedback such as “I don’t know if the authors have already read the papers [1,2] but the assumptions of the authors seem a little bit unrealistic, and may jeopardize the relevance of [..].”
  • help the TPC to take decisions. Thus, the reviewers must detail their opinion. In particular, you should avoid the reviews such as “I don’t really know because I’m not an expert, and it may be interesting but I’m unsure”.
  • hierarchize the critics. Is the critic minor / major / critical? Start your review with few lines, e.g., 5, summarizing your overall opinion on the paper (why you want to accept/reject). Then, you can go into more details. Group your comments into different sections, to make your review more readable.

In your review, you should try to make a distinction between two following types of critics:

  1. the contribution of the paper is flawed;
  2. the contribution is badly explained, but the paper seems technically correct.

Typically, the authors may be able to address the second type of comments for the camera-ready version.

Here are a few resources to help the TPC to provide good reviews: