CHAIR HOW-TO: Reviewer Suggestion

OVERVIEW

Meta-Reviewers can suggest Reviewers they consider suitable to review a paper.
There are different ways to use Suggestions.

1. Suggestions can be used during manual assignment. Chairs can assign papers to reviewers with higher suggestion rank.
2. Suggestions can be used individually or in combination with Bids and Relevance during Automatic Assignment. When used together with Bids and Relevance, Chairs can assign weight.
3. For larger conferences, Chairs can use TPMS scores to create a list of suggestions offline and work with CMT Support to upload the list to the conference (to restrict papers shown to reviewers to bid).

PREREQUISITE

The Meta-Reviewer must have already been assigned papers.

META-REVIEWER SETTINGS

On the Meta-Review Settings page, there are two settings related to candidate suggestions:

1. Enable Meta-Reviewer to suggest reviewers for submission.
2. Do not allow Meta-Reviewer to edit reviewer suggestions for submission.

Chairs can enable the first setting to allow meta-reviewers to enter suggestions. When the deadline for making suggestion is over, they would select the second setting (keep the first one selected).

• Go to the ‘Settings’ tab, click ‘Features>Meta-Review.’ Scroll down and check the ‘Enable Meta-Reviewer to suggest reviewers for submission’ checkbox, then ‘Save changes.’

• From the same ‘Settings’ tab, click ‘Activity Timeline>Deadline’ and enable ‘Meta-Review Submission’ with a proper Deadline, then ‘Save changes.’

WHAT THE META-REVIEWER SEES

• Chairs should at this time request Meta-Reviewers to enter suggestions. In the Meta-Reviewer Console there will be a dropdown enabling the Meta-Reviewer to edit suggestions for each assigned paper.

DEADLINE

• Once the deadline for making suggestions is over, go to the ‘Settings’ tab, click ‘Features>Meta-Review.’ Check the ‘Do not allow Meta-Reviewer to edit reviewer suggestions for submission’ checkbox (leaving the ‘Enable Meta-Reviewer to suggest reviewers for submission’ checkbox enabled), then ‘Save changes.’

REVIEWER BIDDING

In larger conferences, Chairs may want to restrict Reviewers to bid only on suggested papers.

• Go to the ‘Settings’ tab, click ‘Features> Review’, check the "Allow bidding only for suggested submissions" checkbox, then ‘Save changes.’

WHAT THE REVIEWER SEES

• The Reviewer will now only be able to bid on the suggested papers. The screenshot below shows 2 papers suggested for Reviewer A.

MANUAL ASSIGNMENTS

Meta-Reviewers create suggestions that the Chairs can use to make manual assignments.

• For the chair to get to the 'Edit Suggestions' page, go to the right of the row of the paper needed, click More>'Edit Suggestions by Meta-reviewer. The 'Edit Suggestion' page opens.

Chairs see the suggestions in the Candidate Suggestion Rank column of the Edit Assignments (Reviewer) page.

EXPORTING SUGGESTIONS

Chairs may download suggestions in a text file.

• Go to Actions>Export to Tab Delimited>Reviewer Suggestions

• Save the file locally; click ‘OK’.

The file shows the suggestions with Paper ID, Meta-Reviewer email, Reviewer email and rank. This can be used with importing Reviewer assignments in bulk.

ASSIGNMENTS WIZARD

-When assigning papers to Reviewers using the ‘Automatic Assignment Wizard,’ you may give ‘Suggestions’ a weight in the ‘Configure options’ page.

CREATING AN OFFLINE SUGGESTION LIST

For larger conferences, Chairs can use TPMS scores to create a list of suggestions offline.

Prerequisite

TPMS was enabled and scores retrieved.

• Go to the TPMS Download Scores page and Download the TPMS scores

From the downloaded CSV, Chairs can create a list of suggestions for all papers. Chairs then contact CMT Support: support@msr-cmt.org to have the list uploaded into the conference.