CHAIR HOW-TO: Automatic Assignment of Meta-Reviewer
Assigning Meta-Reviewers to Papers can be done with the Auto Assign Meta-Reviewers. This will be broken down in the following sections:
- Select Papers
- Select Reviewers
- Configure Options
- Confirm Assignments
Use the status filter in the Chair Console to filter out unwanted papers, such as Desk Reject.
There must be Reviewers in the conference before using the Automatic Assignment Wizard.
Click Actions, then select Automatic Assignment>Meta-Reviewer.
Emails for assignments are not sent out automatically. They must be sent out manually.
From the Automatic Assignment Wizard page, you select the minimum number of Meta-Reviewers per Paper. You can first do this in bulk and then edit individual papers if needed.
- To do it in bulk, enter the minimum number of Meta-Reviewers per Paper, then click apply. In this case, we chose 2 Meta-Reviewers per Paper.
- This sets all selected Papers equally to 2.
- Now, we want to adjust the minimum number of Meta-Reviewers for 2 papers. Let’s say you want Paper ID 28 to have a minimum of 4 meta-reviewers, and Paper ID 26 to have a minimum of 1. Enter the number of meta-reviewers in the corresponding rows and click ‘Next.’
Here you select the maximum number of Papers per Meta-Reviewer. You can first do this in bulk and then edit individual papers if need be.
- To do it in bulk, enter the maximum number of Papers per Meta-Reviewer. In this case we chose 4. Click apply.
This sets all selected Reviewers equally to 4.
Sometimes there are quotas set for some meta-reviewers. For example, MetaTwo@cmt.cmt has a quota of 1. Quotas will not be exceeded when setting maximum number of papers.
- As before, set the maximum number of papers to 4 and click apply. You see that maximum number of papers is set to their respective quota instead of 4.
- You can also edit the maximum number of papers per meta-reviewer individually. For example, you can set firstname.lastname@example.org to handle no more than 5 papers. When you are satisfied with the maximum number of papers for each meta-reviewer, click ‘Next.’
You have the option to run the Auto Assign Meta-Reviewers with default settings or with your own configuration.
The Automatic Assignment algorithm uses reviewers’ bids, candidate suggestions, subject area relevance and TPMS scores to suggest assignments.
By default, it uses all three above and assigns certain weights to them. You may specify your own custom weights (must be between 0 and 1) to obtain desired assignment result.
- Click on the ‘Custom’ radio button. Then enter the weights you want to assign, making sure the weights add up to 1.
- Click ‘Run Automatic Assignment’.
- A progress bar will be visible. When progress shows ‘100%’, click ‘View Assignment Result.’
The ‘Possible assignments suggestion’ page gives the results. The detail section has 2 tabs; ‘By papers’ and ‘By meta-reviewers.’
- This is the view of the ‘By papers’ tab. It shows meta-reviewer suggestions for each paper.
- This is the view of the ‘By meta-reviewer’ tab. It shows paper suggestions for each meta-reviewer.
- If you are happy with the results, click the ‘Save assignments’ button. Click ‘Start over’ to re-run the Auto Assign Meta-Reviewers from the beginning. Note that all values and filters that were entered will be lost when starting over. No assignments will be made without clicking the ‘Save Assignments’ button.
- This wizard can be used for incremental assignment. This wizard can be run at different/multiple times to gradually add more meta-reviewers
- Use ‘Paper status’ to set minimum number of meta-reviewers for different groups of papers in automatic assignment.
- In the ‘Settings’ tab of CMT, select Features>Paper Status to create new Paper status labels (for example, ‘Group1’, ‘Group2’, etc.)
- On the ‘Submission’ page, change the Status of the papers belonging to the group, to ‘Group 1.’
- Apply filter (‘Group 1’, ‘Group 2’, etc.) for the status column in the ‘Select papers’ section of the wizard and set the minimum number of meta-reviewers in bulk, respectively.
Import Meta-Reviewer Assignments