FAQ-Chair: Manage Meta-Reviewers

Why can't Meta-Reviewers view their assignments?

There are a number of reasons why Meta-Reviewers cannot see their assignments.

1. Chairs need to enable Meta-Review Submission Activity and set an appropriate deadline (for each track separately).

2. Make sure no conflicts have arisen.

3. Make sure papers are assigned .

4. Make sure papers are not withdrawn, desk-rejected or rejected.

5. Make sure the 'Meta-Review: Read Only' column in the Chair Console is set to 'No'.

6. Make sure there are Meta-Review questions for Meta-Reviewers to answer otherwise there will be no 'Enter Meta-Review' link.

Why do Invited Meta-Reviewers get "invitation has expired" when clicking the accept link even though the date of expiration is set in the future?

"Invitation has expired" appears either when the link actually expires or because the invitation was resent. The user is clicking on the previous invitation's links. It is also important to know that if a user is imported into the conference, there should be no invitation sent to the same user. Look at the Meta-Reviewer Invites page.

Can Meta-Reviewers invite Reviewers?

Meta-Reviewers can be enabled to invite Reviewers only for specific submissions. Inviting Reviewers is only supported for Chairs and Track Chairs.

Can Meta-Reviewers set their own quotas?

If enabled, only Reviewers can set their own quotas when accepting an invitation from the conference.

How can we assign a Meta-Reviewer to a Senior Meta-Reviewer?

Chairs can import SMR-MR assignments. From "Users -> Senior Meta-Reviewers", Chairs can use the "Actions -> Import Assignments" menu option to import assignments. On the same page, Chairs can also use "More -> Edit Assignments" to edit assignments for individual SMRs.

Are Meta-Reviewers automatically notified of their assignments?

No, there are no automatic emails sent out. They must be sent manually.

Why do email template placeholders show in the body of the email?

Placeholder names show because you are sending the email from a different page than where the template was created. Each page that you email from, may have different placeholders. Not all placeholders will work in all pages.

Why are the Meta-Reviewers unable to enter ratings for reviews?

Make sure this setting is enabled. Additionally, the Meta-Review Submission activity must be enabled with proper future date for Meta-Reviewers to enter their ratings.

Why can't Meta-Reviewers find their conferences after they log in?

This is more than likely because they are logging in with an incorrect email. Please note that all conferences from 2018 and prior have been removed from CMT. The data is irretrievable.

Why are Meta-Review questions grey-out?

Go to the Meta-Review Forms, find the question they cannot answer and click 'Edit.' Make sure the 'Locked for edit' box is unchecked.

Why are Meta-Reviewers unable to see reviews?

When a review question is added to the review form, the question’s visibility needs to be configured as being "Visible to Meta-Reviewer". Chairs can mark them as visible using "Edit" icon corresponding to the review question.
If Meta-Reviewers are to also see aggregate values, the 'Show aggregate columns on consoles' setting needs to be enabled in Meta-Review settings. Note the review questions need to be value-weighted.

I invited a Meta-Reviewer and it was accepted, so why do I just see a string of alpha-numeric characters in all of the fields?

This is because after the invitation was accepted, the user deleted their CMT account.

We are going to use a triplet of Meta-Reviewers. How can they see each other's assigned papers?

Go to Users>Meta-Reviewer Group and create group manually by using the “Create New Group” button. “Implicit” needs to be selected to allow triplet members to see each other’s assigned papers. You also want to make sure triplet members have no conflicts with other members’ papers.

Why are Meta-Reviewers seeing Desk Reject papers?

CMT does not automatically filter out desk reject papers from reviewers. In the Meta-Review settings page, select 'Do not show desk reject papers'.

How do I modify the parameter of the placeholder {Invite.Expiration}, for Meta-Reviewer invitations?

The time span is adjusted in the Settings tab>Features>Meta-Reviewer. Enter the amount of days in the field next to 'Meta-reviewer invite expiration time span (in days).' Save the changes. This is not retroactive to invitations already sent.

Can the Meta-Reviews be hidden from the Authors?

Chairs can configure Meta-Review questions as not "Visible to author during author feedback".

How do I get the Review values to show in the Meta-Reviewer Console?

Presuming the papers with the weighted review answers (values) are checked as 'Visible to Meta-Reviewer' and they are assigned to the Meta-Reviewer, the ‘Show aggregate columns’ box needs to be checked in Settings>Features>Review page.

Why do we get a permissions error adding a Note to a paper as a Meta-Reviewer?

A Meta-reviewer can only add/edit a Note when 'Meta-Review Submission' activity is enabled and the deadline has not passed. There is no such restriction for a Senior Meta-Reviewer.

Is there a way Meta-Reviewers can invite Reviewers?

Currently Meta-Reviewers can only invite Reviewers for a particular submission. They can't invite Reviewers for later assignments (like chairs can).

What is the difference between 'Allow Meta-Reviewer to add reviewer' and 'Allow inviting non-reviewer user to review submission?'

• When the Chair enables 'Allow Meta-Reviewer to add reviewer,' the Meta-Reviewer can then add a Reviewer from the 'Manage Reviewers' page. This gives the Meta-Reviewer the ability to add either an existing CMT user or a new CMT user as a Reviewer to the Conference.
  An email will not automatically be sent to the Reviewer.
• The 'Allow inviting non-reviewer user to review submission' setting only allows for non-Reviewers to review the paper for which they were invited.
  An email will be sent to the potential Reviewer.

I deleted a Meta-Reviewer from our conference. Will they still receive conference emails?

Once the Meta-Reviewer is deleted from the conference, they will not receive any meta-reviewer emails or meta-reviewer notifications from the conference. If that user is also an Author with a submitted paper to the same conference, then that user will continue to receive emails pertaining to Authors.

How do I change the expiration of the Meta-Reviewer invitation?

In the Meta-Reviewer settings tab you can change the duration of the invitation, but this has to be saved before you send the invites. Changing the invitation expiration date does not affect invitations that have already been sent.
The expiration will be what the initial invitation had.

Can I change a meta-reviewer question from 'option' to 'option with value' without losing any meta-reviews?

The meta-reviews will not be affected as long as the question is not deleted.

Why does the Meta-Reviewer only see a few assigned papers, when the others that are not visible have no conflicts at all?

If the Meta-Reviewer sees some papers and conflicts are not an issue, look at the paper status of the missing ones to see if they are withdrawn, desk-rejected or rejected.

How do I stop Meta-Reviewers from changing their Meta-Reviews?

If the Meta-Reviewer gave a response, yet more Meta-Reviewing is to be done, enable 'Lock for edit' in the Meta-Review questions form. This will lock the question so it cannot be edited by the Meta-Reviewer.

Why can't Meta-Reviewers see Reviewer emails in their history?

The Meta-Reviewer setting 'Show reviewer identity to meta-reviewer' needs to be enabled.

Why can't Meta-Reviewers see Reviews in Discussion?

Here is a quick gude to settings for the Chairs to follow.

Why can a Meta-Reviewer add a Reviewer to a few papers but cannot assign more to that Reviewer?

Have the Chair check the Reviewer quotas. It is likely the Reviewer assignments met the quota.
It can also be domain conflicts.

How can I bulk delete Meta-Reviewer assignments?

From Chair Console, click Actions>Bulk Delete Assignment>Meta-Reviewer.

How do I get the Meta-Reviewer consoles to show the Min-Max-Avg-Spread values for the Reviewer Questions?

In the Reviewer Settings, enable 'Show aggregate columns on consoles.'

I enabled Meta-Review Submission with a proper date but there is no link for them to enter their reviews.

Make sure there are Meta-Review questions for Meta-Reviewers to answer otherwise there will be no 'Enter Review' link. Chairs can follow this Cheat Sheet.

How do I save a Meta-Reviewer Invite template?

Invitation emails cannot be saved as template like other emails.

Why are Reviewer ratings shown on the 'Reviews' page and not on the 'All Reviewing Details' page?

The Review rating entered by Meta-Reviewers should not be visible to Reviewers.

Why does a Meta-Reviewer get a 'Conflict error 409' when adding a Reviewer?

If a reviewer was added by meta-reviewer, then he/she cannot be added by other meta-reviewers.