HOW-TO: Reviewer Compares Papers
OVERVIEW
When enabled by the Chair, the Reviewer will have access to a page that will allow comparisons of papers to be made - which paper is better than another, or which paper is equal to another. It uses Paper IDs to distinguish the papers.
Note
The Chair must also enable the Review Submission Activity and set the deadline to a date in the future in order for Reviewers to be able to compare papers.
REVIEWER CONSOLE
- From the Reviewer Console click 'Actions>Paper Comparison.' The 'Paper Comparison' page appears.
PAPER COMPARISON PAGE
Here you can add any number of non-contradicting comparisons between papers. For instance,
- A > B You prefer paper A over B, i.e. given a choice, you would accept paper A instead of B.
- A ~ B You consider paper A and B to be comparable and about the same.
- Click 'Add Comparison +'
The action row appears.
In this case we have 6 papers. Lets' say Paper ID 12 is about the same as Paper ID 18.
- Click the 'Left paper' dropdown. Pick Paper ID 12.
- Click the 'Operatore' dropdown, select the tilde
- Click the 'Right paper' dropdown, select Paper ID 18
- Click 'Save'
It shows the comparison equation that Paper ID 12 is approximately equal to Paper ID 18.
If a comparison is deemed invalid then an alert 'Comparison violation detected' will appear at the bottom of the page.
- For example we add a comparison that paper 18 is better than 12. Click 'Save'. The alert appears.
Now input the following by clicking 'Add comparison +' and saving after each equation:
14 ~ 18
15 > 12
16 > 18
17 ~ 14Click 'Get Summary'
It shows 2 rows. What can be derived from the summary is that Papers 16 and 15 are better papers than 12, 14, 17, 18.
Note
Papers from different tracks cannot be compared.
If a Reviewer (assigned papers in different tracks of a multi-track conference with the comparison enabled in both tracks) tries to compare papers from different tracks, an alert 'Bad Request: Left and Right papers should be from the same track' will appear at the bottom of the page.
Related:
Chair enables Submission Comparison