One of the Review Module’s most useful features has always been the ability for reviewers to save incomplete or provisional reviews. It’s especially handy for reviewers who want to record their initial assessment of a submission then finalize it later, after they’ve analyzed their other assignments.

Up to now, though, there was no way for reviewers to “lock in” their reviews. Unfortunately this meant call admins and review chairs couldn’t be 100% certain a review was ever final, even after the review deadline had passed.

That’s no longer the case! Reviewers now have the option to submit completed reviews, marking them as final for call admins and review chairs. Reviewers are still able to save their work and return to make changes later, but only until they submit a review. Once that happens the review is locked, allowing the reviewer to view it but not to make further changes to it (unless a call admin unlocks it).

On the admin side, values for reviewer progress and average score are no longer calculated based on all reviews. Instead, the default is to calculate those values using only submitted reviews. The Tracker does, however, have a handy toggle that allows admins and chairs to include unsubmitted reviews. (See screenshots to the right showing information for the same two submissions. Each submission has been assigned to two reviewers. The reviewers have completed their reviews but have not submitted them. The top image shows the default view, which is based solely on submitted reviews. The bottom image shows the same view but with unsubmitted reviews included.)

Other new features for admins and chairs include the ability to submit reviews on behalf of reviewers and to unlock/return submitted reviews to make them editable. The data-export feature also includes the review status of each review in the output file.

Reviewers see changes in two places: the main review form and the page that displays their review assignments.

On the main review form, reviewers no longer have to click a button to save their work. Changes are instead saved automatically. Because of that, the Save button is no longer needed and has been replaced with a Submit button, which is inactive until the review is eligible for submission (i.e. the reviewer has answered all of the required questions on the review form).

Over on the review-assignments page, reviewers now see two new columns in the assignments table: Review Status and Actions. (See the screenshot to the right.)

The Review Status column displays one of the following four values for each review assignment:

• Not started = None of the review questions have been answered.
• Incomplete = Some review questions have been answered but at least one required question is missing an answer.
• Complete = All required review questions have been answered. The review is eligible for submission.
• Submitted = The review has been submitted and may not be changed.

(BTW, these are the same values used in the data-export file.)

The Actions column always displays a button to edit the review (or to view it if it’s been submitted). If the review is complete but has not yet been submitted, the column also displays a button to submit the review.

I really hope these changes improve the user experience for call admins and review chairs by making it possible for them to know for certain when reviewers are done with their work while not taking away any of the features and functionality already enjoyed by reviewers. As always, if you have any feedback about the changes or want to pass along ideas for other improvements, please let me know!