Order of Review Assignments – New Feature?

I came across an interesting article by Maarten Boudry outlining his Seven Laws of Pessimism. In it, Boudry explains the various forces that lead people to believe a situation is getting worse when it is actually getting better.

One of his seven laws, the Law of Conservation of Outrage, proposes that the total amount of outrage remains constant despite the amount of progress. One passage in particular stood out for me:

When I review a stack of student essays, the ones that were submitted last (which tend to be rushed and therefore poorly written) are on top of the pile. So, the quality of the essays improves as I read through the pile. Without even realizing it, I start raising my bar as I mark, and I have to recalibrate my grades at the end, otherwise I grade the better essays too harshly (or the early ones too leniently).

This made me wonder if ProposalSpace should allow call organizers and/or reviewers to determine the order in which assignments are displayed. Currently, they are always displayed in the order in which the proposals were started. So it’s possible—if not likely—that the quality declines as a reviewer works through their assignments since the later submissions are ostensibly rushed. (This is actually the opposite of Boudry’s situation, where the quality improves, but the underlying effect should be the same. Namely, that reviewers unknowingly adjust the bar as the quality changes.)

I would love to hear feedback from call organizers regarding whether this would be a useful feature and if so, how it should be implemented. I can imagine a few possibilities:

  • Change the default behavior for the entire system so that assignments are always displayed randomly.
  • Add a checkbox in the call settings to allow admins to display assignments randomly.
  • Add a set of options in the call settings to allow admins to select whether to display assignments randomly, by when they were created, or by when they were submitted.
  • Add an option for each reviewer to control how their own assignments are ordered—randomly, by creation date, or by submission date.

Good idea? Bad idea? Worth a try? Let me know what you think!

Bulk File Downloads Are Here!

File-upload fields are one of the most convenient ways to collect information from authors. And in some cases, like for photos, there really is no better option.

But one downside to using file-upload fields has been that in order to collect the files, you previously had to pull up each submission and download the files individually.

Not any more! Now you can download files all at once using the Data-export tool.

Continue reading Bulk File Downloads Are Here!

New Feature: Upload Restrictions

Call admins now have the ability to restrict which types of files are allowed for uploads.

The new setting is in two places: the form builder and the Publishing Module. In both cases it is a safelist (as opposed to a blocklist) meaning authors are only allowed to upload files with extensions you specify. Or put another way, authors are prohibited from uploading files not in the list.

Continue reading New Feature: Upload Restrictions

PubMod Updates

The Publishing Module has a number of exciting improvements in place:

  • The module now uses the domain mtgsked.com instead of the main proposalspace.com domain.
  • Virtually all of the ProposalSpace-specific branding is gone, making the meeting-specific branding stand out more.
  • The meeting name, location, and dates can now be customized in the module’s settings.
  • Pages are responsive (i.e. they display nicely on mobile devices).

Continue reading PubMod Updates

New Feature: Admin Session Copies

Calls using the Advanced Scheduling Module now have a new option when creating admin sessions:

Previously all admin sessions had to be started from scratch. This posed a bit of a challenge, however, if someone wanted to combine multiple submissions into a single session (for a panel discussion, for example).

Now admins have the option of either starting an admin session from scratch or making a copy of a submission or another admin session. Hopefully this helps to minimize the need to copy and paste most, if not all, of the content into the new session.

Submittable Reviews

One of the Review Module’s most useful features has always been the ability for reviewers to save incomplete or provisional reviews. It’s especially handy for reviewers who want to record their initial assessment of a submission then finalize it later, after they’ve analyzed their other assignments.

Up to now, though, there was no way for reviewers to “lock in” their reviews. Unfortunately this meant call admins and review chairs couldn’t be 100% certain a review was ever final, even after the review deadline had passed.

That’s no longer the case! Reviewers now have the option to submit completed reviews, marking them as final for call admins and review chairs. Continue reading Submittable Reviews

New Feature: Enhanced Activation

Quick update for call admins: Now, when you go to activate your call you have the option to activate optional modules too!

Previously, a call had to be activated first and then each optional module (Advanced Scheduling, Publishing, Custom Submission Receipt, etc.) would have to be activated individually. With this update, you can now choose which, if any, optional modules you want to include with the initial activation.

You can still activate optional modules individually like you could before; This just gives you the added convenience of including them with the initial activation.