Skip to content
  • There are no suggestions because the search field is empty.

Setting Up Round-Based Evaluations (Sessions 2.0)

Create evaluation plans with multiple review rounds, custom scorecards, and flexible evaluator assignments to manage your review process from start to finish

In this article:

How evaluations work in Sessions 2.0

Round-based evaluations replace the single-stage evaluation model from the previous version. Instead of one review pass, you can build a multi-stage process that mirrors how peer review actually works, for example: Initial Screen → Peer Review → Committee Decision.

Each round has its own timeline, scorecard, anonymization settings, and evaluator pool. This means your triage committee can use a simple pass/fail scorecard in Round 1, while your expert reviewers use a detailed rubric with multiple criteria in Round 2.

Key changes from the previous evaluation model:

  • Multiple rounds per plan: Chain as many review stages as your program requires.
  • Scorecards replace rigid rubrics: Each round has its own scorecard with flexible question types (ratings, text, dropdowns, file uploads).
  • Reviewer View Configuration: Control which submission fields evaluators can see, filter by, and view on submission cards.
  • Flexible assignment logic: Choose from four distribution modes: All to All, Per Submission, Per Reviewer, or Individual Reviewer.
  • Real-time admin visibility: See all evaluator progress across all rounds from one dashboard.

The evaluation plan wizard

When you create or edit an evaluation plan, Sessionboard opens a 4-step wizard with a sidebar navigation:

  1. Overview, Name and basics (plan name, instructions, review period, and global toggles).
  2. Rounds, Review rounds (configure round names, types, voting, and scorecards).
  3. Evaluators, Select users who can evaluate (choose from existing event users).
  4. Assignments, Who evaluates what (filter sessions and set assignment logic).

You can navigate between steps using the sidebar or the Continue → and ← Back buttons at the bottom. Your progress is saved automatically as you work.

To create a new plan, navigate to Program → Evaluations  in the left-hand sidebar and click Evaluation Plans → + Add Plan. The wizard opens to the Overview step.

Step 1: Overview

The Overview step (labeled "Name and basics" in the sidebar) is where you configure the plan's core settings.

Name and Instructions

  • Name: Give the plan a descriptive name (e.g., "My Evaluation Plan").
  • Set Plan as Open: Toggle this on to open the evaluation plan and begin letting your evaluators review submissions. Leave it off while you are still configuring.
  • Instructions: A rich-text editor for guidance that applies to your default round (Round 1). Use formatting, links, and lists to clearly describe the evaluation criteria and expectations.

Review Period

Set the window during which evaluators can complete reviews for the default round:

  • Opens: The date evaluators can begin submitting reviews.
  • Closes: The deadline after which evaluators can no longer submit or modify their reviews.

Additional settings

  • Enable Anonymized Review: Same as blind review in round settings for your default round (Round 1). Hides submitter and participant identity from evaluators.
  • Enable Weekly Reminders: Applies to your default round (Round 1). Evaluators get reminder emails for sessions pending in this round.
  • Include Uploaded Files: Applies to your default round (Round 1). Lets evaluators download files attached to sessions in this plan.

Tip: Keep Set Plan as Open toggled off until you have finished configuring all four steps. Once you open the plan, evaluators with assignments can begin reviewing immediately.

Step 2: Rounds

The Rounds step (labeled "Review rounds" in the sidebar) is where you configure review rounds (similar to award program rounds). A default Round 1 is created automatically. Click + Add round to add additional stages.

Basic Information

  • Round Name: A descriptive label (e.g., "Round 1," "Peer Review," "Final Selection").
  • Round Type: Select the round type (e.g., "Review Only"). This setting cannot be changed after creation.

Voting

  • Reviewer Type: Define who can vote in this round (e.g., "Reviewers," where only selected evaluators can vote).
  • Voting Type: Choose the voting method (e.g., "Score Voting").

Note: Round Type cannot be changed once the round is created. Make sure you select the correct type before proceeding.

Tip: You can add as many rounds as you need. A typical 3-stage process might look like: 

  • Round 1: Triage (quick pass/fail by committee chairs)

  • Round 2: Peer Review (detailed scoring by subject-matter experts)

  • Round 3: Final Selection (committee vote on top-scored submissions)

Configuring scorecards

Each round has its own Scorecard, the set of questions your evaluators answer when reviewing a submission. The scorecard is configured within the round settings.

Click + Add Question to choose from:

Scoring Questions

  • 1-3 Scale: Rate on a 1 to 3 scale.
  • 1-5 Scale: Rate on a 1 to 5 scale.
  • 1-10 Scale: Rate on a 1 to 10 scale.
  • Numeric Score: Custom min/max input for non-standard scoring ranges.
  • Custom Dropdown: Custom options with point values (e.g., "Accept = 3, Revise = 2, Reject = 1").

Input Fields

  • Free Text: Open comments and written feedback.
  • File Upload: Document upload (e.g., marked-up manuscripts or review forms).

Layout

  • Separator: Section divider to organize longer scorecards visually.

Note: Each round can have a completely different scorecard. Your triage round might use a single 1-3 rating question, while your full peer review round might include three rating scales, a free-text justification field, and a recommendation dropdown.

Reviewer View Configuration

The Reviewer View Configuration section controls what evaluators see when reviewing submissions. It has three sub-sections:

Visible Fields to Reviewers

Choose which submission fields evaluators can see during their review. All fields are visible by default. Uncheck any field to hide it from reviewers.

Each field shows its type (e.g., Text, Wysiwyg, Dropdown). Common fields include ID, Type, Title, Description, Format, and Tags.

Filterable Fields for Reviewers

Choose which fields reviewers can filter submissions by. Only checkbox, dropdown (single or multi-select), and multi-select fields are eligible.

Category is always included and cannot be removed. You can optionally enable filtering by Format, Tags, Level, Language, Status, Submitter, Track, Location, and other dropdown-type fields.

Submission Card Fields

Customize what information appears on the submission card in the evaluator's dashboard. Title is always the card heading and cannot be removed. You can add up to 3 optional fields below the title, displayed in the order you set.

See Add a field to choose from available fields like ID, Type, Description, Format, Tags, Level, Language, Status, and Submitter.

Tip: Use Submission Card Fields to help evaluators quickly identify submissions without opening each one. Adding Format and Track to the card lets reviewers see at a glance what type of session they are about to evaluate.

Step 3: Evaluators

The Evaluators step (labeled "Select users who can evaluate" in the sidebar) is where you choose who will review submissions for this plan.

Evaluators must already be users on this event. To add an evaluator:

  1. Use the Search by name or email... bar to find the user.
  2. Click on their name to select them.

At the top of the page, the Rounds section shows which rounds new evaluators are enrolled in. New evaluators are enrolled in the rounds you select here. Check or uncheck rounds to control which review stages each evaluator participates in.

Note: Evaluators must already exist as users on this event. If you need to add someone who is not yet a user, add them to the event first, then return to this step to select them as an evaluator.

Step 4: Assignments

The Assignments step (labeled "Who evaluates what" in the sidebar) is where you define which sessions each evaluator reviews. Choose which sessions to assign and how evaluators are matched, preview the impact, then confirm when you are ready.

Assignment Rules

Start by creating an assignment rule. You can create a new rule or use an existing one. Each rule has a round selector (e.g., "Round 1") so you can configure assignments per round.

Filter Sessions

Choose which sessions to include in this assignment rule:

  • All sessions in plan scope: Uses this plan's custom filters and limits to include all matching sessions.
  • By track: Narrow to one or more tracks or track prefixes.
  • Individual sessions: Pick specific sessions by title search.

Assignment Logic

Choose how filtered sessions are distributed to evaluators:

  • All to All: Every reviewer receives all filtered submissions.
  • Per Submission: Each submission gets a set number of reviewers (e.g., "Reviewers per submission: 3"). Use this to ensure every abstract gets a consistent number of reviews.
  • Per Reviewer: Each reviewer gets a maximum number of submissions (e.g., max [Y] submissions per reviewer). Use this to cap evaluator workload.
  • Individual Reviewer: Assign filtered sessions to one or more specific reviewers you select.

Preview Assignments (mandatory)

After configuring your filter and assignment logic, you must complete the Preview Assignments flow before your assignments will be saved. Click the Preview Assignments button to begin.

The preview flow has three steps:

Step 1: Configuration

This is the filter and assignment logic you already set up. If you need to make changes, you can return here from any later step.

Step 2: Preview table

A table displays every assignment that will be created, with columns for Session and Reviewer. On the right, an Impact Preview panel summarizes:

  • Filtered sessions: the number of sessions matching your filter.
  • Evaluators in this rule: how many evaluators are included.
  • New Assignments: the total number of assignments that will be created.

Use ← Back to configuration to adjust your settings, or Continue to confirm → to proceed.

Step 3: Ready to Assign

A confirmation screen displays how many assignments will be created (e.g., "15 assignments will be created"). It also explains that any existing assignments for the selected evaluators will be preserved, and only new assignments will be added.

Click Save assignments to finalize, or ← Back to preview to review the table again.

Note: If you skip the Preview Assignments flow, your assignments will not be saved. You must complete all three steps and click Save assignments for the distribution to take effect.

Managing your evaluation plans

From the Evaluation Plans list, you can perform the following actions on any plan:

  • Review: If you're an admin & a reviewer, you'll be able to review.
  • Edit: Re-open the plan wizard to modify settings, rounds, evaluators, and assignments.
  • Open / Close: Control whether evaluators can currently access and submit reviews.
  • Export Results: Download evaluation data in two formats: Individual Grades Report (every evaluator's responses per submission) or Cumulative Grades Report (aggregated scores per submission).
  • Duplicate: Create a copy of the plan with the same configuration, useful for running similar evaluations across multiple tracks or events.
  • Notify: Notify evaluators.
  • Delete: Permanently remove the plan and all associated data.

Best practices

  • Start simple, add complexity: If this is your first round-based evaluation, start with a single round. You can always add more rounds in future programs.
  • Use anonymization strategically: Enable anonymized review for peer evaluation rounds to reduce bias, but consider disabling it for committee decision rounds where context matters.
  • Set clear instructions: Use the rich-text Instructions field on the Overview step to clearly describe evaluation criteria, scoring guidelines, and expectations.
  • Customize the reviewer view: Hide irrelevant submission fields and configure card fields so evaluators can quickly scan and prioritize their assigned sessions.
  • Enable weekly reminders: Turn on weekly reminder emails to keep evaluators on track without manual follow-up.
  • Preview assignments before saving: Use the Impact Preview panel and the Preview Assignments button to verify session counts and evaluator workload before anything goes live.
  • Stagger round timelines: Leave a buffer between rounds for results review and submission filtering. This gives your committee time to decide which submissions advance.
  • Export early and often: Download interim reports to track trends, even before all evaluations are complete. This helps you spot issues (e.g., an evaluator who has not started) before the deadline.

Next steps

With your evaluation plan configured, continue setting up your review process: