Monitoring & Moderation

While the evaluation is open

The evaluation view shows a live rollup: current per-student averages as scores come in, plus coverage: who has enough evaluators and who does not.

• The evaluation matrix (Insights plan, on the Network & matrix tab) shows, evaluator by target, what has been submitted. Click a check mark to see that evaluation’s scores and notes.
• A close warning lists students who are still incomplete if you close now.

Moderating notes

Student notes can be auto-flagged (aggressive or profane language, or notes too short to be meaningful). Flagged notes are held out of view until you act.

When note approval is on, close the evaluation and choose Start moderation. The Notes tab then lists the written notes, with a badge counting the flagged ones.

• Filter by Flagged, Waiting (not flagged, still to approve), Approved or Redacted. Each note shows who it is about and who wrote it.
• Approve or Redact each note. A redacted note is withheld from the student even after release. The text is kept, only its status changes.
• Approve N unflagged approves every waiting note that was not flagged in one step.
• Review notes in the status strip opens this tab.
• Only approved notes ever appear in a student’s released results. Private notes are for you only and need no review.

Unusual scoring patterns

The app flags scoring that looks off. Some flags are about how someone rated their peers:

• Same Score Repeated: gave everyone the same score on every criterion, when other evaluators saw those peers differently.
• Submitted Very Fast: only a few seconds of active time on the evaluation.
• Identical Scores: every criterion on one evaluation got the same number.
• Unusually Generous / Unusually Harsh: scored peers far above or below the rest of the class.
• Scores Don’t Match the Class: rated many peers consistently higher (or lower) than everyone else did.
• Score-Trading Pattern: the scores someone gave closely mirror the scores they got back from the same peers.

Others are about a pair or group:

• Mutual Low Score: two students rated each other near the bottom (often a personal conflict rather than a performance signal).
• Mutual High Score: two students gave each other unusually high scores.
• Possible Score-Swapping Group / Small Group Pattern: three or more students who all (or mostly) rated each other unusually high.

And some describe the student being rated:

• Rated Low / High Overall: self-rating and peer average both near the class extremes.
• Overconfident / Underconfident: rated themselves well above (or below) what peers gave them, by more than most classmates.
• Peers Disagreed: this student’s evaluators disagreed with each other more than usual.

These are heuristics, not verdicts. On the Results tab, students with a flag appear under Needs a look (Simple view) and in the Flags column (Detailed view, where the Flagged filter shows only them). Click a flag to open the scores behind it for you to judge. A student needs at least three evaluators before the “student being rated” flags apply.

Excluding a rating

On the Results tab, open a student (their row in the Detailed view, or See each rating in the Simple view’s side panel) to see every rating they received and gave. There you can exclude a specific evaluator’s rating of a specific target, with a reason. Excluded ratings are dropped from the aggregation before the report is computed.

• The original submission is never altered or deleted; exclusion is a separate override flag.
• You can include it again to undo.
• Do this during the reviewing stage, before you release.

After you release

Released results are what students see. If you spot a problem afterward you can still exclude ratings and the report recomputes, but students may have already seen the earlier numbers, so prefer to get this right before release.

To keep the rater anonymous when something changes after release:

• Excluding a rating also removes that rater’s public note. A student who compares before and after may be able to match that note to the scores that left their averages, but not to a person.
• If Show evaluator names is on, the “Who evaluated you” list is turned off for good for every student in that evaluation the first time you exclude or include a rating, or reopen it, after release. It stays off after you release again. Otherwise a student could compare the list before and after and see exactly who was removed or added.