Learning Outcomes Assessment demo
Assessment offices score sampled student work against outcome rubrics and draft reports 1:33
What the demo shows
Assessment offices score sampled student work against outcome rubrics and draft reports
Seven agents run the cycle. The Sampler draws a stratified random sample by campus, modality and dual enrollment and records the seed; the Redaction agent hides names; two Scorer agents on different models score every paper blind, citing the passage behind each level; the Agreement agent sends gaps of more than one level to a faculty panel and measures agreement against a 10% faculty calibration set the agents never see first; the Reporter drafts results and use of results; the Evidence agent builds the accreditor's pack. Faculty own the rubric, the director signs, the VP approves funds.
Transcript
- Learning Outcomes Assessment scores sampled student work against faculty rubrics and drafts reports.
- Accreditors require colleges to show what students learned, and what changed.
- Brandvale Community College: 312 papers from 47 sections, sampled with a recorded seed.
- Each stack is the papers scored there: 38 percent sit at Developing on evidence.
- Click a paper: the passage behind its score opens, name hidden.
- Two Scorer agents score each paper blind; faculty settled their 21 disagreements.
- On a hidden faculty calibration set, agents agree at kappa 0.74.
- North Campus online scores 0.4 lower: the Reporter finds an old prompt.
- Director of Assessment Gwen Talbot signs, with the department's action plan attached.
- Agents never sign, faculty own the rubric, every step is audited.
- The result: a signed report where every number opens its papers.
- Built on Botminds: agents, data, screens and governance on one platform.
The narration as written for the film. Captions are on the player.