Why investing in a good rubric is important

Have you tried using ChatGPT to provide feedback to your students? Maybe you are one of the innovative bunch and tried using it to grade.

If so, I bet you can relate to some of the words in this list: inconsistency, irrelevant, inaccurate, random, hallucinates, etc. A general rule of thumb with AI is that the more guard rails you give, the more precise it will be. This includes assignment outlines, instructions, and, most importantly, rubrics.

Without a rubric, there are so many things AI can provide feedback on: grammar, content, spelling, and other things that might be irrelevant to the learning objective. Garbage-in, garbage-out.

A good rubric can change that and here’s why.

1. Ensure Relevance: A rubric shackles the AI to focus its output on a specific set of criteria, so you know the feedback is relevant to your assessment objectives.

2. Consistency and Fairness: Asking the same question twice can yield significantly different results. If each rating on your rubric has clear descriptors, it significantly decreases the randomness of the output.

How do you create a good rubric? Here’s a couple of resources to get you started:

1) Using rubrics from Cornell University: https://teaching.cornell.edu/teaching-resources/assessment-evaluation/using-rubrics

2) Grading Fairly and Efficiently with Rubrics: https://blogs.illinois.edu/view/1499/183698

A few extra tips from me:

1. Each criterion should be precise and focused. A rubric with ‘Grammar’, ‘Vocabulary’, and ‘Sentence Structure’ is better than a rubric with just ‘Writing’.

2. Every rating under each criterion should be properly defined. Good means something differently to you, me, and your neighbour. Describe what it means to be ‘Good’, ‘Fair’, or ‘Average’. Check below for an idea.

I promise you a good rubric will improve the usefulness of your output!

Chris Du

CEO, TimelyGrader.ai