Introducing Cognitive Apprenticeships

Realizing our true value proposition

One of the biggest things we learned from user feedback in Fall and in early Spring was that the real value TimelyGrader was providing was not simply

1. Helping instructors grade faster by saving them time

2. Giving students instant feedback

We didn’t want to just promote ourselves as an ‘AI grading tool that saves you time!’ either.

Instead, we found that TimelyGrader was actually enabling

1. The ability for instructors to see student ‘process’ over just ‘product’

2. Allowing students to learn from productive struggle

3. Students are the vital ‘human-in-the-loop’

That’s why we are….

Introducing Cognitive Apprenticeships: a way to support the productive struggle and combat cognitive offloading

Student Iterative Feedback

This is our first expansion of the student instant feedback feature. Prior to this update, feedback provided to students was siloed, meaning feedback could be different from draft to draft. They are now connected and each draft's feedback will build on the previous one. TG will be able to tell how well a student incorporates previous feedback (or not) and get them to build on top of what they did.

‘Inquire’ AI

A common ask from our student users was the ability to clarify feedback further. This is now possible. Students can now engage with our ‘Inquire’ AI to ask clarifying questions or brainstorm ideas. We will create sample questions for students to select from in case they don’t know what to ask, but they always have the option to ask their own questions.

A couple of things we are actively working on:

• Ability to see 'before' and 'after' views, allowing students to compare their original draft with suggested improvements

• Location data for each inquiry will also be shown, which is already available for instructors when grading

Instructor Insights

Of course, instructors will also be given a whole slew of new formative learning insights. These are insights that were impossible to obtain at scale. Some of these new data insights include:

1. How did students incorporate feedback from one draft to another?

   

   Seeing feedback adapt over each draft

2. How did the quality of each draft improve over time?

   

How we are different

We’ve already been asked this a couple of times: why can’t the students just use a generic LLM or chatbot? A common answer might be ‘Socratic Thinking’, but that’s a simple addition to the prompt that anyone can do.

Here is our response: we have built TimelyGrader that contains incentives for instructors to fine-tune the AI because they will be the ones using TimelyGrader for grading. That means they will be creating this curated environment that contains all their grading nuances, their expectations, specific definition of terminology, and materials - not just writing or grammar.

Another benefit is trust. A theme from ASU’s independent research of TimelyGrader was that:

While instant student feedback and ‘Inquire’ AI will be delivered directly to students, students will find comfort in that it is rigorously trained by their instructors instead of a generic AI tool that does not have any training or makes instructors do the heavy lifting that creates more work for them.

Students won’t be getting a generic chatbot experience but one that is tailored for the course, for the assignment, and for each individual.

The instructor is the subject matter expert, and the fine-tuned AI is the medium through which the instructor's evaluative thinking becomes visible at moments when the instructor can't be present.

Beta Results

As expected, with better feedback, we saw better academic performance and also reduced anxiety for students.

When is it coming?

Sooner than you think! We will be deploying the ‘Cognitive Apprenticeship’ update on June 1st at 4PM PST, 7PM EST.

This will be made available to all institutional partners with the LMS integration. It will be backward compatible and will be turned off by default.