See it in action FR
Menu

Episode 6 · 6:47

AI score and verdict

Feedback is good, but a score Storyline can act on is better. Make the Q&A grade each answer against the store’s returns policy, and let Storyline show a reminder that quotes the broken rule.

YouTube loads only when you play the video. Watch on YouTube

Chapters

What you’ll learn

  • Have the AI grade answers with a score and a pass or fail
  • Grade answers against your own documents, such as a company policy
  • React in Storyline when the learner fails

Before you start

  • Episode 5 done: the “Customer apology” Q&A as a SlideMind widget, in a Storyline Web Object
  • The learner memory from episode 4: Sam, as test-learner-1
  • Articulate Storyline 360, with the project from episode 5 open

Follow along

Step by step

A score and a verdict

  1. Open “Customer apology”. Until now, it ran on Feedback only: a few sentences the learner reads, but just text for Storyline. With Score and verdict, the AI fills in a short form instead. Each line is a return variable: the widget shows it on the card, and Storyline receives it in its own variable, ready for your triggers.
  2. In Results & memory, under What comes back?, switch from Feedback only to Score and verdict. The feedback field is already there.
  3. Select Add return variable: name it score, choose Number and describe it as “Score from 0 to 10”. Add one more: passed, Boolean, described as “True if the score is 7 or more”.
  4. At the end of the Instructions, add one sentence about the score.
  5. In Look, open More styling. Leave Score display on Fraction (8 / 10), pick score for Score, type 10 in Score out of and pick passed for Pass / fail.
  6. Send an answer in Try it: the card flips with the feedback, a verdict and a score, like “Good answer” and 8 / 10. Select Save slide.

The store’s policy

  1. Reference documents give the AI a policy, a procedure or a slide deck, and it uses that text when it answers. To follow along, download the one-page returns policy (PDF) used in the video.
  2. On the “Customer Service Essentials” course page, under Reference documents, select Add a document and choose the file. It’s read right in your browser: SlideMind keeps only the text, never the file.
  3. Under What the AI will read, check that the text came through, then select Add document. Added to the course, it’s read by every slide of the course, unless a slide turns it off. A short document like this one is read in full with each answer; with longer ones, the AI reads only the passages that match the question.
  4. Learners can get anything in these documents by asking the AI, so leave out anything confidential.
  5. Back in “Customer apology”, the Course context sheet lists the policy under From the course, with Use in this slide on. Add one more sentence to the Instructions.
  6. In What comes back?, add one more return variable, so Storyline can tell the learner exactly what went wrong: name it policy_check, choose Text and paste its description.
  7. In Try it, answer the way a rushed agent might: “Sorry, Maya. Please send the broken lamp back with a photo, and we’ll ship a new one.” The card shows “Not quite”, 3 / 10. The policy check doesn’t show on the card: it goes to Storyline. Select Save slide.

The verdict in Storyline

  1. In the slide’s Installation step, open Storyline variables. You need two names: the one ending in _verdict, which holds pass or fail, and the one ending in _policy_check.
  2. Storyline only keeps a value in a variable that already exists. Copy each name, and in Storyline, open the Variables window, select Create a new variable, paste the name and choose Text. Twice.
  3. In the Layers panel, select New Layer and name it Policy reminder. On it, add a text box that shows the policy check: type the variable ending in _policy_check between percent signs, like %aiResp_customerApology_policy_check%. The reminder repeats what this learner said and quotes the exact rule it breaks.
  4. Back on the base layer, select Create a new trigger. Action: Show layer, Policy reminder. When: Variable changes, with the verdict variable. Under Conditions, add one: that same variable, equal to fail. Pass, and the learner moves on; fail, and the reminder explains what went wrong.

Test in SCORM preview

  1. As in episode 5, publish the course for an LMS as a SCORM package, zip it and upload it in Test & monitor › SCORM preview with Choose a .zip.
  2. Select Launch, with test-learner-1 as the learner, so the course runs as Sam, then Open in a window.
  3. Give the rushed answer. The card flips: “Not quite”, 3 / 10, with feedback on what to do instead. Since the verdict is fail, the Policy reminder layer appears beside the card, with what Sam asked for and the exact rule it breaks.

Transcript

Show

Scenario 5: Score the learner’s answer

Hi and welcome back! From here on, the videos get a bit more advanced, but they show how far you can take personalized interactions for your learners. In this video, our Q&A learns to grade: it will give a score and a verdict, check each answer against the store's real returns policy, and when the learner misses the mark, Storyline will step in with a reminder. Here's our Q&A again. The learner answers… and the card flips with a score and a verdict. And when the answer breaks the store's policy, a reminder pops up in the course. Let's build it.

A score and a verdict

Let's start with our Q&A, "Customer apology". Until now, it ran on "Feedback only": the AI sends back a few sentences of feedback, and that's it. The learner reads them, but for Storyline it's just text. "Score and verdict" changes that. Instead of free text, the AI fills in a short form: the feedback, plus a score and whether the learner passed. Each line of that form is a return variable. The widget shows them on the card, and Storyline receives each one in its own variable, like the one ending in "score", ready to use in your triggers. Let's set it up. In "What comes back", switch from "Feedback only" to "Score and verdict". The feedback field is already there. Click "Add return variable": we name it score, pick "Number" in the type menu, and describe it as "Score from 0 to 10". One more: passed, we pick "Boolean" and describe it as "True if the score is 7 or more". In the instructions, we add one sentence: "Also give a score out of 10, and set passed to true when the score is 7 or more." Under "Look", "More styling", we leave "Score display" on "Fraction", pick score for "Score", type 10 in "Score out of", and pick passed for "Pass / fail". In "Try it"… the card flips: feedback, a verdict and a score. Let's save. We'll come back to that verdict in Storyline in a moment.

The store’s policy

Next, the store's returns policy. For this, we'll use a feature we haven't seen yet: reference documents. You give the AI a policy, a procedure or a slide deck, and it uses that text when it answers. On the course page, under "Reference documents", click "Add a document" and choose the file: here, our one-page PDF. It's read right in your browser: SlideMind keeps only the text, never the file. Under "What the AI will read", we check that the text came through, then click "Add document". Because we added it to the course, every slide of the course reads it, unless a slide turns it off. A short document like this one is read in full with each answer; with longer ones, the AI reads only the passages that match the question. One thing to keep in mind: learners can get anything in these documents by asking the AI, so leave out anything confidential. Back in "Customer apology", the "Course context" sheet now lists the policy under "From the course", and "Use in this slide" is on. We add one more sentence to the instructions: "Check the learner's promises against the store's returns policy, and fail any answer that breaks it." Then, back in "What comes back", one more return variable, so Storyline can tell the learner exactly what went wrong: we name it policy_check, pick "Text", and describe it as "If the answer breaks the returns policy: what the learner promised or asked, then the rule it breaks, quoted from the policy. Empty otherwise." In "Try it", let's answer the way a rushed agent might: "Sorry, Maya. Please send the broken lamp back with a photo, and we'll ship a new one." "Not quite", 3 out of 10. The policy check doesn't show on the card: it goes to Storyline, with what the learner asks for and the rule it breaks. We'll see it in the course in a moment. Let's save.

The verdict in Storyline

Now, let's use that verdict in Storyline. Back on the slide, in the "Installation" step, we open "Storyline variables". We need two of them: the one ending in "verdict", which holds pass or fail, and the one ending in "policy_check". Storyline only keeps a value in a variable that already exists, so we copy each name, and in Storyline, we open the "Variables" window, click "Create a new variable", paste it and choose "Text". Twice. Next, a layer: in the "Layers" panel, we click "New Layer" and name it Policy reminder. On it, instead of a fixed message, a text box that shows the policy check: we type the variable ending in "policy_check" between percent signs. That way, the reminder repeats what this learner said and quotes the exact rule it breaks. Last, the trigger. Back on the base layer, we click "Create a new trigger". Action: "Show layer", Policy reminder. When: "Variable changes", and we pick our verdict variable. Under "Conditions", we add one: that same variable, "Equal to", fail. Pass, and the learner moves on; fail, and the reminder explains what went wrong.

Test in SCORM preview

Let's test. We publish the course for "SCORM preview" again, and launch it as Sam. Sam gives the rushed answer. The card flips: "Not quite", 3 out of 10, with feedback on what to do instead… and since the verdict is fail, the policy reminder fades in beside the card, with what Sam asks for and the exact rule it breaks. Your Q&A now grades answers, reads your documents, and tells Storyline whether the learner passed. In the next video, it's the chat's turn: Maya's face, a coach and three endings will follow the conversation. See you there!

Keep the idea going.

Bring one Storyline activity.

Start with a question worth answering. Build the feedback around your learning objective, then test it in your own course.