Lesson Not Performing as Expected
When a Lesson is reported as not working, Skyllful Studio already knows which step Learners are failing on, so this guide has Content Developers read the step data first, look at the expected control in the Heatmap Viewer, rule out a version change, and only then open the Lesson to fix it.
A Learner reports that a Lesson is not working. Buttons do not respond, a mode does not appear, or the Lesson seems stuck.
The instinct is to open the Lesson and look at it. Do the opposite first. Skyllful Studio has been collecting data on every attempt at every step, so it can usually tell you which step is failing and how often before you open anything.
What do you need from the reporter?
Get this before you start, because it decides where you look.
- Which Lesson, and which screen, step or mode
- What the Learner expected, and what happened instead
- Every time, or only sometimes
- The device and browser
- A screenshot or screen recording, if there is one
An intermittent fault on one device is a different investigation from a step that fails for everyone, and the step data below will tell you which one you have.
Which step is actually failing?
Open Lessons Dashboard from Quick Links in Skyllful Studio. It reports Incorrect Actions: how often Learners did something other than what the step expected.
Step |
Action |
|---|---|
| 1 | Find the Lesson in the grid. Use Search, or Filter By to narrow by Course, Module, Job Role or Learner. |
| 2 | Read across the step columns. Each cell shows the incorrect-action rate for that step, colored green, yellow or red. |
| 3 | Note which step is red, and check Total Attempts so you know whether the rate is built on real volume. |
Note: A high rate on one step and green everywhere else points at that step's configuration. High rates across many steps point at something broader, such as the wrong mode being enabled or a change to the whole Lesson.
Note: A rate means little on a handful of attempts. Read Total Incorrect against Total Attempts before drawing a conclusion, and be careful with a newly published version, which starts at zero attempts and zero incorrect actions on every step. That looks like a clean Lesson and is actually an empty one.
For what each control and color means, see What the Lessons Dashboard Measures, Read the Step-by-Step Grid and Filter and Focus the Step-by-Step Grid.
What is the step expecting the Learner to do?
Once you know the step, select the See Heatmap link in its cell. The Heatmap Viewer opens in a new tab, headed with the Lesson and step name.
It shows you three things worth having before you touch the Lesson:
- The Step Description, which is the instruction the Learner was given
- The screen the Learner saw, as its Screen Replica
- The control the step expects, outlined in bright green
That green outline is the interaction zone: the part of the screen the step accepts as correct. It comes from Selected Controls on the step's General tab. Anything a Learner does outside it is recorded as an incorrect action, which is what the dashboard is counting.
If |
Then |
|---|---|
| The green outline is on the wrong control | The step is pointing at the wrong thing. Fix Selected Controls on the step's General tab. |
| The green outline sits where the Learner would not look, or is too small to hit on a phone | The step is technically correct and practically wrong. Consider a different control, or a clearer Step Description. |
| The Step Description does not match what the green outline expects | The Learner is being told to do one thing and measured on another. Fix whichever is wrong. |
| The outline is on the right control and the description matches | The step is configured correctly. Check whether a version change caused this. |
See Use the Heatmap Viewer for the Timespan and mode controls on that screen.
Did a new version cause it?
Select the Lesson name in the Lessons Dashboard to open the Lesson Versions Dashboard. It lists one row per published version, newest first, with the same step columns and colors.
This answers a question nothing else will: whether the step was always failing, or started failing when a version shipped. If an earlier version was green on that step and the current one is red, the change is in what shipped, and the Changelog in Lesson Builder will tell you what changed.
See Compare Lesson Versions.
Are the right modes enabled?
A mode that is switched off does not appear to the Learner, which reads as a Lesson that will not progress.
If |
Then |
|---|---|
| A mode that should be available is excluded | Include it, check the Lesson in the Previewer, publish a new version, and tell the reporter. |
| A mode is included that should not be | Exclude it, check in the Previewer, publish, and tell the reporter. |
| The modes are right | Move on to reproducing it. |
Note: Exclusion is per step as well as per Lesson. A step excluded from a mode shows a crossed out eye on that mode's tab, so a Lesson that works for most steps and stalls on one may have a single step excluded rather than a broken mode.
Can you reproduce it?
Use the Previewer in Skyllful Studio to walk the Lesson in the mode the Learner was in.
If |
Then |
|---|---|
| You can reproduce it | You have what you need. Note the step, the action and the result. |
| You cannot reproduce it, and the step data is green | Treat it as device or browser specific. Get the Learner's device and browser, and whether it happens on another device. |
| You cannot reproduce it, but the step data is red | The Learners are hitting something you are not. The Heatmap Viewer is the better evidence here than your own attempt. |
| It happens in the published Lesson but not in the Previewer | Say so when you escalate. That distinction narrows the investigation considerably. |
What to send when you escalate
Raise a support ticket with Skyllful and include the evidence you gathered rather than a description of the symptom.
- The Lesson name, the version, and the step
- The incorrect-action rate on that step and the total attempts behind it
- Whether earlier versions were failing on the same step
- What the Heatmap Viewer showed: the expected control, and whether the Step Description matched it
- Whether you could reproduce it in the Previewer, and in which mode
- The Learner's device and browser
- Screenshots or a recording, if you have them
If you have already raised a ticket and it is unresolved, reply on that ticket rather than opening a second one, so the history stays together.
Skyllful Knowledge Base | Content Developers