Dwaine McLean

A standard courses have to pass before anyone sees them.

Five pass/fail gates, then 43 criteria worth 100 points across nine pillars. Fail any gate and the score is capped, however good the rest of the course is.

Most e-learning quality checks are a reviewer’s opinion written down after the fact. This one runs before launch, produces a number, and can be argued with — which is the point. I built it from four independent expert reviews and eight staff focus groups, and it carries a version history and a six-monthly review, because a standard that never changes stops describing the work.

The gates

Gates are not scored. They pass, fail, or need a human to look. A course that fails one doesn’t get a good score with a caveat attached — it gets capped, because the things in this list aren’t trade-offs.

Five gates, checked before scoring begins
GateFails when
1Accuracy. Every factual claim, figure and link holds up.
2Policy. Nothing in the course contradicts the organisation’s own rules on data, tools or responsible use.
3Alignment. Every stated objective is actually taught, practised and checked somewhere in the course.
4Accessibility. WCAG 2.2 AA, including a screen reader pass on the built course rather than the storyboard.
5Build integrity. No placeholder text, broken embeds or orphaned blocks in the published version.

Gate 5 sounds trivial. It catches more courses than any of the others.

The nine pillars

The 43 criteria sit under these. Points are weighted towards what learners do, not what they are shown.

The most contested decision in the current version was allowing a course to end in a work product or a structured reflection instead of a quiz. A quiz is easy to score and easy to pass without having learned anything. If someone finishes a course holding a plan they will actually use, that is better evidence than four correct answers out of five.

Scoring it with AI, honestly

Scoring 43 criteria by hand is slow enough that it quietly stops happening. So the standard ships with a prompt and five rules for AI scorers:

  1. Quote the evidence for every score. No evidence, no score.
  2. When two scores are defensible, take the lower one.
  3. Score the built course, not the storyboard.
  4. Run two different models independently and compare criterion by criterion.
  5. Where they disagree by more than one point, a person decides.

Point four is the one that matters. A single model agrees with itself; two models disagreeing tells you exactly which criteria are ambiguous, and those are the ones worth rewriting in the standard.

The honest limitation: the scale has been calibrated on a small number of courses so far. Its scoring bands will move as that number grows, and a version history exists so anyone using it can see when and why.

Why publish it

A quality standard written by the person whose courses it scores is only meaningful if the method is open. Publishing it means the number can be challenged, the criteria can be borrowed, and the weighting can be argued over — all of which make it better.

If you want to talk about applying something like this in your organisation, email dwaine.mclean92@gmail.com.