Measurement often appears to bring clarity to learning design because it gives visibility to activity, progress, completion, participation, assessment performance and satisfaction.
It helps teams identify patterns, compare experiences, report outcomes and make decisions about where support may be needed. In complex educational environments, that kind of visibility can be valuable, but measurement does not simply describe learning. It shapes what is noticed, what is prioritised, and what becomes easier to defend.
Once a measure becomes part of a system, it rarely remains neutral. This is especially important when the things that are easiest to measure are not always the things that matter most.
What Measurement Makes Visible
Learning platforms can usually tell us a great deal about visible activity. They can show whether learners accessed a resource, completed a quiz, clicked through a lesson, submitted an activity, opened feedback, or participated in a discussion. They can count attempts, timings, scores, logins and completions.
These data points can be useful because a learner who has not accessed key material may need support, a large number of failed quiz attempts may indicate confusion, and low participation in a discussion may suggest the activity is unclear, poorly timed, inaccessible, or insufficiently meaningful. But, as I’ve discussed in other blog posts, activity is not the same as understanding, completion is not the same as learning, participation is not the same as engagement, and satisfaction is not the same as impact.
Measurement can show that something happened, but the conclusions that can be drawn from these data points alone are limited.
The Pull of Easy Evidence
The pull of clear data points is easy to understand. Completion rates, attendance figures, quiz scores and platform analytics create an impression of control. They fit neatly on a dashboard to be compared and reported, allowing teams to demonstrate impact relatively quickly and easily. Dashboards full of data points appear to reduce uncertainty and can be disproportionately influential.
Learning design is judged through institutional systems, reporting cycles, funding requirements, workload pressures, quality assurance processes and stakeholder expectations in addition to educational quality. These institutional requirements influence what becomes measurable and therefore what becomes valuable.
When a measure is easy to collect, it is more likely to be used. Measuring quality can be a slower, harder process than simply downloading a report. ‘Easy evidence’ that can be presented clearly carries clear practical power and over time, substitutes for slower, perhaps more difficult evidence. The measure can become treated as if it represents the thing itself.
Easy measures, even when they are incomplete, can become attractive precisely because they appear to reduce uncertainty without requiring deeper investigation. That makes them easier to use both internally and externally and so they are used more regularly and eventually shape decisions at all levels.
That is a clear risk.
When Measures Become Targets
Another clear risk is that measures change behaviour. Particularly when they are the basis of decision making. For example, if completion is emphasised, people may design for completion. Or if satisfaction is emphasised, people may design for positive immediate responses. Emphasis isn’t automatically wrong, but who is paying the price if that emphasis becomes a target?
Completion matters but a course designed to maximise completion may reduce friction in ways that help learners, but it may also remove useful challenge. It may also create undue pressure for learners that causes them to disengage entirely. Participation can matter but a discussion designed to increase participation may generate more posts without improving the quality of thinking and a whole course designed to increase participation may privilege learners with time and ability to engage synchronously with the materials.
Targets focus attention. Targets based on incomplete evidence and information can shape designs into privileging particular learners. It is a designer’s responsibility to question measures, and the targets that may come from them, to ensure that privilege is visible.
What Remains Invisible
Some of the most important aspects of learning are difficult to measure directly. Learner identity, belonging, persistence, confidence, understanding, knowledge transfer, professional judgment and agency may all be shaped through subtle patterns of experience rather than single measurable events.
Accessibility also complicates measurement. A learner may complete an activity, but only by spending disproportionate effort. Another may appear inactive because they are engaging with downloaded materials, assistive technologies, alternative formats, or support outside the platform. A third may pass an assessment while still experiencing avoidable cognitive or navigational burden.
If the measurement system only captures visible platform behaviour, much of this remains unseen. What remains invisible is not necessarily unimportant. It is often simply harder to capture.
The Ethics of Interpretation
Measurement becomes ethically significant when limited evidence is used to support stronger claims than it can reasonably bear.
A dashboard may show that learners are not accessing a particular resource. It cannot, by itself, explain why. The resource may be unnecessary, badly signposted, inaccessible, poorly timed, irrelevant, intimidating, duplicated elsewhere, or simply not needed by learners with prior knowledge.
A low completion rate might reasonably trigger further investigation, but it would be difficult to defend using that measure alone as evidence that learners were disengaged. It may also indicate workload issues, unclear expectations, poor sequencing, external pressures, technical barriers, or a design that asks too much at the wrong moment. The same data point can support several interpretations.
This is where professional judgement becomes essential. The ethical issue is not only what data is collected. It is what is inferred from it, what is reported, what is ignored, and what decisions follow.
Designing for Better Questions
If measurement shapes learning design, then measurement decisions need to be made earlier. Not as an evaluation afterthought or a reporting requirement added at the end. Nor should it be whatever the platform happens to capture.
Measurement needs to be treated as part of the design itself.
That means asking what evidence would actually help us understand whether the learning is doing what it needs to do. It means distinguishing between activity, participation, understanding, transfer, confidence, access and impact. It means recognising which forms of evidence are useful, which are partial, and which might mislead if interpreted without context.
It also means accepting that some important questions will not produce clean numbers.
This can be uncomfortable in systems that prefer dashboards and simple indicators. But educational decisions often require evidence that is provisional, mixed, contextual and incomplete. The task is to design measurement that remains proportionate to the claims being made.
Measurement as a Design Responsibility
The question, then, is not whether learning designers should use data because they should. But they should also be weighing whether the data being used is capable of supporting the decisions being made from it.
Easy measures have a place. Completion, attendance, participation, access and performance data can all provide useful signals. But signals are not explanations. They require interpretation, context and caution because when easy measures become the only measures, learning design begins to narrow around what can be seen.
When dashboards are treated as accounts of learning rather than partial views of behaviour, important experiences disappear and educational priorities can quietly shift when dashboard metrics become targets.
This is why measurement belongs in a series about ethics and responsibility.
Measurement decisions shape attention, influence behaviour, and affect what can reasonably be claimed about learners and learning. The responsibility is to recognise what a measure can show, what it cannot show, and what becomes harder to see when the easy evidence is allowed to stand in for the important question.
Leave a Reply