The technical building blocks for AI-assisted marking have existed for at least two years. The open questions are no longer technical. They are pedagogical. This workshop, held at the 51st annual conference of the International Association for Educational Assessment, focused on one field: how students build their writing, and how AI feedback could help.
Writing is everywhere. Feedback is not.
Take a typical class in economics and social sciences. Students write all the time: notes, group work, occasional board work, homework. Most of this writing is never reviewed individually by the teacher.
Individual feedback happens mainly in formal assessment: one or two substantial written assessments a month, rarely more. With several classes and dozens of students in each, reading every response carefully, identifying weaknesses, writing comments and assigning a grade is a considerable workload. Papers often come back three or four weeks later.
Two consequences follow.
- The grade is final. By the time students get their work back, the process is over. They may read the comments, but they are rarely asked to revise the same piece and submit a new version.
- Feedback arrives too late. The class has moved on. The concepts and difficulties present when the student wrote are no longer at the centre of the lesson.
Language also takes a back seat. “This is not a French lesson”: the teacher must first assess concepts, evidence and argument. Unclear sentences, weak syntax and imprecise vocabulary are only partly addressed.
The paradox is clear. Writing is frequent; individual feedback is rare. The constraint is not a lack of interest. It is largely a question of time, and it keeps us far from formative assessment.
Visible difficulties, quietly normalised
Reading difficulties can stay invisible until a specific assessment reveals them. Writing difficulties are on every page. Correcting everything is unrealistic, so the standard quietly drops: the teacher reads generously, rebuilds the meaning, passes over errors. The problem is not that these difficulties escape notice. It is that they are so frequent and so costly to treat that they stop being treated.
Digital tools as new masks
Each tool conceals the difficulty it was meant to relieve. Spelling and grammar checkers removed surface errors from the page, without changing the command of the language underneath. Generative AI now does the same for the substance: composition, structure and argument can be outsourced whole.
Two competences are now required, and they pull in opposite directions: learning to use these tools, and learning to distrust them. The second is harder to teach. And the students who would gain most from them are often those who use them least, or least effectively.
From frontier capability to commodity
Early chatbots could comment on student work, but were too unstable for reliable assessment. From 2023, models became able to apply complex rubrics and justify their judgements. The current generation follows detailed rubrics and structured protocols far better. With calibration, worked examples and controlled prompting, high consistency becomes achievable in well-defined domains.
What was a frontier capability in 2023 is now widely available in open-weight models. That changes four things:
- Cost: inference becomes a marginal expense rather than a barrier.
- Sovereignty: models can run on-premise, keeping student data under institutional control.
- Independence: no lock-in to a single commercial provider.
- Reproducibility: a pinned open-weight model can be frozen, versioned and audited; a hosted model can change without notice.
The turning point is economic as much as technical. Teachers have always known that individual feedback works. The constraint was its cost per student.
What AI-assisted assessment is grounded in
Three foundations, usually combined:
- The model itself: its own knowledge, reasoning and language.
- Reference content: the course materials and expected answers taught to the students.
- An assessment rubric: the structure that guides the analysis.
Most disappointing results come from expecting the first foundation to do the work of the second.
The role of rubrics is shifting. With less capable systems, they broke a complex judgement into smaller questions. That function is becoming optional. Two others remain essential: constraining judgement to the intended criteria, and guiding attention to what matters pedagogically. The degree of structure becomes a pedagogical choice, not a technical necessity.
Consistency of judgement
A mark or a comment cannot change completely because a parameter was overlooked. A low sampling temperature, a fixed prompt, a pinned model version and a structured output all matter. Three practical rules follow:
- Measure stability. Mark the same script several times; the score and the main points should come back near-identical.
- Rule out ambiguity first. Instability often comes from the question, not from the model.
- Put models in competition. Compare several models with teacher marking. Agreement is a strong signal; disagreement shows where the task is under-specified.
The AI’s correction can be read by both the student and the teacher. It is not a verdict, but a proposal, to be discussed, validated or overruled.
What feedback addresses, and how deep it goes
Keep two questions apart: what the feedback is about, and how it is expressed. It can address five areas, and a teacher should be able to switch whole categories on or off:
| Area | Examples |
|---|---|
| Knowledge | Missing expected or prerequisite knowledge |
| Understanding | Misconceptions, confusion between concepts |
| Method and reasoning | Weak argument, unsupported claims, weak use of evidence |
| Quality of communication | Lack of clarity or concision, poor organisation |
| Written language | Spelling, grammar, syntax, register |
Good feedback is not only about errors. It identifies what is done well, prioritises what needs improving, suggests how, and encourages revision without overwhelming the student. An exhaustive error list is the easiest thing for a model to produce, and one of the least useful things a student can receive.
Feedback also has depth. It can stop at detection (“this concept is missing”), go on to explanation (“this weakens your argument, because…”), then guidance (“you need to distinguish X from Y”), and finally a prompt for revision (“can you rewrite this paragraph, making that distinction explicit?”). Detection alone suits summative contexts. The prompt for revision only makes sense where the student can resubmit.
The workshop
Participants used an interactive application, on their phone or computer. In a first phase, groups debated for and against AI-assisted feedback. In a second, assuming AI would be adopted, they helped define a research agenda, ranking priorities individually before the group ranking was computed.
Technology as pharmakon
In Plato’s Phaedrus, writing is offered as a pharmakon: a gift that supports memory, and a risk of forgetfulness for those who rely on it. Derrida stressed that the word means both remedy and poison. Stiegler argued that every technology is pharmacological: the same gesture can extend a human capacity or dispossess us of it.
AI is no exception. It may make feedback abundant, immediate and affordable. Badly designed, it may also reduce effort, dialogue and human agency. The real question is not whether AI is good or bad. It is what kind of learning environment we choose to build around it, and what must remain ours.