The EU AI Act just ruled out most school safety AI. Here's what survives.
Article 5 prohibits emotion inference in education outright. That single clause disqualifies a large share of the student-wellbeing products currently being sold to schools.
Most conversations about the EU AI Act focus on the high-risk tier: the conformity assessments, the documentation, the CE marking. For schools, that is not the clause that matters most.
The clause that matters is Article 5, which does not regulate certain practices. It prohibits them.
What Article 5 says about classrooms
Among the prohibited practices is the use of AI systems to infer emotions of a person in the areas of workplace and education institutions. There are narrow carve-outs for medical or safety reasons, but the general position is unambiguous: a system that looks at a student and produces a claim about how that student feels is not permitted in a European school.
This is a stricter position than the Act takes almost anywhere else. Emotion inference in most other contexts lands in the high-risk category — permitted, with obligations. In education it is simply off the table.
The reasoning is not hard to reconstruct. Students are a captive population with a significant power imbalance relative to the institution, they are frequently minors, and the scientific basis for inferring internal emotional states from facial expression is contested. Put those together and a prohibition is a defensible regulatory conclusion.
Why this disqualifies a lot of products
A meaningful share of "student wellbeing AI" on the market does exactly what Article 5 prohibits. The pattern is familiar:
- Analyse classroom video for facial expressions
- Produce an engagement, attention, stress or mood score per student
- Surface a dashboard showing which students are "struggling"
Every product built on that pattern is now selling something a European school cannot lawfully deploy. Some vendors have responded by renaming the output — "engagement analytics" rather than "emotion detection" — which does not help, because the Act regulates what the system does rather than what the feature is called. If the system infers an emotional state from observation, the label on the dashboard is irrelevant.
The distinction that actually holds up
There is a bright line here, and it is worth stating precisely, because it is the line every school safeguarding purchase should now be evaluated against.
Detecting an event is not inferring an emotion.
A system that identifies a physical altercation in a corridor is detecting an event: two people, specific movements, an observable occurrence. It makes no claim about anyone's internal state. It says a fight is happening here, not this student is angry.
A system that watches a student's face and outputs "distressed" is inferring an emotion. It makes a claim about an internal state from external observation, and it does so about a specific identified student in an education setting.
The first is a safety detection system. The second is prohibited. They can look superficially similar in a product demo, and they are legally worlds apart.
Where wellbeing monitoring can still live
None of this means schools must abandon early identification of struggling students. It means the data has to come from a different place: the student.
Self-reported, opt-in check-ins built on validated psychological instruments are not covered by the Article 5 prohibition, because nothing is being inferred. The student is telling you. The system is recording what they said and tracking how it changes over time.
That distinction has a second benefit that is not legal at all. Inferred emotional data is, in practice, unreliable — the mapping from facial expression to internal state is weak, culturally variable, and worse for exactly the students most likely to be struggling. A self-report is a genuine signal from the person who actually has the information.
The compliant design and the effective design turn out to be the same design.
Questions worth asking a vendor
If you are evaluating safeguarding technology for a school in the EU — or in the UK, where equivalent scrutiny is coming through a different route — these questions separate the products that will survive from the ones that will not:
- Does the system produce any output describing a student's emotional or psychological state?
- If it flags a student as at risk, what is that flag derived from — observation, or something the student submitted?
- Was the product built without emotion inference, or was the feature removed to comply?
- Can you show me, in the data model, where a wellbeing signal originates?
Question three matters more than it looks. A product built around emotion inference and then stripped of it has a hole where its core feature used to be. A product designed without it never depended on it. Those are different products with different roadmaps, whatever the current feature list says.
The compliance-first position
There is a version of this argument that treats regulation as an obstacle to route around. That version ages badly.
The AI Act's prohibitions on educational settings exist because the drafters concluded that some categories of processing should not happen to children in institutions that hold power over them. A school evaluating safeguarding technology is not just checking a legal box. It is deciding what it is willing to do to its own students.
The technology that clears the regulatory bar is, in this case, also the technology that treats students as people who can be asked rather than subjects to be read. That is not a coincidence, and it is not a constraint to be minimised.