What the EU AI Act Changes About How You Build Software

Articles 9 to 15 are usually read as a documentation exercise. They are not. A CTO's view of what actually changes in the codebase, the pipeline and the team.

By Rafal Skucha

Most EU AI Act coverage is written by lawyers for lawyers, and it treats compliance as a documentation problem. Produce the technical file, run the conformity assessment, apply the CE marking.

That framing is wrong in a way that costs money. The obligations in Articles 9 to 15 are not things you write down about a system. They are properties the system has to have.1 You cannot document your way into logging that was never built, or human oversight that the interface makes impossible. Teams that treat this as paperwork discover in the final quarter that they are rewriting.

This is the engineering read. It applies if you are building anything that lands in Annex III - biometrics, critical infrastructure, education, employment, essential services including credit scoring, law enforcement, migration - where the full regime starts on 2 December 2027. If you are not in that territory, most of this is optional good practice rather than law, and I will say so at the end.

Risk management stops being an annual document

Article 9 requires a risk management system maintained across the lifecycle: identify risks, evaluate them, mitigate, test, and document residual risk with someone accountable for accepting it.

The engineering consequence is that “we did a risk assessment before launch” does not satisfy this. It is continuous. When you change a model, retrain on new data, or expand into a new user population, the analysis is supposed to run again.

In practice this means a risk register that lives in version control next to the code rather than in a governance tool nobody opens, and a trigger in your change process that asks whether this change alters the risk profile. Teams that already do threat modelling properly will recognise the shape of it.

Data governance becomes a build-time concern

Article 10 requires training, validation and test data to be relevant, sufficiently representative and, as far as possible, free of errors - with active examination for bias.

The uncomfortable engineering truth: most teams cannot answer basic questions about their training data. Where did it come from, what licence covers it, when was it collected, what population does it represent, what changed between v1 and v2. That information was never captured because nobody needed it.

Retrofitting data lineage is one of the most expensive things on this list. If you build anything that might land in Annex III, start recording provenance now, even if the deadline is sixteen months away. This is the item where late starts hurt most.

The Omnibus made one part of this easier. Processing special-category data - health, biometrics, race - specifically to detect and mitigate bias now has a clearer legal basis with safeguards.2 Before that, teams were told to eliminate bias while being unable to lawfully hold the data that would let them measure it.

Logging is a product requirement, not an ops convenience

Article 12 requires high-risk systems to log automatically over their lifetime, to a standard that supports traceability.

Most systems log for debugging. That is a different thing. Debug logs are sampled, rotated aggressively, unstructured, and frequently contain personal data nobody intended to retain. What is required here is a record that lets you reconstruct what the system did and why, months later, for a specific decision affecting a specific person.

That is a design decision with real cost: storage, retention policy, structure, and access control tight enough that the audit trail is not itself a data protection problem. It has to be in the architecture. It is painful to add afterwards.

Human oversight has to be designed, not asserted

Article 14 requires that systems be built so people can effectively oversee them - understand the output, interpret it correctly, decide not to use it, and intervene or stop the system.

This is the requirement that most often collides with product design, because “a human reviews it” is usually a fiction. If your interface shows a recommendation with an Accept button and a queue of four hundred waiting items, you have not built oversight. You have built a rubber stamp with a human liability shield attached.

Doing this honestly means showing the reviewer why the system produced that output, making the override path as easy as the accept path, and measuring override rates. If nobody ever overrides, that is not evidence the model is good; it is evidence the oversight is theatre.

Accuracy, robustness and cybersecurity get stated in public

Article 15 requires appropriate levels of accuracy, robustness and cybersecurity - and accuracy metrics declared in the instructions for use.

Publishing your accuracy figures changes behaviour. It forces a decision about what you actually measure, on which population, and how it degrades. Teams that have been quietly reporting a flattering aggregate number find that harder to sustain when it is in a document a regulator can read.

The cybersecurity limb pulls in AI-specific attacks - data poisoning, adversarial inputs, model extraction, prompt injection where relevant. The NCSC guidelines are the clearest free reference for this and map well onto the requirement.3

What this does to your delivery process

Five practical consequences, in rough order of how much they hurt.

Definition of done grows. Technical documentation and logging become acceptance criteria for high-risk work rather than follow-up tickets. Estimate accordingly.

Model changes become releases. Swapping the underlying model, changing a prompt that materially alters behaviour, retraining - these need the same change control as a code deploy, with the same record.

You need a rollback story for models. Version them, keep the previous one deployable, and know what data each version saw.

Someone owns this. Not a committee. In the businesses that handle it well there is one named person - usually the CTO or a lead engineer - who signs off residual risk and whose name is on the file.

Your suppliers are in scope of your obligations. If a third party supplies a component inside your high-risk system, you need a written agreement covering the information and technical access you require from them. That is Article 25, and it works in the other direction too - we have written it up from the supplier’s side in supplying tech services to EU clients.

What if you are not building high-risk systems?

Then almost none of the above is legally required of you, and you should be sceptical of anyone telling you otherwise.

What applies to ordinary businesses is transparency: tell people when they are interacting with AI, and mark synthetic content. That has applied since 2 August 2026. Systems already on the market before that date have until 2 December 2026 for the machine-readable marking requirement.

But look again at the list. Data provenance. Structured logging. Versioned models with a rollback path. Meaningful human review. Declared accuracy. Every one of those is something a competent engineering organisation should want regardless of jurisdiction, and the measured evidence on AI-assisted development points the same way. Every one of them is also a question your customers’ procurement teams will eventually ask.

My honest view: build the cheap 80% of this because it makes the system better, not because Brussels says so. Skip the conformity assessment apparatus until you are actually in scope. The failure mode I see most often is a company with no high-risk exposure spending a quarter on compliance theatre while shipping nothing.

If you want a straight answer on which side of that line your product sits, book a free half-hour. Our AI governance policy guide has the written template we use with engineering teams, free and without a form.

References

  1. Regulation (EU) 2024⁄1689, Articles 9 to 15, on EUR-Lex.
  2. Orrick - EU AI Act Update: Digital Omnibus Finalizes 8 Compliance Changes, July 2026, and the European Commission’s regulatory framework overview.
  3. National Cyber Security Centre - Guidelines for secure AI system development.
← Back to Blog