Most contemporary discussions of artificial intelligence in courts centre on regulation that has yet to arrive. This piece concerns a legal problem that is already present, embedded in a provision most practitioners associate with credit scoring and automated hiring decisions rather than with the administration of justice.
Article 22 of the General Data Protection Regulation grants individuals the right not to be subject to decisions based solely on automated processing that produce legal or similarly significant effects.[1] Most practitioners understand this as a constraint on banks, insurers, and automated recruitment systems. Few have paused to consider what it means when a judge, facing a dense commercial case file and a caseload that never shrinks, reaches for an AI drafting assistant, adopts the text it produces, and signs off on the result.
The answer is more uncomfortable than most courts have yet acknowledged.
In October 2023, the Court of Justice of the European Union delivered its judgement in OQ v Land Hessen, also known as the SCHUFA case.[2] The Court clarified that a decision is based solely on automated processing not only when no human is formally present in the process, but also when human involvement is purely formal. When a decision-maker does not independently assess the relevant circumstances but simply endorses an automated result, Article 22 is already engaged.[3] This has immediate consequences for courts that almost no one is yet discussing.
It is tempting to frame AI in courts as a future risk: something to regulate before it becomes a problem; nonetheless, that framing is already outdated. Generative AI tools are being used today to summarise case files, retrieve legal precedents, draft procedural sections of judgements, and structure lines of reasoning. Some of this use is disclosed; much is not. Courts in Singapore, the UAE, and England and Wales have issued guidance on responsible use[4], and the EU AI Act expressly classifies AI systems used to assist judicial authorities as high-risk.[5] But none of these instruments has squarely addressed the Article 22 question.
The question is the following: if a judge adopts an AI-generated reasoning paragraph without independently verifying it against the evidentiary record and the applicable law, not as a matter of bad faith, but because the docket is full and the text appears coherent, is that decision considered based solely on automated processing? Under the SCHUFA standard, the answer could credibly be yes.
A judgement is among the most significant legal effects imaginable. It can extinguish a right, transfer property, liquidate a company, or end a commercial relationship. There is no sector-specific exemption from Article 22 for courts, only a general authorisation for Member States to legislate appropriate safeguards. If those safeguards are absent or unclear, a party holds a credible legal basis to challenge the validity of a judgement on data protection grounds. No such challenge has yet succeeded in an EU court. But the absence of precedent is not the same as the absence of risk. It reflects, primarily, the absence of awareness.
The problem compounds when the EU AI Act is brought into the picture. Article 14 requires that high-risk systems be subject to meaningful human oversight: the user must understand the system’s limitations, remain alert to the tendency to over-rely on outputs, and retain the capacity to override suggestions.[6] The AI Act and the GDPR do not operate as alternatives; they run in parallel. A judge who mechanically accepts AI-generated content satisfies neither instrument.
What meaningful oversight should require in adjudication is not yet settled. I propose, at minimum, four things: that the judicial user has received documented training on the tool’s known limitations and error patterns; that AI-assisted reasoning is actively tested against the case record rather than passively adopted; that, for high-stakes dispositive decisions, a second reviewer checks AI-generated content before it is finalised; and that an internal note documenting the use and its extent is retained for appellate inspection. These are not principles invented for AI governance. They reflect what Article 6 of the European Convention on Human Rights, which requires decisions attributable to a reasoning human mind, has always demanded.[7] AI does not create new obligations so much as expose how thin existing compliance can be under operational pressure.
The regulatory gap runs in both directions. The European Data Protection Board has not issued sector-specific guidance on Article 22 in judicial contexts.[8] The AI Act’s implementing provisions continue to be developed. Courts across the EU are therefore navigating the intersection of two major regulatory frameworks without a map for where they meet.
Under the Brussels Ibis Regulation and the New York Convention, a party resisting recognition or enforcement of a judgement or arbitral award may raise public policy objections.[9] An argument that the originating court produced its decision through a process that violated Article 22 GDPR is legally cognisable under existing law. The longer the regulatory vacuum persists, the more enforcement proceedings risk becoming a forum for relitigating AI governance failures that should have been addressed by the institutions that deployed the tools.
What I propose, and what no jurisdiction has yet adopted, is sector-specific guidance from the European Data Protection Board clarifying three things: which judicial functions cannot be delegated to automated systems under EU law; what genuine human involvement in adjudication requires, as distinct from mere formal approval; and what contestation rights parties hold when AI has materially shaped proceedings.[10] Alongside this, courts deploying AI tools require structured, case-level disclosure mechanisms, not abstract statements about the general use of technology, but specific information telling parties what was used, for what purpose, and under what oversight.
The technology is already in use. The legal exposure already exists. The regulatory clarity does not.
Courts have always understood that legitimacy depends on decisions being traceable to a reasoning human mind. Article 22 GDPR, read through the SCHUFA lens, simply puts that principle into a form that generates enforceable rights. The question now is whether judicial institutions will respond to that before a challenge forces them to.
[1] Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data and repealing Directive 95/46/EC (General Data Protection Regulation) [2016] OJ L119/1 (GDPR), art 22(1).
[2] Case C-634/21 OQ v Land Hessen (SCHUFA) EU:C:2023:950.Regulation
[3] ibid paras 57–63. The Court held that a scoring value produced by a credit reporting agency constituted automated decision-making within the meaning of art 22(1) GDPR where the bank granting or refusing credit relied heavily on that score, even though the bank formally made the final decision.
[4] Courts and Tribunals Judiciary (England and Wales), Artificial Intelligence (AI): Guidance for Judicial Office Holders (12 December 2023, updated 15 April 2025); Supreme Court of Singapore, Guide on the Use of Generative Artificial Intelligence Tools by Court Users (23 September 2024); DIFC Courts, Practical Guidance Note No 2 of 2023: Guidelines on the Use of Large Language Models and Generative AI in Proceedings before the DIFC Courts (21 December 2023).
[5] Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence [2024] OJ L 172/1 (EU AI Act), Annex III, point 8(a) (systems ‘intended to be used by a judicial authority or on their behalf to research and interpret facts and the law and to apply the law to a concrete set of facts’).
[6] EU AI Act (n 5) art 14(4)(a)–(b).
[7] Convention for the Protection of Human Rights and Fundamental Freedoms (adopted 4 November 1950, entered into force 3 September 1953) ETS 5 (ECHR), art 6; Council of Europe, Commission for the Efficiency of Justice (CEPEJ), European Ethical Charter on the Use of Artificial Intelligence in Judicial Systems and their Environment (December 2018), Principle V (‘under user control’).
[8] The EDPB’s existing guidance — Guidelines on Automated Individual Decision-Making and Profiling for the Purposes of Regulation 2016/679 (adopted 6 February 2018) — addresses private-sector profiling and contains no provisions specific to judicial or quasi-judicial decision-making.
[9] Regulation (EU) No 1215/2012 of the European Parliament and of the Council of 12 December 2012 on jurisdiction and the recognition and enforcement of judgements in civil and commercial matters [2012] OJ L351/1 (Brussels Ibis Regulation), art 45(1)(a); Convention on the Recognition and Enforcement of Foreign Arbitral Awards (New York, 10 June 1958) 330 UNTS 3 (New York Convention), art V(2)(b).
[10] GDPR (n 1) art 22(2)(b) permits Member State law to authorise automated decisions with ‘suitable measures to safeguard the data subject’s rights and freedoms and legitimate interests’; no EU Member State has enacted such a measure specific to the judicial context.
