ChatGPT arrives as the Council fixes its position on the AI Act

On 30 November OpenAI released ChatGPT, a model that answers questions in a "dialogue format" and can respond to follow-up questions[1]. OpenAI offers it as a "research preview" to collect feedback, and "usage of ChatGPT is free" during that period[1]. Two days later, users on Twitter are sharing essays, code and poems written by the system, and OpenAI's chief executive has said that demand is "a lot more" than expected[2]. On 6 December the Telecommunications Council is due to adopt its general approach on the proposed Regulation laying down harmonised rules on artificial intelligence, on the basis of a Czech presidency text of 25 November[5]. The two events do not refer to each other, and they should.

What was released

ChatGPT is fine-tuned from "a model in the GPT-3.5 series" and trained with "Reinforcement Learning from Human Feedback" as a "sibling model to InstructGPT"[1]. OpenAI states the limits itself. The system "sometimes writes plausible-sounding but incorrect or nonsensical answers," and the company uses its Moderation API to "warn or block certain types of unsafe content," with false negatives and false positives[1]. Critics have already shown a detailed and wrong answer to an algebra problem, and prompts that override the limits on gore, crime and racism[2].

The precedent of the fortnight is closer. Meta released Galactica on 15 November, a language model trained on 48 million scientific papers and meant to write literature reviews and wiki entries; after reports that it produced racist, antisemitic and false text, including instructions for making napalm, Meta paused the demo after three days[3]. ChatGPT has survived its first three days. The difference is that OpenAI built a dialogue model with refusals and a moderation layer, and that it published its limitations on the launch page[1]. Whether those measures hold under mass use is now a public experiment, and nobody in Europe has been asked to consent to it.

Where the Regulation stands

The Commission proposed the Artificial Intelligence Act on 21 April 2021 as "the first ever legal framework on AI"[4]. The proposal sorts systems into four tiers. Unacceptable risk is banned, including "systems that allow 'social scoring' by governments"; high risk is permitted under obligations on risk management, data quality, logging, documentation and human oversight; limited risk carries transparency duties; minimal risk is free[4]. For chatbots the proposal is explicit: "users should be aware that they are interacting with a machine"[4]. A model that can write a student's essay, a job application or a legal letter on request does not fit that tier neatly. The text that the Commission wrote in 2021 assumes systems built for a purpose. ChatGPT has none.

The Council has seen the problem. The Czech presidency text of 25 November narrows the definition of an AI system in Article 3(1) to machine learning and logic- and knowledge-based approaches, and it inserts a new Title IA on general purpose AI systems[5]. The French presidency had tabled the first draft of that title, Article 4a, on 13 May[8]. Under the Council text, certain high-risk requirements would apply to general purpose systems, but implementing acts adopted by the Commission would specify how[5]. The recital states that "implementing powers should be conferred on the Commission" to adapt the requirements to such systems[6]. The text also excludes military, defence and national security purposes from the Regulation whatever the entity involved, extends the social scoring prohibition "also to private actors," adds a layer so that systems whose output is "purely accessory" to a decision are not high risk, and permits "unsupervised real world testing of AI systems" in sandboxes under safeguards[5]. On interaction with people, recital 70 keeps the rule that "natural persons should be notified that they are interacting with an AI system"[5].

The Parliament is slower. The Internal Market and Civil Liberties committees received some 3,000 amendments and debated them on 30 June; the rapporteur Brando Benifei called a fundamental rights impact assessment before deployment "a crucial point that is missing in the Commission's proposal," and Members disagree on the definition itself[7]. The committees published a first set of compromise amendments in July[9]. No committee vote has taken place, and the co-rapporteurs, Brando Benifei and Dragoș Tudorache, have been in charge since 1 December 2021[8]. On liability, the Commission proposed on 28 September an AI Liability Directive and a revised Product Liability Directive; the first creates a presumption of causality where a fault is proven and "a causal link to the AI performance seems reasonably likely," and a right of access to evidence from providers of high-risk systems[10][11]. Both proposals await the co-legislators[10].

What the Parliament should add

The Council answers general purpose models with a delegation: the Regulation would name the category and leave its obligations to implementing acts written later by the Commission[5][6]. That choice puts the substance of the rule that will govern ChatGPT and its successors outside the legislative procedure. For a generation that will spend its working life with these systems, that is the wrong place. We make four recommendations.

We recommend that the Parliament write the obligations of providers of general purpose AI systems into the Regulation itself, including technical documentation, a description of training data, testing against foreseeable misuse and an incident reporting duty, and that it leave to implementing acts only the technical detail of compliance.

We recommend that the co-legislators extend the transparency rule for interaction with natural persons to the output of such systems, so that text, images and audio generated at scale and disseminated to the public carry a machine-readable indication of their origin.

We recommend that the Commission publish, before the Parliament's committee vote, guidance for schools and universities on assessment in the presence of dialogue models, since examination boards will meet ChatGPT before market surveillance authorities do.

We recommend that the Council and Parliament treat the AI Liability Directive and the Product Liability Directive as part of the same package as the Regulation and conclude all three in this legislature, so that a presumption of causality exists when the first harm from a general purpose system reaches a court.