Reglog

← Guides & analysis

Is a customer-service chatbot high-risk under the EU AI Act?

· 9 min read

Usually not. A chatbot that answers questions, tracks orders, handles returns or books appointments is not doing any of the jobs listed in Annex III, and Annex III is the list that makes a system high-risk under Article 6(2). It becomes a different question when the chatbot itself evaluates creditworthiness, prices life or health insurance, assesses eligibility for essential public benefits by or on behalf of a public authority, filters job applications, or classifies emergency calls. Those are Annex III use cases. Some rules apply to every chatbot regardless, including the Article 50(1) duty to tell people they are talking to an AI system, which has applied since 2 August 2026, and the Article 5 prohibitions on manipulative and exploitative AI.

This is an information service to help you orient, not legal advice. For the obligations tied to your own AI use, see which apply to your company.

Your chatbot, row by row

Your chatbot does… High-risk? Why (article / annex point)
Answers FAQs, tracks orders, handles returns or complaints No Not an Annex III use case (Art. 6(2)). Art. 50(1) disclosure applies.
Collects a loan application and passes it on; a separate system does the scoring Usually not the chatbot; yes the scoring system Creditworthiness evaluation is Annex III point 5(b). On our reading, a bot that only gathers information is either not intended to evaluate creditworthiness at all or performs a preparatory task (Art. 6(3)(d)); this is not settled. If the bot screens people out, that reading weakens; if it profiles them, the Art. 6(3) derogation cannot apply.
Itself decides or scores credit eligibility, or prices life or health insurance Yes Annex III point 5(b) (creditworthiness, credit score; fraud detection is excluded) or 5(c) (risk assessment and pricing in life and health insurance).
Is used by or on behalf of a public authority to evaluate eligibility for essential public assistance benefits, or to grant, reduce, revoke or reclaim them Yes Annex III point 5(a). Answering general questions about a benefit is a different purpose and, on our reading, not covered.
Screens job applicants, filters applications or evaluates candidates Yes Annex III point 4(a). See AI in HR and recruitment.
Evaluates and classifies emergency calls, or sets dispatch priority Yes Annex III point 5(d). A support bot that only escalates "urgent" product tickets is not dispatching emergency first response services.
Infers a caller's emotions from their voice Likely yes An emotion recognition system infers emotions from biometric data (Art. 3(39)), which includes behavioural characteristics (Art. 3(34)); it is Annex III point 1(c), and Art. 50(3) adds a notice duty for the deployer (guidelines, para. 102). Merely detecting a raised voice is not emotion recognition unless used to infer emotions (Recital 18). Inferring your own agents' emotions at work is prohibited (Art. 5(1)(f)). Text-only sentiment scoring is not based on biometric data on our reading; the line is not settled.
Is built by you on a third-party model, or bought as a vendor product Depends on the purpose, not the source Built and put into service under your name: you are the provider (Art. 3(3)). Bought and used: you are a deployer (Art. 3(4)). Repurposed for an Annex III task: Art. 25(1)(c) makes you the provider of a high-risk system.

Why customer service is not in Annex III

Article 6 has two doors into the high-risk tier. The first, Article 6(1), covers AI that is a safety component of, or is itself, a product under the EU product-safety legislation in Annex I that needs third-party conformity assessment. A support chatbot is almost never that, and since July 2026 the Act states that AI used solely for non-safety aspects of user assistance, service efficiency or convenience does not qualify as a safety component (Art. 6(1a), added by Regulation (EU) 2026/1744). The second, Article 6(2), says that "AI systems referred to in Annex III shall be considered to be high-risk."

Annex III is a list of eight areas (the Commission can add use cases within them by delegated act, Art. 7): biometrics, critical infrastructure, education, employment, access to essential private and public services and benefits, law enforcement, migration and border control, and the administration of justice and democratic processes. "Customer service" is not one of them. What Annex III lists are specific purposes inside those areas, and the test is what the system is intended to be used for. The Act defines intended purpose as the use the provider intends, including the context and conditions of use, as set out in its instructions, promotional materials and technical documentation (Art. 3(12)).

That is why the chat window is legally irrelevant. A bank's chatbot that explains opening hours is not high-risk; one that decides who gets a loan is, because of what it decides.

Two refinements matter at the edges:

  • The Article 6(3) derogation. An Annex III system is not high-risk where it does not pose a significant risk of harm to health, safety or fundamental rights, including by not materially influencing the outcome of decision-making. At least one of four conditions must be met: a narrow procedural task, improving a completed human activity, detecting deviations from decision patterns without replacing human review, or a preparatory task. A provider relying on it must document the assessment and register the system (Art. 6(4)). The details are in our Article 6(3) guide.
  • Profiling is never exempt. Article 6(3) says an Annex III system "shall always be considered to be high-risk where the AI system performs profiling of natural persons", using the GDPR definition of profiling (Art. 3(52)). A loan-intake bot that builds a picture of an applicant's financial situation to sort them is a much weaker candidate for the derogation than one that only fills in a form.

For the general test, see the series hub: is your AI system high-risk?

What applies to every customer-service chatbot

Article 50(1): tell people it is AI. Providers must design AI systems intended to interact directly with people so that those people are informed they are interacting with an AI system, unless that is obvious to a reasonably well-informed, observant and circumspect person, taking into account the circumstances and the context of use. Article 50(5) requires the information to be clear and distinguishable and given at the latest at the first interaction. The Commission's Article 50 guidelines (approved in July 2026) list "chatbots/conversational agents" in customer support and complaints management as in scope (examples after para. 31), and give helpdesk chatbots whose replies users may perceive as human-generated as an example that does not fall under the obviousness exception (examples after para. 45). A single, prominent notification before the first interaction "is likely to suffice in most instances", but periodic reminders are likely to be necessary in riskier contexts, which the guidelines say include financial advice, insurance and complaints handling (para. 40). For wording you can adapt, use the Article 50 disclosure template; for the full rule, the Article 50 guide.

Because its replies are AI-generated text, the provider may also owe the separate Article 50(2) marking duty, which has applied since 2 August 2026; providers of systems placed on the market before that date have until 2 December 2026 (Art. 111(4), added by Regulation (EU) 2026/1744). See watermarking when you build on a model API.

Article 4: AI literacy. Providers and deployers must take measures to support the development of AI literacy of their staff and others operating AI on their behalf. The Digital Omnibus softened this wording in July 2026. See what Article 4 requires.

Article 5: the prohibitions. Article 5(1)(a) bans AI that uses subliminal, purposefully manipulative or deceptive techniques to materially distort behaviour in a way that causes, or is reasonably likely to cause, significant harm. Article 5(1)(b) bans exploiting vulnerabilities due to age, disability or a specific social or economic situation with the same effect. A retention bot designed to pressure a financially vulnerable customer into a harmful decision is the kind of design these provisions were written for. Both have applied since 2 February 2025. See the prohibited practices.

Who owes what. The Article 50(1) design duty sits with the provider, defined in Article 3(3) as whoever develops an AI system, or has one developed, and puts it on the market or into service under its own name. A company that builds a support bot in-house and puts it into service for its own use under its own name is its provider (the guidelines give this example after para. 11). If you run a vendor's chatbot, you are its deployer (Art. 3(4)). For a role-by-role map, see deploying AI agents in the EU.

If your chatbot does become high-risk

The high-risk obligations for Annex III systems apply from 2 December 2027, after Regulation (EU) 2026/1744 amended Article 113. If you deploy an Annex III high-risk chatbot that makes or assists in making decisions about people, Article 26(11) adds a separate duty to tell them they are subject to a high-risk system. The Commission's guidelines describe that as complementary to the Article 50 notice, though the two can be combined in one notification (guidelines, footnote 14 to para. 25). Certain deployers, including public bodies, private entities providing public services, and users of credit-scoring and life-and-health-insurance pricing systems, must also carry out a fundamental-rights impact assessment (Art. 27). The full list is in deployer obligations under Article 26. For lenders specifically, see credit scoring under the AI Act.

What to do now

  1. Write down what each chatbot decides, not what it is called. If it only informs, routes or collects, say so. If it scores, prices, filters or rejects, flag it.
  2. Check every flagged bot against Annex III points 4 and 5, and against point 1(c) if it analyses voices.
  3. Separate intake from decision. If a bot only gathers data for a scoring system, document that boundary and check whether it profiles anyone.
  4. Show the AI notice before the first interaction on every channel (chat, voice, email), and keep a record of where it appears.
  5. Review retention and sales flows for pressure tactics aimed at vulnerable customers.
  6. Record your role for each bot: built (provider) or bought (deployer).

To check your own case, answer a few questions about your AI use, or go back to the high-risk test. To get a weekly email of verified changes once it launches, join the waitlist.

The official text is Regulation (EU) 2024/1689, as amended by Regulation (EU) 2026/1744. This article is an information service to help you orient — it is not legal advice.

Frequently asked questions

Is a customer-service chatbot high-risk under the EU AI Act?

Usually not. Article 6(2) makes an AI system high-risk through Annex III only when it is intended for one of the listed use cases, and answering customer questions, tracking orders or handling returns is not on that list. A chatbot can become high-risk if it is used for an Annex III purpose itself, for example evaluating a person's creditworthiness, risk assessment and pricing in life or health insurance, evaluating eligibility for essential public assistance benefits by or on behalf of a public authority, filtering job applications, or evaluating and classifying emergency calls.

What applies to a customer-service chatbot that is not high-risk?

Three things. Article 50(1), applicable since 2 August 2026, requires the provider to design the chatbot so that people are informed they are interacting with an AI system, unless that is obvious, at the latest at the first interaction (Article 50(5)). Article 4, as amended in July 2026, requires providers and deployers to take measures to support the development of AI literacy of their staff. And the Article 5 prohibitions, including manipulative or deceptive techniques and the exploitation of vulnerabilities that cause significant harm, apply to every AI system. Articles 4 and 5 have applied since 2 February 2025.

Does it matter whether we built the chatbot on a third-party model or bought a vendor product?

It changes your role, not the risk class. If you build the chatbot and put it into service under your own name, you are its provider under Article 3(3) and carry the Article 50(1) design duty. If you use a vendor's chatbot under your authority, you are a deployer under Article 3(4). If you take a chatbot that was not high-risk and change its intended purpose so that it becomes high-risk, Article 25(1)(c) makes you the provider of a high-risk system.

When would the high-risk obligations apply to a chatbot used for an Annex III purpose?

From 2 December 2027. Article 113, as amended by Regulation (EU) 2026/1744, applies Chapter III Sections 1 to 3, which contain the classification rules, the requirements and the provider and deployer obligations, from that date for systems that are high-risk through Article 6(2) and Annex III. The Article 50 transparency duties were not moved to 2027: they have applied since 2 August 2026, with a 2 December 2026 deadline for the Article 50(2) marking duty for systems placed on the market before 2 August 2026 (Article 111(4)).

See which obligations apply to your company → or join the waitlist

This is an information service, not legal advice. How we check our guides: methodology.