The AI Act
Cybersecurity is essential to building trust in AI and to protecting users and society, under the EU's AI Act, which introduces comprehensive legal requirements from 2027.
The regulation divides AI into four risk levels. Harmful applications are banned. High-risk AI systems are subject to requirements on transparency, technical documentation and logging. They must be designed to withstand and respond to cyberattacks and be protected against manipulation and unauthorised access — in other words, be robust.
The documentation must provide details on data protection and protection against cyber threats. Logging must make it possible to trace and record all significant events so that security flaws can be detected and remedied and incidents investigated.
AI systems must be regularly updated and maintained to remain secure over time. Providers must be able to demonstrate that they have procedures in place for managing updates in a way that does not compromise the security of the system.
The regulation specifically notes that inadequate cybersecurity opens the door to deliberately induced errors in machine learning, such as the risk of poisoning attacks against training datasets (which affects integrity, in the confidentiality-integrity-availability triad) and the risk of membership inference attacks (which affects confidentiality and can breach privacy), whereby an attacker can determine whether a particular data record was part of the model's training data.
The legal requirements apply to companies that develop, supply, operate and maintain AI, as well as to importers, distributors, product manufacturers and, to some extent, other actors in the value chain. Providers of high-risk systems must have security controls in place and manage the underlying infrastructure.
Sanctions for breaching the regulation can include substantial fines
(EUR 20 million or four percent of the company's annual turnover)
for example for using prohibited AI, or moderate but significant fines
(EUR 10 million or two percent of turnover)
for example for insufficient transparency or documentation.
In Sweden, several authorities are likely to cooperate on supervision. These include the Swedish Authority for Privacy Protection (IMY), the Swedish Post and Telecom Authority (PTS),
the Swedish Civil Contingencies Agency (MSB), the Swedish Consumer Agency, the Swedish Work Environment Authority and the Swedish Financial Supervisory Authority (Finansinspektionen), working together with the European Artificial Intelligence Board. National authorities must ensure an appropriate level of cybersecurity. The EU's cybersecurity agency, ENISA, plays an advisory role.
Scope limitations and exemptions cover AI that makes decisions without human oversight but according to "rules defined solely by natural persons to automatically execute operations." The rules do not apply to scientific research and development, commercial research, development and prototyping prior to market launch, personal non-professional use, or use by the military or for national security purposes. There are also certain exemptions for open-source AI.
Narrow procedural tasks, improving the outcome of a previously completed human activity, detecting decision-making patterns or deviations from patterns, and preparing an assessment are not considered high-risk — unless they involve profiling individuals.
Biometric systems used solely for cybersecurity and data protection purposes are not considered high-risk. Components used solely for cybersecurity purposes are excluded, such as fire alarms in server rooms and data centres.
AI that meets other EU cybersecurity regulations is considered to satisfy the cybersecurity requirements of the AI Act. Certifications and declarations of conformity issued under other EU regulations are also valid for the cybersecurity requirements of the AI Act.
The regulation was adopted by the European Parliament in March and the Council of the EU in May 2024. High-risk systems must meet the requirements after 36 months, i.e. by 2027. Most of the legal requirements take effect after 24 months. The ban on unacceptable AI systems applies after six months, codes of conduct after nine months, and transparency rules after 12 months.
The regulation encourages the development and use of international and European cybersecurity standards.