# data[].attributes.attachment_link.title
Implicatiile GDPR in Tech & AI Cristina Iacobescu v1

# data[].attributes.attachment_link.readable_text
COM (2021) 206: Proposal for a REGULATION OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL LAYING DOWN HARMONISED RULES ON ARTIFICIAL INTELLIGENCE (ARTIFICIAL INTELLIGENCE ACT) AND AMENDING CERTAIN UNION LEGISLATIVE ACTS

Please enable javascript to display the timeline.

# data[].attributes.attachment_link.description
Do you want to help improving EUR-Lex ? This is a list of experimental features that you can enable. These features are still under development; they are not fully tested, and might reduce EUR-Lex stability. Don’t forget to give your feedback!

# data[].attributes.attachment_link.readable_text
Brussels, 28.9.2022

COM(2022) 496 final

2022/0303(COD)

Proposal for a

DIRECTIVE OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL

on adapting non-contractual civil liability rules to artificial intelligence
(AI Liability Directive)

(Text with EEA relevance)

{SEC(2022) 344 final} - {SWD(2022) 318 final} - {SWD(2022) 319 final} - {SWD(2022) 320 final}

EXPLANATORY MEMORANDUM

This explanatory memorandum accompanies the proposal for a Directive on adapting non-contractual civil liability rules to artificial intelligence (AI). In a representative survey of 2020 , liability ranked amongst the top three barriers to the use of AI by European companies . It wa s cited as the most relevant external obstacle (43%) for companies that are planning to, but have not yet adopted AI.

In her P olitical Guidelines , Commission President Ursula von der Leyen laid out a coordinated European approach on AI . In its White Paper on AI of 19 February 2020 , the Commission undertook to promote the uptake of AI and to address the risks associated with some o f its uses by fostering excellence and trust. In the Report on AI Liability accompanying the White Paper, the Commission identified the specific challenges posed by AI to existing liability rules. In its conclusions on shaping Europe's digital future of 9 June 2020, the Council welcomed the consultation on the policy proposals in the White Paper on AI and called on the Commission to put forward concrete proposals. On 20 October 2020, the European Parliament adopted a legislative own-initiative resolution under Article 225 TFEU requesting the Commission to adopt a proposal for a civil liability regime for AI based on Article 114 of the Treaty on the Functioning of the EU (TFEU) .

If a victim brings a claim, national courts, faced with the specific characteristics of AI, may adapt the way in which they apply existing rules on an ad hoc basis to come to a just result for the victim. This will cause legal uncertainty. Businesses will have difficulties to predict how the existing liability rules will be applied, and thus to assess and insure their liability exposure. The effect will be magnified for businesses trading across borders, as the uncertainty will cover different jurisdictions. It will particularly affect small and medium-sized enterprises (SMEs), which cannot rely on in-house legal expertise or capital reserves.

National AI strategies show that several Member States are considering, or even concretely planning, legislative action on civil liability for AI. Therefore, it is expected that, if the EU does not act, Member States will adapt their national liability rules to the challenges of AI. This will result in further fragmentation and increased costs for businesses active throughout the EU.

The open public consultation informing the Impact Assessment of this proposal, confirmed the problems explained above. In the opinion of the public, the 'black box' effect can make it difficult for the victim to prove fault and causality and there may be uncertainty as to how the courts will interpret and apply existing national liability rules in cases involving AI. Furthermore, it showed a public concern as to how legislative action on adapting liability rules initiated by individual Member States, and the ensuing fragmentation, would affect the costs for companies, especially SMEs, preventing the uptake of AI Union wide.

Thus, the objective of this proposal is to promote the rollout of trustworthy AI to harvest its full benefits for the internal market. It does so by ensuring victims of damage caused by AI obtain equivalent protection to victims of damage caused by products in general. It also reduces legal uncertainty of businesses developing or using AI regarding their possible exposure to liability and prevents the emergence of fragmented AI-specific adaptations of national civil liability rules.

This proposal is part of a package of measures to support the roll-out of AI in Europe by fostering excellence and trust. This package comprises three complementary work streams:

a legislative proposal laying down horizontal rules on artificial intelligence systems (AI Act);

In the AI Act proposal, the Commission has proposed rules that seek to reduce risks for safety and protect fundamental rights. Safety and liability are two sides of the same coin: they apply at different moments and reinforce each other. While rules to ensure safety and protect fundamental rights will reduce risks, they do not eliminate those risks entirely . Where such a risk materialises, damage may still occur. In such instances , the liability rules of this proposal will apply .

Effective liability rules also provide an economic incentive to comply with safety rules and therefore contribute to preventing the occurrence of damage . In addition, this proposal contributes to the enforcement of the requirements for high-risk AI systems imposed by the AI Act, because the failure to comply with those requirements constitutes an important element triggering the alleviations of the burden of proof. This proposal is also consistent with the general and sectoral product safety proposed rules applicable to AI-enabled machinery products and radio equipment.

The Commission takes a holistic approach in its AI policy to liability by proposing adaptations to the producer's liability for defective products under the Product Liability Directive as well as the targeted harmonisation under this proposal. These two policy initiatives are closely linked and form a package, as claims falling within their scope deal with different types of liability. The Product Liability Directive covers producer's no-fault liability for defective products, leading to compensation for certain types of damages, mainly suffered by individuals. This proposal covers national liability claims mainly based on the fault of any person with a view of compensating any type of damage and any type of victim. They complement one another to form an overall effective civil liability system.

Together these rules will promote trust in AI (and other digital technologies) by ensuring that victims are effectively compensated if damage occurs despite the preventive requirements of the AI Act and other safety rules.

The proposal is coherent with the Union 's overall digital strategy as it contributes to promoting technology that works for people, one of the three main pillars of the policy orientation and objectives announced in the Communication 'Shaping Europe's digital future' .

In this context, this proposal aims to build trust in the AI and to increase its uptake. This will achieve synergies and is complementary with the [Cyber Resilience Act] , which also aims to increase trust in products with digital elements by reducing cyber vulnerabilities and to better protect business and consumer users.

This proposal does not affect the rules set by [the Digital Services Act (DSA)], which provide for a comprehensive and fully harmonised framework for due diligence obligations for algorithmic decision making by online platforms, including its exemption of liability for providers of intermediary services.

In addition, by promoting the roll-out of AI, this proposal is linked to the initiatives under the EU strategy for data . It also strengthens the Union's role to help shape global norms and standards and promote trustworthy AI that is consistent with Union values and interests.

The proposal also has indirect links with the European Green Deal . In particular, digital technologies, including AI, are a critical enabler for attaining the sustainability goals of the Green D eal in many different sectors (including healthcare, transport, environment and farming) .

The Directive will contribute to the rollout of AI. The conditions for the roll-out and development of AI-technologies in the internal market can be significantly improved by preventing fragmentation and increasing legal certainty through harmonised measures at EU level, compared to possible adaptations of liability rules at national level. The economic study underpinning the Impact Assessment of this proposal concluded - as a conservative estimate - that targeted harmonisation measures on civil liability for AI would have a positive impact of 5 to 7 % on the production value of relevant cross-border trade as compared to the baseline scenario. This added value would be generated notably through reduced fragmentation and increased legal certainty regarding stakeholders' liability exposure. This would lower stakeholders' legal information/representation, internal risk management and compliance costs, facilitate financial planning as well as risk estimates for insurance purposes, and enable companies - in particular SMEs - to explore new markets across borders. Based on the overall value of the EU AI market affected by the liability-related problems addressed by this Directive, it is estimated that the latter will generate an additional market value between ca. EUR 500mln and ca. EUR 1.1bln.

In terms of social impacts, the Directive will increase societal trust in AI-technologies and access to an effective justice system. It will contribute to an efficient civil liability regime, adapted to the specificities of AI, where justified claims for compensation of damage are successful. Increasing societal trust would also benefit all companies in the AI-value chain, because strengthening citizens' confidence will contribute to a faster uptake of AI. Due to the incentivising effect of liability rules, preventing liability gaps would also indirectly benefit all citizens through an increased level of protection of health and safety (Article 114(3) TFEU) and the obviation of sources of health risks (Article 168(1) TFEU).

As regards environmental impacts, the Directive is also expected to contribute to achieving the related Sustainable Development Goals (SDGs) and targets. The uptake of AI applications is beneficial for the environment. For instance, AI systems used in process optimisation make processes less wasteful (e.g. by reducing the amount of fertilizers and pesticides needed, decreasing the water consumption at equal output, etc.). The Directive would also impact positively on SDGs because effective legislation on transparency, accountability and fundamental rights will direct AI's potential to benefit individuals and society towards achieving the SDGs.

The legal basis for the proposal is Article 114 TFEU, which provides for the adoption of measures to ensure the establishment and functioning of the internal market.

The problems this proposal aims to address, in particular legal uncertainty and legal fragmentation, hinder the development of the internal market and thus amount to significant obstacles to cross-border trade in AI-enabled products and services.

The proposal addresses obstacles stemming from the fact that businesses that want to produce, disseminate and operate AI-enabled products and services across borders are uncertain whether and how existing liability regimes apply to damage caused by AI. This uncertainty concerns particularly Member States where businesses will export to or operate their products and services. In a cross-border context, the law applicable to a non-contractual liability arising out of a tort or delict is by default the law of the country in which the damage occurs. For these businesses, it is essential to know the relevant liability risks and to be able to insure themselves against them.

In addition, there are concrete signs that a number of Member States are considering unilateral legislative measures to address the specific challenges posed by AI with respect to liability. For example, AI strategies adopted in Czech ia , Italy , Malta , Poland and Portugal mention initiatives to clarify liability. Given the large divergence between Member States' existing civil liability rules, it is likely that any national AI-specific measure on liability would follow existing different national approaches and therefore increase fragmentation .

Therefore, adaptations of liability rules taken on a purely national basis would increase the barriers to the rollout of AI-enabled products and services across the internal market and contribute further to fragmentation.

The objectives of this proposal cannot be adequately achieved at national level because emerging divergent national rules would increase legal uncertainty and fragmentation, creating obstacles to the rollout of AI-enabled products and services across the internal market. Legal uncertainty would particularly affect companies active cross-borders by imposing the need for additional legal information/representation, risk management costs and foregone revenue. At the same time, differing national rules on compensation claims for damage caused by AI would increase transaction costs for businesses, especially for cross-border trade, entailing significant internal market barriers. Further, legal uncertainty and fragmentation disproportionately affect start-ups and SMEs, which account for most companies and the major share of investments in the relevant markets.

In the absence of EU harmonised rules for compensating damage caused by AI systems, providers, operators and users of AI systems on the one hand and injured persons on the other hand would be faced with 27 different liability regimes, leading to different levels of protection and distorted competition among businesses from different Member States.

Harmonised measures at EU level would significantly improve conditions for the rollout and development of AI-technologies in the internal market by preventing fragmentation and increasing legal certainty. This added value would be generated notably through reduced fragmentation and increased legal certainty regarding stakeholders' liability exposure. Moreover, only EU action can consistently achieve the desired effect of promoting consumer trust in AI-enabled products and services by preventing liability gaps linked to the specific characteristics of AI across the internal market. This would ensure a consistent (minimum) level of protection for all victims (individuals and companies) and consistent incentives to prevent damage and ensure accountability.

The proposal is based on a staged approach. In the first stage, the objectives are achieved with a minimally invasive approach; the second stage involves re-assessing the need for more stringent or extensive measures.

The first stage is limited to the burden - of - proof measures to address the AI-specific problems identified. It builds on the substantive conditions of liability currently existing in national rules , such as causality or fault , but focuses on targeted proof-related measures , ensuring that victims have the same level of protection as in cases not involving AI systems . Moreover, from the various tools available in national law for easing the burden of proof , this proposal has chosen to use rebuttable presumptions as the least interventionist tool. Such presumptions are commonly found in national liability systems, and they balance the interests of claimants and defendants . At the same time they are designed to incentivise compliance with existing duties of care set at Union or national level . T he proposal does not lead to a reversal of the burden of proof, to avoid expos ing providers, operators and users of AI systems to higher liability risks , which may hamper inn ovation and reduce the uptake of AI-enabled products and services .

The second stage included in the proposal ensures that, when assessing the effect of the first stage in terms of victim protection and uptake of AI, future technological, regulatory and jurisprudential developments will be taken into account when re-assessing the need to harmonise other elements of the claims for compensation or other tools related to liability claims, including for situations where strict liability would be more appropriate, as requested by the European Parliament. Such assessment would also likely consider whether such a harmonisation would need to be coupled with mandatory insurance to ensure effectiveness.

A directive is the most suitable instrument for this proposal, as it provides the desired harmonisation effect and legal certainty, while also providing the flexibility to enable Member States to embed the harmonised measures without friction into their national liability regimes.

A mandatory instrument would prevent protection gaps stemming from partial or no implementation. While a non-binding instrument would be less intrusive, it is unlikely to address the identified problems in an effective manner. The implementation rate of non-binding instruments is difficult to predict and there is insufficient indication that the persuasive effect of a recommendation would be strong enough to produce consistent adaptation of national laws.

This effect is even more unlikely for measures concerning private law, of which non contractual liability rules form part. This area is characterised by long-standing legal traditions, which makes Member States reluctant to pursue coordinated reform unless driven by the clear prospect of internal market benefits under a binding EU instrument or the need to adapt to new technologies in the digital economy.

The existing significant divergences between Member States' liability frameworks are another reason why a recommendation is unlikely to be implemented in a consistent manner.

An extensive consultation strategy was implemented to ensure a wide participation of stakeholders throughout the policy cycle of this proposal. The consultation strategy was based on both public and several targeted consultations (webinars, bilateral discussions with companies and various organisations).

After the initial questions on liability which were part of the public consultation on the White Paper on AI and the Commission report on safety and liability, a dedicated online public consultation was open from 18 October 2021 to 10 January 2022 to gather views from a wide variety of stakeholders, including consumers, civil society organisation s , industry associations , businesses , including SMEs, and public authorities. After analysing all the responses received, the Commission published a summary outcome and the individual resp onses on its website .

In total, 233 responses were received from respondents from 21 Member States, as well as from third countries. Overall, the majority of stakeholders confirmed the problems with burden of proof, legal uncertainty and fragmentation and supported action at EU level.

EU citizens, consumer organizations and academic institutions overwhelmingly confirmed the need for EU action to ease victims' problems with the burden of proof. Businesses, while recognising the negative effects of the uncertainty around the application of liability rules, were more cautious and asked for targeted measures to avoid limiting innovation.

A similar picture appeared regarding the policy options. EU citizens, consumer organizations and academic institutions strongly supported measures on the burden of proof and harmonising no-fault liability (referred to as 'strict liability') coupled with mandatory insurance. Businesses were more divided on the policy options, with differences depending in part on their size. Strict liability was considered disproportionate by the majority of business respondents. Harmonisation of the easing of the burden of proof gained more support, particularly among SMEs. However, businesses cautioned against a complete shift of the burden of proof.

Therefore, the preferred policy option was developed and refined in light of feedback received from stakeholders throughout the impact assessment process to strike a balance between the needs expressed and concerns raised by all relevant stakeholder groups.

The proposal builds on 4 years of analysis and close involvement of stakeholders, including academics, businesses, consumer associations, Member States and citizens. The preparatory work started in 2018 with the setting up of the Expert Group on Liability and New Technologies (New Technologies Formation). The Expert Group produced a Report in November 2019 that assessed the challenges some characteristics of AI pose to national civil liability rules.

The input from the Expert Group report was complemented by three additional external studies:

a comparative law study based on a comparative legal analysis of European tort laws focused on key AI-related issues ;

a behavioural economics study on the impacts of targeted adaptations of the liability regime on consumers' decision making, in particular their trust and willingness to take up AI-enabled products and services ;

an economic study covering a number of issues: the challenges faced by victims of AI applications compared to victims of non-AI devices when trying to obtain compensation for their loss; whether and to what extent businesses are uncertain about the application of current liability rules to their operations involving AI, and whether the impact of legal uncertainty can hamper investment in AI; whether further fragmentation of national liability laws would reduce the effectiveness of the internal market for AI applications and services, and whether and to what extent harmonising certain aspects of national civil liability via EU legislation would reduce these problems and facilitate the overall uptake of AI technology by EU companies.

In line with its "Better Regulation" policy, the Commission conducted an impact assessment for this proposal examined by the Commission's Regulatory Scrutiny Board. The meeting of the Regulatory Scrutiny Board on 6 April 2022 led to a positive opinion with comments.

Three policy options were assessed:

Policy option 1 : three measures to ease the burden of proof for victims trying to prove their liability claim.

Policy option 2 : the measures under option 1 + harmonising strict liability rules for AI use cases with a particular risk profile, coupled with a mandatory insurance.

Policy option 3 : a staged approach consisting of:

The policy options were compared by way of a multi-criteria analysis taking into account their effectiveness, efficiency, coherence and proportionality. The results of the multi-criteria and sensitivity analysis show that policy option 3, easing the burden of proof for AI-related claims + targeted review regarding strict liability, possibly coupled with mandatory insurance, ranks highest and is therefore the preferred policy choice for this proposal.

The preferred policy option would ensure that victims of AI-enabled products and services (natural persons, businesses and any other public or private entities) are no less protected than victims of traditional technologies. It would increase the level of trust in AI and promote its uptake.

Furthermore, it would reduce legal uncertainty and prevent fragmentation, thus helping companies, and most of all SMEs, that want to realise the full potential of the EU single market by rolling out AI-enabled products and services cross-border. The preferred policy option also creates better conditions for insurers to offer coverage of AI-related activities, which is crucial for businesses, especially SMEs to manage their risks. It is namely estimated that the preferred policy option would generate an increased AI market value in the EU-27 between ca. EUR 500mln and ca. EUR 1.1bln in 2025.

One of the most important functions of civil liability rules is to ensure that victims of damage can claim compensation. By guaranteeing effective compensation, these rules contribute to the protection of the right to an effective remedy and a fair trial (Article 47 of the EU Charter of Fundamental Rights, referred to below as 'the Charter') while also giving potentially liable persons an incentive to prevent damage, in order to avoid liability.

With this proposal, the Commission aims to ensure that victims of damage caused by AI have an equivalent level of protection under civil liability rules as victims of damage caused without the involvement of AI. The proposal will enable effective private enforcement of fundamental rights and preserve the right to an effective remedy where AI-specific risks have materialised. In particular, the proposal will help protect fundamental rights, such as the right to life (Article 2 of the Charter), the right to the physical and mental integrity (Article 3), and the right to property (Article 17). In addition, depending on each Member State's civil law system and traditions, victims will be able to claim compensation for damage to other legal interests, such as violations of personal dignity (Articles 1 and 4 of the Charter), respect for private and family life (Article 7), the right to equality (Article 20) and non-discrimination (Article 21).

In addition, this proposal complements other strands in the Commission's AI policy based on preventive regulatory and supervisory requirements aimed directly at avoiding fundamental rights breaches (such as discrimination). These are the AI Act, the General Data Protection Regulation, the Digital Services Act and EU law on non-discrimination and equal treatment. At the same time, this proposal does not create or harmonise the duties of care or the liability of various entities whose activity is regulated under those legal acts and, therefore, does not create new liability claims or affect the exemptions from liability under those other legal acts. This proposal only introduces alleviations of the burden of proof for the victims of damage caused by AI systems in claims that can be based on national law or on these other EU laws. By complementing these other strands, this proposal protects the victim's right to compensation under private law, including compensation for fundamental rights breaches.

This proposal will not have implications for the budget of the European Union.

This proposal puts forward a staged approach. To ensure that sufficient evidence is available for the targeted review in the second stage, the Commission will draw up a monitoring plan, detailing how and how often data and other necessary evidence will be collected.

The monitoring mechanism could cover the following types of data and evidence:

The purpose of this Directive is to improve the functioning of the internal market by laying down uniform requirements for certain aspects of non-contractual civil liability for damage caused with the involvement of AI systems. It follows up on the European Parliament's Resolution 2020/2014(INL) and adapts private law to the needs of the transition to the digital economy.

In addition, when asked about more far-reaching changes such as a reversal of the burden of proof or an irrebuttable presumption, businesses provided negative feedback in consultations. Targeted measures to ease the burden of proof in form of rebuttable presumptions were chosen as pragmatic and appropriate ways to help victims meet their burden of proof in the most targeted and proportionate manner possible.

In addition, this Directive does not affect existing rules regulating the conditions of liability in the transport sector and those set by the Digital Services Act.

While this Directive does not apply with respect to criminal liability, it may be applicable with respect to state liability. State authorities are also covered by the provisions of the AI Act as subjects of the obligations prescribed therein.

This Directive does not apply retroactively, but only to claims for compensation of damages that occur as from the date of its transposition.

The proposal for this Directive has been adopted together with the proposal for a revision of the Product Liability Directive 85/374/EEC, in a package aiming to adapt liability rules to the digital age and AI, ensuring the necessary alignment between these two complementary legal instruments.

The definitions in Article 2 follow those of the AI Act to ensure consistency.

Article 2(6)(b) provides that claims for damages can be brought not only by the injured person but also by persons that have succeeded in or have been subrogated into the injured person's rights. Subrogation is the assumption by a third party (such as an insurance company) of another party's legal right to collect a debt or damages. Thus one person is entitled to enforce the rights of another for their own benefit. Subrogation would also cover heirs of a deceased victim.

In addition, Article 2(6)(c) provides that an action for damages can also be brought by someone acting on behalf of one or more injured parties, in accordance with Union or national law. This provision aims to give more possibilities to persons injured by an AI system to have their claims assessed by a court, even in cases where individual actions may seem too costly or too cumbersome to bring, or where joint actions may entail a benefit of scale. To enable victims of damage caused by AI systems to enforce their rights in relation to this Directive through representative actions, Article 6 amends Annex I to Directive (EU) 2020/1828.

This Directive aims to provide persons seeking compensation for damage caused by high-risk AI systems with effective means to identify potentially liable persons and relevant evidence for a claim. At the same time, such means serve to exclude falsely identified potential defendants, saving time and costs for the parties involved and reducing the case load for courts.

In this respect, Article 3(1) of the Directive provides that a court may order the disclosure of relevant evidence about specific high-risk AI systems that are suspected of having caused damage. Requests for evidence are addressed to the provider of an AI system, a person who is subject to the provider's obligations laid down by Article 24 or Article 28 (1) of the AI Act or a user pursuant to the AI Act. The requests should be supported by facts and evidence sufficient to establish the plausibility of the contemplated claim for damages and the requested evidence should be at the addressees' disposal. Requests cannot be addressed to parties that bear no obligations under the AI Act and therefore have no access to the evidence.

According to Article 3(2) the claimant can request the disclosure of evidence from providers or users that are not defendants only in case all proportionate attempts were unsuccessfully made to gather the evidence from the defendant.

In order for the judicial means to be effective, Article 3(3) of the Directive provides that a court may also order the preservation of such evidence.

As provided in Article 3(4), first subparagraph, the court may order such disclosure, only to the extent necessary to sustain the claim, given that the information could be critical evidence to the injured person's claim in the case of damage that involve AI systems.

By limiting the obligation to disclose or preserve to necessary and proportionate evidence, Article 3 (4), first subparagraph, aims to ensure proportionality in disclosing evidence, i.e. to limit the disclosure to the necessary minimum and prevent blanket requests.

The second and third subparagraphs of Article 3(4) further aim to strike a balance between the claimant's rights and the need to ensure that such disclosure would be subject to safeguards to protect the legitimate interests of all parties concerned, such as trade secrets or confidential information.

In the same context, the fourth subparagraph of Article 3(4) aims to ensure that procedural remedies against the order of disclosure or preservation are at the disposal of the person subject to it.

Article 3(5) introduces a presumption of non-compliance with a duty of care. This is a procedural tool, relevant only in cases where it is the actual defendant in a claim for damages who bears the consequences of not complying with a request to disclose or preserve evidence. The defendant will have the right to rebut that presumption. The measure set out in this paragraph aims to promote disclosure but also to expedite court proceedings.

With respect to damage caused by AI systems, this Directive aims to provide an effective basis for claiming compensation in connection with the fault consisting in the lack of compliance with a duty of care under Union or national law.

It can be challenging for claimants to establish a causal link between such non-compliance and the output produced by the AI system or the failure of the AI system to produce an output that gave rise to the relevant damage. Therefore, a targeted rebuttable presumption of causality has been laid down in Article 4 (1) regarding this causal link. Such presumption is the least burdensome measure to address the need for fair compensation of the victim.

The fault of the defendant has to be proven by the claimant according to the applicable Union or national rules. Such fault can be established, for example, for non-compliance with a duty of care pursuant to the AI Act or pursuant to other rules set at Union level, such as those regulating the use of automated monitoring and decision-making for platform work or those regulating the operation of unmanned aircraft. Such fault can also be presumed by the court on the basis of a non-compliance with a court order for disclosure or preservation of evidence under Article 3(5). Still, it is only appropriate to introduce a presumption of causality when it can be considered likely that the given fault has influenced the relevant AI system output or lack thereof, which can be assessed on the basis of the overall circumstances of the case. At the same time, the claimant still has to prove that the AI system (i.e. its output or failure to produce one) gave rise to the damage.

Paragraphs (2) and (3) differentiate between, on the one hand, claims brought against the provider of a high-risk AI system or against a person subject to the provider's obligations under the AI Act and, on the other hand, claims brought against the user of such systems. In this respect, it follows the respective provisions and relevant conditions of the AI Act. In the case of claims based on Article 4(2), the defendants' compliance with the obligations listed in that paragraph have to be assessed also in the light of the risk management system and its results, i.e. risk management measures, under the AI Act.

In case of high-risk AI systems as defined by the AI Act, Article 4(4) establishes an exception from the presumption of causality, where the defendant demonstrates that sufficient evidence and expertise is reasonably accessible for the claimant to prove the causal link. This possibility can incentivise defendants to comply with their disclosure obligations, with measures set by the AI Act to ensure a high level of transparency of the AI or with documenting and recording requirements.

In the case of non-high risk AI systems, Article 4(5) establishes a condition for the applicability of the presumption of causality, whereby the latter is subject to the court determining that it is excessively difficult for the claimant to prove the causal link. Such difficulties are to be assessed in light of the characteristics of certain AI systems, such as autonomy and opacity, which render the explanation of the inner functioning of the AI system very difficult in practice, negatively affecting the ability of the claimant to prove the causal link between the fault of the defendant and the AI output.

In cases where the defendant uses the AI system in the course of a personal non-professional activity, Article 4(6) provides that the presumption of causality should only apply if the defendant has materially interfered with the conditions of the operation of the AI system or if the defendant was required and able to determine the conditions of operation of the AI system and failed to do so. This condition is justified by the need to balance the interests of injured persons and non-professional users, by exempting from the application of the presumption of causality the cases in which non-professional users do not add risk through their behaviour.

Finally, Article 4(7) provides that the defendant has the right to rebut the causality presumption based on Article 4(1).

Such effective civil liability rules have the additional advantage that they give all those involved in activities related to AI systems an additional incentive to respect their obligations regarding their expected conduct.

Various national legal systems provide for different strict liability regimes. Elements for such a regime at Union level were also suggested by the European Parliament in its own-initiative resolution of 20 October 2020, consisting of a limited strict liability regime for certain AI-enabled technologies and a facilitated burden of proof under fault-based liability rules. The public consultations also highlighted a preference for such a regime among respondents (except for non-SMEs businesses), whether or not coupled with mandatory insurance.

However, the proposal takes into account the differences between national legal traditions and the fact that the kind of products and services equipped with AI systems that could affect the public at large and put at risk important legal rights, such as the right to life, health and property, and therefore could be subject to a strict liability regime, are not yet widely available on the market.

A monitoring programme is put in place to provide the Commission with information on incidents involving AI systems. The targeted review will assess whether additional measures would be needed, such as introducing a strict liability regime and/or mandatory insurance.

When notifying the Commission of national transposition measures to comply with this Directive, Member States should also provide explanatory documents which give sufficiently clear and precise information and state, for each provision of this Directive, the national provision(s) ensuring its transposition. This is necessary to enable the Commission to identify, for each provision of the Directive requiring transposition, the relevant part of national transposition measures creating the corresponding legal obligation in the national legal order, whatever the form chosen by the Member States.

2022/0303 (COD)

Proposal for a

DIRECTIVE OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL

on adapting non-contractual civil liability rules to artificial intelligence
(AI Liability Directive)

(Text with EEA relevance)

THE EUROPEAN PARLIAMENT AND THE COUNCIL OF THE EUROPEAN UNION,

Having regard to the Treaty on the Functioning of the European Union, and in particular Article 114 thereof,

Having regard to the proposal from the European Commission,

After transmission of the draft legislative act to the national parliaments,

Having regard to the opinion of the European Economic and Social Committee ,

Having regard to the opinion of the Committee of the Regions ,

Acting in accordance with the ordinary legislative procedure,

Artificial Intelligence ('AI') is a set of enabling technolog ies which can contribute to a wide array of benefits across the entire spectrum of the economy and society. It has a large potential for technological progress and allows new business models in many sectors of the digital economy.

In such cases, the level of redress afforded by national civil liability rules may be lower than in cases where technologies other than AI are involved in causing damage . Such compensation gaps may contribute to a lower level of societal acceptance of AI and trust in AI-enabled products and services.

The laws of the Member States concerning the liability of producers for damage caused by the defectiveness of their products are already harmonised at Union level by Council Directive 85/374/EEC . Those laws do not, however, affect Member States' rules of contractual or non-contractual liability , such as warranty, fault or strict liability, based on other grounds than the defect of the product . While at the same time the revision of Council Directive 85/374/EEC seeks to clarify and ensure that injured person can claim compensation for damages caused by defective AI-enabled products, i t should therefore be clarified that the provisions of this Directive do not affect any rights which an injured person may have under national rules implementing Directive 85/374/EEC. In addition, in the field of transport, U nion law regulating the liability of transport operators should remain unaffected by this Directive.

[ The Digital Services Act (DSA) ] fully harmonises the rules applicable to providers of intermediary services in the internal market, covering the societal risks stemming from the services offered by those providers, including as regards the AI systems they use. This Directive does not affect the provisions of [the Digital Services Act (DSA)] that provide a comprehensive and fully harmonised framework for due diligence obligations for algorithmic decision-making by hosting service providers, including the exemption from liability for the disse mination of illegal content uploaded by recipients of their services where the conditions of that Regulation are met .

National courts should be able, in the course of civil proceedings, to order the disclosure or preservation of relevant evidence related to the damage caused by high-risk AI systems from persons who are already under an obligation to document or record information pursuant to [the AI Act], be they providers, persons under the same obligations as providers, or users of an AI system, either as defendants or third parties to the claim. There could be situations where the evidence relevant for the case is held by entities that would not be parties to the claim for damages but which are under an obligation to document or record such evidence pursuant to [the AI Act]. It is thus necessary to provide for the conditions under which such third parties to the claim can be ordered to disclose the relevant evidence .

To maintain the balance between the interests of the parties involved in the claim for damages and of third parties concerned, the courts should order the disclosure of evidence only where this is necessary and proportionate for supporting the claim or potential claim for damages. In this respect, disclosure should only concern evidence that is necessary for a decision on the respective claim for damages, for example only the parts of the relevant records or data sets required to prove non-compliance with a requirement laid down by [the AI Act] . To ensure the proportionality of such disclosure or preservation measures, national courts should have effective means to safeguard the legitimate interests of all parties involved, for instance the protection of trade secrets within the meaning of Directive (EU) 2016/943 of the European Parliament and of the Council and of confidential information , such as information related to public or national security . In respect of trade secrets or alleged trade secrets which the court has identified as confidential within the meaning of Directive (EU) 2016/943, national courts should be empowered to take specific measures to ensure the confidentiality of trade secrets during and after the proceedings, while achieving a fair and proportionate balance between the trade-secret holder's interest in maintaining secrecy and the interest of the injured person . This should include measures to restrict access to documents containing trade secrets and access to hearings or documents and transcripts thereof to a limited number of people. When deciding on such measures, national courts should take into account the need to ensure the right to an effective remedy and to a fair trial, the legitimate interests of the parties and, where appropriate, of third parties, and any potential harm to either party or, where appropriate, to third parties, resulting from the granting or rejection of such measures. Moreover, to ensure a proportionate application of a disclosure measure towards third parties in claims for damages, the national courts should order disclosure from third parties only if the evidence cannot be obtained from the defendant.

It is necessary to provide for a review of this Directive [five years] after the end of the transposition period. In particular, that review should examine whether there is a need to create no-fault liability rules for claims against the operator, as long as not already covered by other Union liability rules in particular Directive 85/374/EEC, combined with a mandatory insurance for the operation of certain AI systems, as suggested by the European Parliament. In accordance with the principle of proportionality, it is appropriate to assess such a need in the light of relevant technological and regulatory developments in the coming years , taking into account the effect and impact on the roll-out and uptake of AI systems, especially for SMEs . Such a review should consider , among others, risks involving damage to important legal values like life, health and property of unwitting third parties through the operation of AI-enabled products or services. That review should also analyse the effectiveness of the measures provided for in this Directive in dealing with such risks, as well as the development of appropriate solutions by the insurance market. To ensure the availability of the information necessary to conduct such a review, it is necessary to collect data and other necessary evidence covering the relevant matters.

Given the need to make adaptations to national civil liability and procedural rules to foster the rolling - out of AI-enabled products and services under beneficial internal market conditions, societal acceptance and consumer trust in AI technology and the justice system, it is appropriate to set a deadline of not later than [two years after the entry into force] of this Directive for Member States to adopt the necessary transposition measures.

In accordance with the Joint Political Declaration of 28 September 2011 of Member States and the Commission on explanatory documents , Member States have undertaken to accompany, in justified cases, the notification of their transposition measures with one or more documents explaining the relationship between the components of a directive and the corresponding parts of national transposition instruments. With regard to this Directive, the legislator considers the transmission of such documents to be justified ,

HAVE ADOPTED THIS DIRECTIVE:

Subject matter and scope

This Directive does not apply to criminal liability.

Definitions

For the purposes of this Directive, the following definitions shall apply:

'claim for damages' means a non-contractual fault-based civil law claim for compensation of the damage caused by an output of an AI system or the failure of such a system to produce an output whe re such an output should have been produced;

has been injured by an output of an AI system or by the failure of such a system to produce an output whe re such an output should have been produced ;

Disclosure of evidence and rebuttable presumption of non-compliance

In support of that request, the potential claimant must present facts and evidence sufficient to support the plausibility of a claim for damages

In the context of a claim for damages, the national court shall only order the disclosure of the evidence by one of the persons listed in paragraph 1, if the claimant has undertaken all proportionate attempts at gathering the relevant evidence from the defendant.

In determining whether an order for the disclosure or preservation of evidence is proportionate, national courts shall consider the legitimate interests of all parties, including third parties concerned, in particular in relation to the protection of trade secrets within the meaning of Article 2(1) of Directive (EU) 2016/943 and of confidential information, such as information related to public or national security.

Member States shall ensure that, where the disclosure of a trade secret or alleged trade secret which the court has identified as confidential within the meaning of Article 9(1) of Directive (EU) 2016/943 is ordered, national courts are empowered, upon a duly reasoned request of a party or on their own initiative, to take specific measures necessary to preserve confidentiality when that evidence is used or referred to in legal proceedings.

Member States shall also ensure that the person ordered to disclose or to preserve the evidence mentioned in paragraphs 1 or 2 has appropriate procedural remedies in response to such orders.

Where a defendant fails to comply with an order by a national court in a claim for damages to disclose or to preserve evidence at its disposal pursuant to paragraphs 1 or 2, a national court shall presume the defendant's non-compliance with a relevant duty of care , in particular in the circumstances referred to in Article 4(2) or (3), that the evidence requested was intended to prove for the purposes of the relevant claim for damages.

The defendant shall have the right to rebut that presumption.

Rebuttable presumption of a causal link in the case of fault

the AI system is a system which makes use of techniques involving the training of models with data and which was not developed on the basis of training, validation and testing data sets that meet the quality criteria referred to in [Article 10(2) to (4) of the AI Act];

Evaluation and targeted review

Amendment to Directive (EU) 2020/1828

"(67) Directive (EU) .../... of the European Parliament and of the Council of ... on adapting non contractual civil liability rules to artificial intelligence (AI Liability Directive) (OJ L ..., ..., p. ...).".

Transposition

When Member States adopt those provisions, they shall contain a reference to this Directive or be accompanied by such a reference on the occasion of their official publication. Member States shall determine how such reference is to be made.

Entry into force

Addressees

This Directive is addressed to the Member States.

Done at Brussels,

For the European Parliament For the Council

The President The President

# data[].attributes.attachment_link.description
Brussels, 28.9.2022 COM(2022) 496 final 2022/0303(COD) Proposal for a DIRECTIVE OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL on adapting non-contractual civil liability rules to artificial intelligence (AI Liability Directive) (Text with EEA relevance) {SEC(2022) 344 final} - {SWD(2022) 318 final} - {SWD(2022) 319 final} - {SWD(2022) 320 final} EXPLANATORY MEMORANDUM This explanatory memorandum accompanies the proposal for a Directive on adapting non-contractual civil liability rules to artificial intelligence (AI).

# data[].attributes.attachment_caption
The OECD Artificial Intelligence (AI) Principles - OECD.AI

# data[].attributes.attachment_link.title
The OECD Artificial Intelligence (AI) Principles - OECD.AI

# data[].attributes.attachment_link.readable_text
Creation of an Artificial Intelligence Department (AID)

Five CNIL's agents will work in the Artificial Intelligence Department, including legal experts and specialized engineers. This department will be part of CNIL's Technology and Innovation Directorate, whose director, Bertrand PAILHES, is a former National coordinator for the Artificial Intelligence strategy.

The main missions of the Artificial Intelligence Department will be:

improve the understanding of AI systems for the CNIL, professionals and individuals;

strengthen CNIL's expertise for the identification and prevention of privacy risks related to the implementation of these systems;

prepare for the implementation of the European regulation on AI (under discussion at the European level);

develop relationships with the ecosystem (academia, start-ups, companies, etc.).

A transversal role within the CNIL

The AID will closely collaborate with the directorate in charge of legal advice, led by Thomas DAUTIEU, to issue "soft law" documentation (guidelines, recommendations, etc.) and to address requests for advice submitted by the government. This directorate may also request the assistance of AID to advise public or private actors on projects involving the use of AI systems of particular complexity.

More generally, , this new department is intended to work with all CNIL departments. The AID will also support the investigation of claims and the adoption of corrective measures in case of breaches related to the use of an AI system.

The AID will also support the investigation of claims and the adoption of corrective measures in case of breaches related to the use of an AI system.

It will be in charge of the technical expertise related to artificial intelligence with specific aspects to this technology. It will contribute to the work of the European Data Protection Board (EDPB) and will also conduct experimentation projects in connection with the LINC.

This evolution of the CNIL notably echoes a study by the French Council of State published on August 30, 2022 on the use of artificial intelligence in administrations. The Council of State recommends strengthening the CNIL's resources and changing its role so that it also becomes one of the national supervisory authorities responsible for regulating AI systems. It also highlights the importance for the CNIL to play the role of coordinating and supervising authority, according to the provisions of the future European regulation, and thus to ensure the networking of public institutions ranging, from market surveillance authorities to sectoral regulators.

As of now, and notwithstanding future legislative evolutions, the creation of this department responds to a societal challenge that is growing in importance every day and is in line with the work on artificial intelligence initiated long ago by the CNIL.

The answer to a society-wide issue

Artificial intelligence is making spectacular technological progress, and is about to assist humans in a wide range of tasks. In the private sector, AI is seen as a factor for performance and profitability, while its use in administrations could improve the quality of public service provided to the user.

The use of this technology regards all kind of actors, regardless of their field of activity or size. By ricochet, every person is necessarily directly or indirectly confronted to AI, for example, when using a voice assistant, in a selection process for job applications or in the fight against tax fraud.

If the benefits linked to the use of algorithms and AI are undeniable, their operation relies on the processing of a large amount of data, very often personal data, and their implementation thus entails risks for privacy.

In addition, AI raises issues related to the information for users as well as the understanding of the way these systems operate, but also issues for data processors and regulators in the context of their audit and control missions.

The creation of the AI department responds to these challenges by providing a new tool to the regulator to organize transparency and understanding of a technology that is often perceived as a "black box" in order to ensure balanced regulation and to enable organisations, individuals and the CNIL to control the risks to privacy.

A dynamic initiated in 2017 and continued today

The CNIL has been working for several years on the subject of artificial intelligence, which constitutes a new generation of algorithms or, in the sense of the General Data Protection Regulation, a "data processing". To date, its action has taken three forms:

the identification of ethical and legal issues related to the use of AI;

the instruction of cases involving the use of AI and the issuance of tools and resources improving the understanding of this technology;

and the management of associated risks.

The creation of an AI department thus structures and consolidates a long-standing dynamic.

Identification of ethical and legal issues

In 2017, the CNIL organised a broad public debate on algorithms and AI that was complemented by a citizen consultation. This was conducted in conjunction with several research institutes (CREOGN, CNAM, ENSC, INSA), public institutions (ministries and universities), professional federations and companies. It resulted in the production of a summary report containing recommendations and highlighting the major issues raised by the use of AI systems when processing personal data (How can Humans keep the upper hand ? The ethical matters raised by algorithms and artificial intelligence).

Field investigation of cases involving the use of AI

The CNIL has already and for many years had the opportunity to work regularly on the subject of artificial intelligence through cases of operational implementation (the so-called "intelligent" or "augmented" cameras in public spaces, the white paper on voice assistants or the work related to the deployment of facial recognition that implements AI processing). In addition to systems that may be used in the private sector, the CNIL has also given its opinion on projects on which the government has requested its opinion, for example, the project of targeting fraud and valuing requests (CFVR) which aims to improve tax control operations.

The publication of tools and resources to improve the understanding of AI and the management of associated risks

The CNIL published on April 5, 2022 a web section that clarifies the challenges posed by artificial intelligence and the conditions under which to deploy AI systems that respect the key principles of the GDPR. The content published is aimed at the general public and professionals, who are offered a self-assessment guide for artificial intelligence systems. In addition, content intended for a more informed public is available on the CNIL's Digital Innovation Laboratory (LINC) website. In particular, it describes the new security risks posed by AI systems.

CNIL launches work on machine learning databases today

AI systems, especially those based on machine learning, very often require the use of large volumes of personal data to train the algorithms. The CNIL publishes a work program to support professionals and guarantee the respect of people's rights.

What are the objectives of this project?

Many public or private organizations wishing to build databases for the training and development of AI have expressed to the CNIL questions about the legality of certain uses.

The purpose of the upcoming work is therefore to clarify the CNIL's position on this point and to promote good practices, under the requirements of GDPR, but also in view of the proposed regulation on AI currently debated at the European level.

The provision of a framework for analysing the regulations on the protection of personal data and of concrete answers should enable organizations to create or use databases in compliance with the fundamental rights and freedoms of individuals.

The CNIL will regularly publish support tools on the subject that will complement its initial practical information sheets.

What is the scope?

The scope of the reflections initiated by the CNIL covers:

AI systems whose development or improvement requires the constitution of a database (machine learning systems and expert systems);

data collection from all types of sources (data collection from data subjects, data collection from data brokers, data collection from open sources, etc.);

the phases of the development of an AI system necessary for its production or improvement (design system, data pre-processing, training, continuous training, etc.). The production phase is therefore excluded from this project;

various uses relating to the development or improvement of an AI system (scientific research, research and development, improvement of a commercial product, etc.), regardless of the purpose of the AI system in production and the legal regime applicable to the processing (General Data Protection Regulation, Law Enforcement Directive and the French Data Protection Act).

On the other hand, issues related to the AI models learned from the collected data, and in particular those related to their dissemination and reuse, are excluded from this project and will be the subject of separate work.

What will be the publications at the end of this project?

CNIL's work will result in several publications during 2023 on:

tools to support the creation and use of databases: these discussions will be conducted in close collaboration with the CNIL working group on the opening and sharing of data announced during the air 2021 event;

practical sheets to respond to the most common situations encountered by learning database users (building a warehouse, using pseudonymized data, etc.).

What are the next steps?

On the basis of its existing expertise but also meetings with public and private actors on the constitution of databases for the development or improvement of an AI system, the CNIL will propose recommendations. They will be submitted to a public consultation allowing the greatest number of people to express their views on the tools thus developed. The contributions will be analysed at the end of the consultation in order to adjust and clarify its future tools, which it will publish on cnil.fr.

# data[].attributes.attachment_link.title
The CNIL creates an Artificial Intelligence Department and begins to work on learning databases

# data[].attributes.attachment_link.description
Creation of an Artificial Intelligence Department (AID) Five CNIL's agents will work in the Artificial Intelligence Department, including legal experts and specialized engineers. This department will be part of CNIL's Technology and Innovation Directorate, whose director, Bertrand PAILHES, is a former National coordinator for the Artificial Intelligence strategy.

# data[].attributes.attachment_link.readable_text
Face à la complexité et au volume croissants des cyberattaques, il est désormais évident que l'IA peut apporter de nouvelles réponses aux risques de sécurité. En effet, l'IA peut, avec sa capacité à analyser un contexte donné, contribuer à y détecter des anomalies ou des comportements inhabituels, révélateurs d'attaques et ainsi, à renforcer les outils de protection, détection, réponse et remédiation : augmentation du taux de détection, détection au plus tôt des attaques, amélioration de la capacité d'adaptation aux évolutions permanentes des systèmes d'information, etc. A titre d'exemple, de nombreuses sondes réseaux (IDS, intrusion detection systems) et déployées sur des terminaux ( EDR, endpoint detection and response) intègrent désormais des technologies d'IA, en complément de leur moteur de détection par signature permettant de reconnaître un programme malveillant. Ces systèmes apprennent ainsi le comportement " normal " de l'infrastructure et toute déviation observée par la suite est signalée à un analyste comme un potentiel risque cyber (d'autres exemples peuvent être trouvés ici).

Si les technologies d'IA peuvent améliorer la sécurité des systèmes d'information, elles engendrent également des risques spécifiques. En effet, le problème réside dans le fait qu'avec les nouvelles capacités de l'apprentissage automatique et dans la perspective de son utilisation de plus en plus large, sont introduites de très nombreuses vulnérabilités que des attaquants sont susceptibles d'exploiter. Certaines de ces vulnérabilités peuvent permettre de perturber le fonctionnement du modèle et l'amener à émettre une prédiction incorrecte. D'autres, en revanche, laissent un attaquant libre d'extraire des informations sensibles du modèle, telles que les données sous-jacentes ou le modèle lui-même.

Plusieurs travaux scientifiques recensent les différents types d'attaques comme par exemple ( Pitropakis et al., 2019). Par ailleurs, le laboratoire MITRE - une organisation à but non lucratif américaine dont l'objectif est de travailler pour l'intérêt public dans les domaines de l'ingénierie des systèmes, la technologie de l'information, les concepts opérationnels, et la modernisation des entreprises - a lancé l'initiative ATLAS ( Adversarial Threat Landscape for Artificial Intelligence Systems). Celle-ci est une base de connaissances des méthodes, techniques et études de cas d'attaques menées à l'encontre de systèmes d'apprentissage automatique, constituée à partir d'observations du monde réel, de démonstrations réalisées par des experts en sécurité ( pentesters, red teamers, etc.) et de connaissances issues de la recherche universitaire.

Les travaux sur le sujet de la sécurité des modèles d'IA ont en effet connu un intérêt croissant au cours des dernières années, ce qui est mesurable au nombre de publications scientifiques sur le sujet. La Figure 1 illustre cette tendance sur une sous-partie du domaine, les exemples adversaires introduits en 2014 par ( Goodfellow et al., 2014). En pratique, de la même façon que ce qui existe dans le domaine de la cryptographie depuis toujours, on observe une course permanente entre les méthodes d'attaque et de défense, aucune de ces dernières ne garantissant à ce jour la robustesse des systèmes dans 100% des cas.

Figure 1. Evolution du nombre d'articles dédiés au sujet des exemples adversaires
(sourceblog de Nicholas Carlini).

Au cours des dernières années, de grandes entreprises telles que Google, Amazon, Microsoft et Tesla ont vu certains de leurs systèmes d'IA attaqués. Cette tendance est amenée à s'accentuer. Selon un rapport de l'entreprise de conseil Gartner de 2019, 30% des cyberattaques d'ici 2022 impliqueront des vols de modèles ( model theft), l'utilisation d'exemples contradictoires ( aversarial examples) ou l'empoisonnement de données ( data poisoning). Ces attaques sont d'autant plus probables que les systèmes d'IA seront progressivement installés dans de nombreux environnements et a fortiori serviront de plus en plus de support à des décisions automatiques pouvant présenter un intérêt pour une personnelle malveillante. A titre d'exemple, un concurrent peut envisager tenter de " saboter " un logiciel d'IA utilisé pour le contrôle qualité dans un environnement industriel, un attaquant peut chercher à optimiser l'analyse d'une demande de crédit, etc. Dans son rapport d'avril 2021, la startup israélienne Adversa, spécialisée dans la sécurité des systèmes d'IA, détaille les applications les plus attaquées aujourd'hui (voir Figure 2).

Aujourd'hui, le monde de l'industrie apparaît mal préparé à faire face à de telles menaces comme l'indique une enquête menée en 2021 auprès de 28 organisations, petites et grandes, par ( Shankar et al., 2021) : vingt-cinq d'entre elles ne savaient pas comment sécuriser leurs systèmes d'IA. Pour ces raisons, il est essentiel de sensibiliser les organismes aux problématiques de sécurisation de leurs systèmes d'IA et de proposer des outils d'analyse de risque adaptés.

Il existe de nombreuses raisons pour lesquelles il est très difficile de protéger un système d'IA. Ces raisons tiennent aux limitations de ces derniers. Ainsi en raison de leur construction statistique, les modèles d'IA ne sont jamais parfaits. Ils demeurent donc sujets à erreur et exposés à des attaques. Ces comportements peuvent être le fait de données trop peu nombreuses ou déséquilibrées pour la phase d'apprentissage, de limitations des ressources informatiques pour la constitution d'architectures de systèmes toujours plus complexes, etc.

La plupart du temps, les modèles d'apprentissage automatique fonctionnent efficacement, mais seulement sur une portion limitée des données qu'ils sont susceptibles de traiter (en général ayant des caractéristiques très proches de celles utilisées pour l'entraînement des modèles). Ce constat est d'ailleurs l'objet d'une publication de personnels de Google ( D'Amour et al, 2020). Ainsi, dans le cas du traitement d'image ( computer vision), une très petite perturbation de chaque pixel d'entrée dans un espace de très grande dimension peut suffire à provoquer un changement radical des sorties fournies par le réseau de neurones. Intuitivement, il s'agit donc de déplacer l'image d'entrée vers un point de l'espace que les systèmes d'IA, comme par exemple les réseaux de neurones, n'ont jamais exploré auparavant. Les espaces à haute dimension utilisés sont en effet si peu denses que la plupart des données d'apprentissage sont concentrées dans une très petite région de l'espace connue sous le nom de " variété " ( manifold).

Les modèles d'IA peuvent donc avoir un comportement imprévisible et présenter une confiance excessive en dehors de la distribution d'apprentissage, ce qui laisse prise à d'éventuels attaquants. Par conséquent, eu égard aux spécificités introduites par construction par les systèmes d'IA et également aux risques d'attaques auxquels ils sont exposés, il convient de mettre en œuvre un ensemble de bonnes pratiques visant à la fois à réduire l'exposition du système à de potentielles attaques (par exemple en " cartographiant " son périmètre fonctionnel) mais également à améliorer sa robustesse.

# data[].attributes.attachment_link.description
Face à la complexité et au volume croissants des cyberattaques, il est désormais évident que l'IA peut apporter de nouvelles réponses aux risques de sécurité.

# data[].attributes.attachment_link.readable_text
The innovation, opportunities and potential value to society of AI will not need emphasising to anyone reading this guidance.

Nor is there a need to underline the range of risks involved in the use of technologies that shift processing of personal data to complex computer systems with often opaque approaches and algorithms.

This guidance helps organisations mitigate the risks specifically arising from a data protection perspective, explaining how data protection principles apply to AI projects without losing sight of the benefits such projects can deliver.

What stands out in the following pages is that the underlying data protection questions for even the most complex AI project are much the same as with any new project. Is data being used fairly, lawfully and transparently? Do people understand how their data is being used? How is data being kept secure?

The legal principle of accountability, for instance, requires organisations to account for the risks arising from their processing of personal data -whether they are running a simple register of customers' contact details or operating a sophisticated AI system to predict future consumer demand.

Aspects of the guidance should act as an aide memoire to those running AI projects. There should be no surprises in the requirements for data protection impact assessments or of documenting decisions. The guidance offers support and methodologies on how best to approach this work.

Other aspects of the law require greater thought. Data minimisation, for instance, may seem at odds with systems that allow machine learning to conclude what information is necessary from large data sets. As the guidance sets out though, there need not be a conflict here, and there are several techniques that can ensure organisations only process the personal data needed for their purpose.

Similarly, transparency of processing, mitigating discrimination, and ensuring individual rights around potential automated decision-making can pose difficult questions. Aspects of this are complemented by our existing guidance 'Explaining decisions made with AI guidance', published with the Alan Turing Institute in May 2020.

The common element to these challenging areas, and perhaps the headline takeaway, is the value of considering data protection at an early stage. Mitigation of risks must come at the design stage: retrofitting compliance as an end-of-project bolt-on rarely leads to comfortable compliance or practical products. This guidance should accompany that early engagement with compliance, in a way that ultimately benefits the people whose data AI approaches rely on.

The development and use of AI within our society is growing and evolving, and it feels as though we are at the early stages of a long journey. We will continue to focus on AI developments and their implications for privacy by building on this foundational guidance, and continuing to offer tools that promote privacy by design to those developing and using AI.

I must end with an acknowledgment of the excellent work of one of the document's authors, Professor Reuben Binns. Prof Binns joined the ICO as part of a fellowship scheme designed to deepen my office's understanding of this complex area, as part of our strategic priority of enabling good practice in AI. His time at the ICO, and this guidance in particular, is testament to the success of that fellowship, and we wish Prof Binns the best as he continues his career as Associate Professor of Computer Science at the University of Oxford.

---

Give your feedback

We will continue to develop this guidance to ensure it stays relevant.

We would like to continue to consult with those using the guidance to understand how it works in practice and ensure it remains relevant and consistent with emerging developments.

We are also interested in what tools the ICO could create to compliment the guidance and support you to implement it in practice.

If you would like to contribute to future consultations please provide your details below.

# data[].attributes.attachment_link.description
The innovation, opportunities and potential value to society of AI will not need emphasising to anyone reading this guidance.

# data[].attributes.attachment_link.readable_text
Breaking news regarding accountability

Basic tools for accountability compliance

The AEPD has developed tools and help material to assist with compliance with general data protection regulation for small businesses, entrepreneurs and developers, and other types of controllers. This section lists those tools that are common to all types of processing. In the section "Guides, reports and technical notes" you can find specific material that extends their scope to specific treatments, technologies or controllers.

In any case, controllers and processors should not forget to verify that they comply with all the requirements and obligations that guarantee compliance with GDPR and national rules on data protection.

Risk management and Impact Assessment regarding Data Protection

The following resources support the obligation to carry out a risk analysis of personal data processing, and in case that is needed, the obligation to carry out a data protection impact assessment:

There are more resources about risk management and DPIA in the main section: Risk management

Data Protection by Design and by Default

The following resources support the obligation to take into account, from the initial stages of definition and analysis of the processing, appropriate technical and organisational measures for ensuring, by design and by default, data protection principles implementation:

Personal Data Breach Management

The following resources support the obligation to implement incident recording and notification mechanisms in order to properly manage any possible breach.

Application sectors and technologies

In order to respond to sectors of activity or technologies that incorporate singularities in data processing, referenced is made below to resources of interest, both national and international, that can serve as support to comply with the principle of accountability. At this time, published materials and resources cover the following areas:

Anonymisation and Pseudonymisation

Guidelines and technical surveys

Posts

International recommendations and guidelines

Tools

Artificial Intelligence and automated decisions

Due to the big use of the artificial intelligence in biometrics, in the following link you can find some very interesting information about Biometrics

Guidelines and technical surveys

International recommendations and guidelines

Big data

International recommendations and guidelines

Biometrics

Due to the big use of the artificial intelligence in biometrics, in the following link you can find some very interesting information about Artificial Intelligence and automated decisions

Guidelines and technical surveys

Post

International recommendations and guidelines

Blockchain

Posts

Cloud computing

International recommendations and guidelines

Covid-19 Pandemic

Guidelines and technical surveys

Posts

International recommendations and guidelines

Data protection by design and by default

Guidelines and technical surveys

Posts

International recommendations and guidelines

Encryption and privacy

Posts

International recommendations and guidelines

Governance and data protection policies

Posts

International recommendations and guidelines

Internet and mobile systems

Guidelines and technical surveys

Posts

International recommendations and guidelines

Internet of Things (IoT) and Connected Systems

Guidelines and technical surveys

Posts

International recommendations and guidelines

Personal data breach and security

Guidelines and technical surveys

Tools

Templates and forms

Posts

International recommendations and guidelines

Public Administrations

Guidelines and technical surveys

Templates and forms

International recommendations and guidelines

Risk management

Guidelines and technical surveys

Tools

Templates and forms

Posts

International recommendations and guidelines

Telecommuting

Guidelines and technical surveys

Posts

International recommendations and guidelines

# data[].attributes.attachment_link.description
In order to promote and disseminate knowledge about risk management for the rights and freedoms of natural persons, the AEPD (Spanish Data Protection Authority) develops resources and tools to promote compliance with the RGPD, focusing attention on supporting SMEs and entrepreneurs.

# data[].attributes.headline
ENISA: Cybersecurity of AI and Standardisation

# data[].attributes.subject
ENISA: Cybersecurity of AI and Standardisation

# data[].attributes.attachment_link.readable_text
The overall objective of the present document is to provide an overview of standards (existing, being drafted, under consideration and planned) related to the cybersecurity of artificial intelligence (AI), assess their coverage and identify gaps in standardisation. It does so by considering the specificities of AI, and in particular machine learning, and by adopting a broad view of cybersecurity, encompassing both the 'traditional' confidentiality-integrity-availability paradigm and the broader concept of AI trustworthiness. Finally, the report examines how standardisation can support the implementation of the cybersecurity aspects embedded in the proposed EU regulation laying down harmonised rules on artificial intelligence (COM(2021) 206 final) (draft AI Act).

# data[].attributes.attachment_link.description
The overall objective of the present document is to provide an overview of standards (existing, being drafted, under consideration and planned) related to the cybersecurity of artificial intelligence (AI), assess their coverage and identify gaps in standardisation.

# data[].attributes.attachment_link.description
NIST AI Risk Management Framework (AI RMF) is intended for voluntary use and to improve the ability to incorporate trustworthiness considerations into the design, development, use, and evaluation of AI products, services, and systems.

# data[].attributes.attachment_link.readable_text
UPDATED August 3, 2021. FPF released the white paper, The Spectrum of AI: Companion to the FPF AI Infographic, to provide additional detail and analysis for use of this Infographic tool as an educational resources for policymakers or regulators.

FPF has just completed its newest infographic educational tool, The Spectrum of Artificial Intelligence.

AI is the computerized ability to perform tasks commonly associated with human intelligence, including reasoning, discovering patterns and meaning, generalizing knowledge across spheres of application, and learning from experience. The growth of AI-based systems in recent years has garnered much attention, particularly in the sphere of Machine Learning (ML). A subset of AI, ML systems "learn" from the success or accuracy of their outputs, and can adapt their programming over time, with minimal human intervention. But there are other types of AI that, alone or in combination, lie behind many of the real-world applications in common use.

General AI - truly human-level computational systems - does not yet exist. But Narrow AI exists in many fields and applications where computerized systems enhance human output or outperform humans at defined tasks. This chart is designed to identify and explain the main types of AI, their relationship to each other, and provide some specific examples of how they are currently in place throughout our day-to-day lives. It also demonstrates how AI exists within the timeline of human knowledge and development. Building on knowledge from philosophy, mathematics, physics, ethics, and logic, and more recently, statistical analytics and modeling, it also reflects the foundational role of computer design and security.

There is not 100 percent consensus around the labels for the various types of AI, but for our purposes, we have adopted the following categories as representing generally accepted terms in common use.

Symbolic AI, including subsets: Expert Systems, Search, and Planning & Scheduling

Rules Based

Robotics

Computer Sensing

Knowledge Engineering

Natural Language Processing, and of course,

Machine Learning, including Deep Learning, Neural Networks, Reinforcement Learning, and General Adversarial Networks (GANs)

To aid in understanding these various types of AI programming, we have highlighted a specific use case in a number of broad topic areas.

Finance - Tax Compliance programs that allow you to fill out your tax forms and ensure your information included and provided in a way that is within current legal requirements of the tax code.

Healthcare - Ambient Charting, where conversations between doctors and patients are recorded and added to the patient record as they happen, with key words and followups noted as appropriate.

Tracking - as used in Workplace Monitoring to provide both physical and digital accountability, while enforcing security policies.

Mobility and Transportation - Turn-by-Turn Navigation provides real-time guidance through traffic or even when an area is under construction.

Social Media - platforms use AI to face the challenges around appropriate and effective Speech or Content Moderation.

Forecasting - Supply Chain Management has reached new levels of efficiency and accuracy with AI-based modeling and predictions.

For further information or to provide comments or suggestions, please contact Brenda Leong ([email protected]) or Dr. Sara Jordan ([email protected]).

# data[].attributes.attachment_link.title
The Spectrum of Artificial Intelligence - An Infographic Tool - Future of Privacy Forum

# data[].attributes.attachment_link.description
UPDATED August 3, 2021. FPF released the white paper, The Spectrum of AI: Companion to the FPF AI Infographic, to provide additional detail and analysis for use of this Infographic tool as an educational resources for policymakers or regulators.FPF has just completed its newest infographic educational tool, The Spectrum of Artificial...

# data[].attributes.attachment_link.readable_text
Over the last few months, we have seen generative AI systems and Large Language Models (LLMs), like OpenAI's ChatGPT, Google Bard, Stable Diffusion, and Dall-E, send shockwaves throughout society. Companies are racing to bake AI features into existing products and roll out new services. Many Americans are worrying whether generative AI and LLMs are going to replace them in the workforce, and teachers are downloading ChatGPT specific software to ensure their students are not plagiarizing homework assignments. Some have called for a pause to AI development. But organizations and individuals are adopting LLMs more quickly, and the trend shows no signs of abating.

Organizations have quickly seen employees using generative AI and LLM tools in their workstreams. Few workers are waiting for permission to use the technologies to speed up complex tasks, get tailored answers to full-sentence questions, or draft content like marketing emails. However, the growing use of LLMs creates risks, such as privacy concerns, content inaccuracies, and potential discrimination. Use of LLMs can also be deemed inappropriate in certain contexts and create discontent-students recently criticized their university for lacking empathy when the school used ChatGPT to draft an email notice about a nearby mass shooting.

As organizations navigate these uncertainties, they are asking whether, or when, employees should be permitted to use LLMs for their work activities. Many organizations are establishing or considering internal policies and guidelines for when employees should be encouraged, permitted, discouraged, or prohibited from using such tools. As organizations create new policies, they should be aware that:

1. When workers share personal information with LLMs, it can create legal obligations for data protection and privacy, including regulatory compliance;

2. Many organizations will need to establish norms for originality and ownership, including when it is appropriate for employees to use LLMs or other generative AI systems to create novel content;

3. Organizations need to carefully evaluate any uses for potential bias, discrimination, and misinformation while also considering other potential ethical concerns.

What are LLMs?

In late 2022, OpenAI released its AI chatbot, ChatGPT, which has now undergone several versions and is available as ChatGPT-4. ChatGPT is both a generative AI and a large language model (LLM), which are two distinct but similar AI terms. A "generative AI system" is a type of AI that has been trained on data so that it can produce or generate content similar to what it has been trained on, such as new text, images, video, music, and audio. For example, a growing number of AI tools are available for generating artwork and some even create videos based on text. A "large language model" (LLM) is a type of generative AI that generates text. LLMs can perform a variety of language-related tasks, including translations, text summarization, question-answering, sentiment analysis, and more. They can also generate text that mimics human writing and speech, which have been used in various applications such as chatbots and virtual assistants. To produce human-like responses, LLMs are trained on vast quantities of text data. In ChatGPT's case, it was trained on vast amounts of data from the internet.

#1. Legal Obligations.
a. Data protection and privacy.
In general, users of generative AI and LLMs should avoid inputting personal or sensitive information into ChatGPT or similar AI tools. ChatGPT uses the data that is input by many users to generate individual responses and further train its model for future responses to all users. If an employee inputs data that contains confidential information, such as trade secrets, medical records, or financial data, then that data may be at risk, especially if there is a data breach of the AI system that results in authorized access or disclosure. Similarly, if an individual puts personally identifiable information or sensitive data into the model, and that data is not properly protected, it could also be improperly accessed or used by unauthorized individuals.

Furthermore, personal information disclosed to LLMs could be used in additional ways that violate the expectations of the people to whom the information relates. For example, ChatGPT also continues to use and analyze this data. Thus, the sensitive information that an employee inputs about a customer or patient could potentially be revealed to another user if they pose a similar question or prompt. Further, every engagement with ChatGPT has a unique identifier-there is a login trail of people who are using it. Therefore, an individual's use of ChatGPT is not truly anonymous, raising questions about the retention of sensitive data by OpenAI.
b. Regulatory Compliance.
LLMs are subject to the same regulatory and compliance frameworks as other AI technologies, but as LLMs become more common, they can raise novel questions of how such tools can be used in ways that comply with the General Data Protection Regulation (GDPR) and other regulations. Since ChatGPT processes user data to generate responses, OpenAI or the entities relying on ChatGPT for their own purposes may be considered data controllers under the GDPR, which means they should secure a lawful ground to process users' personal data (such as users' consent) and users must be informed about the controller's ChatGPT-powered data processing activities.

OpenAI and companies relying on ChatGPT's capabilities also need to consider how overarching GDPR principles like data minimization and fairness curtail certain data processing activities, including decisions made on the basis of algorithmic recommendations. Additionally, under the GDPR, data subjects have certain rights regarding their personal data, including the right to access, rectify, and delete their data. But can users really exercise these rights in practice? In regard to data erasure, OpenAI offers users the ability to delete their accounts, but OpenAI has stated that conversations with ChatGPT can be used for AI training. This presents challenges because while it seems that the original input can be deleted, the input can be used to shape and overall improve ChatGPT. Removing a user's complete digital footprint and its effects on ChatGPT may be unfeasible and risks offending the GDPR's " right to be forgotten." Moreover, the repurposing of prompts to train OpenAI's algorithm may raise issues relating to the GDPR's purpose limitation principle, as well as the applicable lawful ground for service improvement after a recent restrictive binding decision from the European Data Protection Board (EDPB) concerning the lawfulness of processing for service improvement purposes.

There are questions about the future regulation of ChatGPT and similar technologies under the European Union (EU) Artificial Intelligence Act (AI Act), which is currently under review by the European Parliament. Proposed in 2021, the regulation is designed to ban certain AI uses such as social scoring, manipulation and some instances of facial recognition. However, recent developments regarding LLMs and related AI services have caused the European Parliament to reassess "high-risk" use cases and how to implement proper safeguards that were not previously accounted for in today's rapidly developing tech environment. EU lawmakers have proposed different ways of regulating general purpose and generative AI systems during their discussions on the text of the AI Act. The consensus at the EU Council is that the European Commission should regulate such systems at a later stage via so-called 'implementing acts'; at the European Parliament, lawmakers may include such systems in the AI Act's high-risk list, therefore subjecting them to strict conformity assessment procedures before they are placed on the market.

#2. Ownership and Originality.

Depending on context, organizations should determine when it is appropriate or ethical for individuals or organizations to use, or take credit for, work generated by LLMs. In education, for example, some teachers are adapting their approaches (e.g., from written to oral presentations) to avoid plagiarism. Schools have even banned ChatGPT due to concerns that the technology will cause students to take shortcuts when writing or forgo doing their own research.

In many cases, these issues raise novel questions about legal rights and liability. For example, software developers have used ChatGPT to write and improve existing code. Yet LLMs, including ChatGPT, have been shown to regularly produce inaccurate content. If an employee uses the code generated by ChatGPT in a product that interacts with the public, organizations may have liability if something goes wrong. In a related issue, if employees input copyrighted, patented, or confidential information (e.g. trade secrets), into a generative AI tool, the resulting output could infringe on intellectual property rights or breach confidentiality obligations.

#3. Ethical Concerns, Bias, Discrimination, and Misinformation.

Finally, organizations must carefully consider all uses of AI, including LLMs, for possible discriminatory outcomes and effects. In general, LLMs reflect the underlying data that they are trained on, which is often incomplete, biased, or outdated. For example, AI training datasets often exclude information from marginalized and minority communities, who have historically had less access to technologies such as the internet, or had fewer opportunities to have their writings, songs, or culture digitized. ChatGPT was trained on internet data, and as a result is likely to reflect and perpetuate societal biases that exist in websites, online books, and articles.

For example, a Berkeley professor asked ChatGPT to create code to determine which air travelers pose a safety risk. ChatGPT assigned a higher "risk score" to travelers who were Syrian, Iraqi, Afghan, or North Korean. A predictive algorithm used for medical decision-making was biased against black patients because it was trained on data that reflected historical bias. Even though the deployers of the algorithm excluded race as a metric when running the system, the algorithm still perpetuated bias against black patients because it took economic factors and healthcare costs into account.

Furthermore, it is clear that generative AI and LLMs can be potentially disruptive and change the way we consume and create information. ChatGPT has demonstrated its ability to write news articles, essays, and television scripts. Supplied with a prompt loaded with disinformation or misinformation, LLMs can produce convincing content that could mislead even thoughtful readers. Audiences are at risk of consuming vast amounts of misinformation if they are not able to fact-check the information given to them or know that content was generated by an LLM or AI. Organizations that use LLMs should be aware that LLMs can generate inaccurate or misleading information, even when prompts are not intended to mislead. Vigilance is required when organizations ask LLMs or generative AI to give clients, customers, or users information solely based on the LLM.

To address ethical concerns, bias, discrimination, and misinformation, organizations have a responsibility to scrutinize their use of generative AI and LLMs. Ethical considerations are incredibly important and progress is being made on transparency in generative AI models, though complete solutions remain elusive. Ethical considerations are especially important when the AI is used in an outcome-determinative way, such as in hiring or healthcare. In some cases, such uses will risk running afoul of employment or other civil rights laws. Organizations must determine the different contexts that this type of AI use can be particularly susceptible to bias and discrimination, and will want to avoid those situations. Organizations should engage and talk to the communities that are most affected in these cases and get stakeholder input when drafting internal policies.

Conclusion

Recent developments concerning LLMs and generative AI demonstrate substantial technological advancements while also presenting many uncertainties. There are many unanswered questions and yet to be discovered risks that may result from use of AI in the workplace. However, these harms can be mitigated if organizations take the time to address these issues internally and develop best practices. We encourage organizations to be inclusive and cross-collaborative when engaging in these conversations with lawyers, engineers, customers, and the public.

# data[].attributes.attachment_link.title
Let's Look at LLMs: Understanding Data Flows and Risks in the Workplace - Future of Privacy Forum

# data[].attributes.attachment_link.description
Over the last few months, we have seen generative AI systems and Large Language Models (LLMs), like OpenAI's ChatGPT, Google Bard, Stable Diffusion, and Dall-E, send shockwaves throughout society. Companies are racing to bake AI features into existing products and roll out new services.

# data[].attributes.attachment_link.readable_text
In April 2021, the European Commission presented its proposal for a Regulation of the European Parliament and of the Council laying down harmonised rules for AI, also known as the AI Act of the European Union.

Here, our AI Lawyers provide relevant background and information as well as the current status on the AI Act proposal:

An article feed with relevant articles and comments,

the Taylor Wessing legislation tracker,

links to all relevant documents,

graphic overviews on specific AI Act topics,

frequently asked questions and answers,

as well as relevant links for other information and further research.

This page is maintained by Taylor Wessing's AI Lawyers within our TMT Team based in Düsseldorf, Germany. We advise our clients on IT, telecommunications and data protection law and have particular experience in legal issues relating to digitalization and artificial intelligence.

The AI Act of the European Union is still in the making and this page is continuously updated. We encourage all visitors to contribute with ideas and suggestions. Do not hesitate to get in touch with us!

# data[].attributes.attachment_link.description
In April 2021, the European Commission presented its proposal for a Regulation of the European Parliament and of the Council laying down harmonised rules for AI, also known as the AI Act of the European Union.

# data[].attributes.attachment_link.title
AI & Machine Learning - Future of Privacy Forum

# data[].attributes.attachment_link.description
An AI-based computer system can gather data and use that data to make decisions or solve problems – using algorithms to perform tasks that, if done by a human, would be said to require intelligence. The benefits created by AI and machine learning (ML) systems for better health care, safer… - Future…

# data[].attributes.headline
ChatGPT: IP, Cybersecurity & Other Legal Risks of Generative AI

# data[].attributes.subject
ChatGPT: IP, Cybersecurity & Other Legal Risks of Generative AI

# data[].attributes.attachment_link.title
Ilia Kolochenko, PhD, MSc, LLM on LinkedIn: IP, Cybersecurity & Other Legal Risks of Generative AI | 28 comments

# data[].attributes.attachment_link.description
While the hype around #ChatGPT is slowing down, technical and legal risks of generative AI are growing. Gordon Platt and I explain in our Bloomberg Law article… | 28 comments on LinkedIn

# data[].attributes.attachment_link.readable_text
Contributors: John Schulman, Barret Zoph, Christina Kim, Jacob Hilton, Jacob Menick, Jiayi Weng, Juan Felipe Ceron Uribe, Liam Fedus, Luke Metz, Michael Pokorny, Rapha Gontijo Lopes, Shengjia Zhao, Arun Vijayvergiya, Eric Sigler, Adam Perelman, Chelsea Voss, Mike Heaton, Joel Parish, Dave Cummings, Rajeev Nayak, Valerie Balcom, David Schnurr, Tomer Kaftan, Chris Hallacy, Nicholas Turley, Noah Deutsch, Vik Goel, Jonathan Ward, Aris Konstantinidis, Wojciech Zaremba, Long Ouyang, Leonard Bogdonoff, Joshua Gross, David Medina, Sarah Yoo, Teddy Lee, Ryan Lowe, Dan Mossing, Joost Huizinga, Roger Jiang, Carroll Wainwright, Diogo Almeida, Steph Lin, Marvin Zhang, Kai Xiao, Katarina Slama, Steven Bills, Alex Gray, Jan Leike, Jakub Pachocki, Phil Tillet, Shantanu Jain, Greg Brockman, Nick Ryder, Alex Paino, Qiming Yuan, Clemens Winter, Ben Wang, Mo Bavarian, Igor Babuschkin, Szymon Sidor, Ingmar Kanitscheider, Mikhail Pavlov, Matthias Plappert, Nik Tezak, Heewoo Jun, William Zhuk, Vitchyr Pong, Lukasz Kaiser, Jerry Tworek, Andrew Carr, Lilian Weng, Sandhini Agarwal, Karl Cobbe, Vineet Kosaraju, Alethea Power, Stanislas Polu, Jesse Han, Raul Puri, Shawn Jain, Benjamin Chess, Christian Gibson, Oleg Boiko, Emy Parparita, Amin Tootoonchian, Kyle Kosic, Christopher Hesse

# data[].attributes.attachment_link.description
We've trained a model called ChatGPT which interacts in a conversational way. The dialogue format makes it possible for ChatGPT to answer followup questions, admit its mistakes, challenge incorrect premises, and reject inappropriate requests.

# data[].attributes.attachment_link.readable_text
What are they faking?
What does it mean for the rest of us?
Co-authored by William Marks and Brenda Leong Introduction
These events seem unlikely, but click the hyperlinks; they were all recorded on video-they must be real! Or so it seems. There have long been many varieties of video manipulation, but those processes were generally expensive, time-consuming, and imperfect. That is no longer the case. Professional tools are increasingly effective and affordable, and open-source code that even relative amateurs can use will soon be available to produce believable digital video forgeries.

The media has recently labeled these manipulated videos of people "deepfakes," a portmanteau of "deep learning" and "fake," on the assumption that AI-based software is behind them all. But the technology behind video manipulation is not all based on deep learning (or any form of AI), and what are lumped together as deepfakes actually differ depending on the particular technology used. So while the example videos above all doctored in some way, they were not all altered using the same technological tools, and the risks they pose - particularly as to being identifiable as fake - may vary.
First (and Still), There Were "Cheapfakes" When Machines Learn, the Fakes Get Smarter
Two more recent versions of machine learning-based video editing are causing more concern. One is what is accurately a "deepfakes," and the other, even newer process, is a deep video portraits (DVP). Because the processes are similar, and the outcomes and risks aligned, most common media references will likely consider them all deepfakes moving forward.

In these examples, the original videos may be fairly easily accessible to demonstrate that the altered versions were faked. Alternatively, observing other aspects of the video (such as height or other physical characteristics) may make it clear the substituted face is not the original person in the video. The technology is not seamless-close observation shows that the "fit" of Oliver's face isn't perfect, particularly when he turns to the side. And in some cases, the editing may be discernible upon examination of the altered file. However, quality is improving steadily and this sort of substitution may be harder to visually identify in the future.

The other process - creating Deep Video Portraits (DVPs) - is even newer, more powerful, and potentially more dangerous. DVPs are also created through GANs and likewise commonly referred to as deepfakes. But whereas actual deepfakes allow someone to place a mask of someone else's face onto an existing video clip, DVPs allow the creator to digitally control the movements and features of an already-existent face, essentially turning them into a puppet in a new video recording. As demonstrated , using a relative short amount of existing video from the target (the person to be faked), the program allows a source actor to move his head and mouth, talk, and blink so that it seamlessly appears that the targeted person is doing exactly those movements and expressions, being controlled to say or express exactly what the editor desires.

In a video created in this manner, Jordan Peele digitally manipulated Obama's face to speak words and make facial expressions that Obama never did. After making this fake video with FakeApp (first released in 2018), Peele edited the file further with Adobe After Effects to create a particularly convincing altered performance.

Since the DVP process is creating a new video file directly (not manipulating an existing file), there is no digital editing trail to technologically identify changes. And there is no original video to find or contrast it with. There AI systems being developed that may be able to detect DVPs based on the inconsistencies of movements and other performance discrepancies, but whether they can keep up with the improving quality of the faked imagery remains to be seen.
"When Machines Compete": How GANS Work
GANs work by having two neural networks compete directly with each other.¹ The first (the "generator") is tasked with creating fake data - in this case, a video - based on a training set of real data (video, audio, or text data from existing files or recordings). The program is trained to emulate the data in the real files: learning what human faces looks like; how people move their heads, lips, and eyebrows when they talk; and what sounds are made in speech.

The second neural net (the "adversary"), a program also trained on the real video data, uses its learned analysis of how people move and speak to try to distinguish an AI-generated video-that is, the job of the adversary is to spot the fakes created by the generator. The more original video data it is fed, the better this adversarial network becomes at catching the fake outputs. But concurrently, the generator uses the experience of being caught out to improve its fakes. This drives an upward spiral of excellence, as each of the algorithms continually gets better and better until the first is finally able to consistently create outputs so believable that the second cannot distinguish them from real footage.
Identifying Fakes: "Technically," It's A Problem
How will we deal with the problem when average users can generate realistic looking images, videos, and stories with nothing more than an easily accessible program that requires minimal input? Potential concerns include an abundance of fake nude photos, admissions of treason or fraud, and fabricated news stories all across the internet and social media. The private sector, government, and individuals will all need to react.²
Identifying Solutions: Political Will
Any proposed solutions are likely to include at least some degree of regulatory intervention. Specific to the U.S. legislative context, one proposal to deal with the potential proliferation of fake videos, news, and photos, is to amend section 230 of the Communications Decency Act. The law has currently established that ISPs and many online services are not responsible for content posted by users. Free speech groups maintain that if companies were responsible for the content posted by users, they would be forced to block or strictly censor at mass scale, with a subsequent chilling effect on internet free speech.

The feasibility of finding practical ways to identify and enforce such requirements is questionable, and there would be inevitable First Amendment challenges to resolve. These questions only expand when considering global regulatory implications.
Identifying Solutions: Social Norms
An information flow in which it is difficult or even impossible to distinguish what is real from what is not is a dangerous one. Forgeries may be accepted as true, and truth can be undermined by doubt. In 1994, Johnson and Seifert called this the "continued influence effect." When a fake is later to be proven false (the Protocols of the Elders of Zion ), or a truth is doubted when clearly proven to be correct (scientific progress such as the history of women's physiology ; or the recent anti-vax movement based on a study suggesting a connection between vaccines and autism), the initial negative or positive associations remain, and can have significant, lingering effects.

Some people may align themselves with sources they deem reliable, while some may deny the pursuit of truth as a meaningful endeavor altogether. But many will want to proactively ensure they are receiving a comprehensive and accurate reporting of the world around them. If unable to confidently determine the truth for themselves, these people may seek an arbiter of some kind, looking to fact-checking applications, reliable media companies, or public and non-profit agencies to supply, verify, or validate information. It may be that more or different organizations are needed to formally fill this role, to objectively and disinterestedly provide news "about" the news.

We need to confront the technical and political issues arising from computer-generated fake videos and media. A first step certainly includes at least increasing public awareness that these technologies exist, are getting better, and that people must assume some responsibility to critically analyze the information that they consume.

¹ This paper addresses the use of GANs to generate fake videos specifically, but the use of GANs in ML developments generally have many different applications and use cases, including advancement of music, recreating voices for those who cannot speak, addressing various medical conditions, and other simulated or synthetic applications that, while "artificial" are not designed with the intent to deceive.

² While detection is one clear, and probably necessary, arm of research, there are multiple ways to consider how to meet this threat, including ways to track original files, establish provenance, and otherwise validate certain files. These methods also raise questions of risk, however, for anonymous uses such as whistleblowers or civil rights activists, so all options include challenges that preclude "easy" fixes.

# data[].attributes.attachment_link.title
Digital Deep Fakes - Future of Privacy Forum

# data[].attributes.attachment_link.description
The media has recently labeled manipulated videos of people "deepfakes," a portmanteau of "deep learning" and "fake," on the assumption that AI-based software is behind them all. But the technology behind video manipulation is not all based on deep learning (or any form of AI), and what are lumped together...

# data[].attributes.attachment_link.readable_text
When Jason Allen submitted his "Théâtre D'opéra Spatial" into the Colorado State Fair's fine arts competition last week, the sumptuous print was an immediate hit, beating 20 other artists in the "digitally manipulated photography" category to win the first-place blue ribbon and a $300 prize.

What Allen had only hinted at, however, was that the artwork had been created in large part by an artificial-intelligence tool, Midjourney, that can generate realistic images at a user's command. The portrait of three figures, dressed in flowing robes, staring out to a bright beyond, was so finely detailed the judges couldn't tell.

Allen's piece offers a clear example of how rapidly AI-generated art has advanced. Trained on billions of internet images, the systems have decisively pushed the boundaries of what computers can create.

But it has also sparked a massive debate over the meaning of art, with Allen facing accusations that he had been deceptive in beating out other human artists with something he asked a machine to create.

Text-to-image tools like DALL-E 2 and Midjourney have quickly increased in sophistication and become one of the hottest topics in AI. They can generate not just fake people, objects and locations but mimic entire visual styles. Users can demand the art piece look like a cartoon storybook or a historical diagram or an Associated Press photograph, and the system will do its best to oblige.

But AI-generated art has been criticized as automated plagiarism, because it relies on millions of ingested art pieces that are then parroted en masse. It has also fueled deeper fears: of decimating people's creative work, blurring the boundaries of reality or smothering human art.

Allen said his art piece shows people need to "get past their denial and fear" of a technology that could empower new inventions and reshape our world. The AI, he said, "is a tool, just like the paintbrush is a tool. Without the person, there is no creative force."

But he also struck a note of defiance at the critics he believed did not appreciate the profound message in his AI-generated art.

"You said AI would never be as good as you, that AI would never do the work you do, and I said, 'Oh really? How about this? I won,'" he said. "It's here now. Recognize it. Stop denying the reality. AI isn't going away."
***
Allen, 39, lives south of his hometown of Colorado Springs and runs a company, Incarnate Games, that makes tabletop fantasy games. He went into the Air Force after high school and got a computer science degree at a Colorado technical school. He does not consider himself an artist and had never entered an arts competition before.

Earlier this year, he said, he started noticing people posting more AI art on social media, but he had been initially skeptical to try it himself due to "spiritual reasons." Elon Musk, he remembered, had compared AI to "summoning the demon," and the practice felt like it could be "a gateway into communicating with the unknown."

"That's what it feels like. This isn't being created by a human," Allen said Thursday while watching a jiujitsu tournament in Las Vegas. "There's a lot to be said about the spiritual meddling of our reality. They would use anything they could to influence humanity."

But the art was just so intricate, he said, that he couldn't stop thinking about it. He started playing with AI-powered art tools: WOMBO Dream, NightCafe, starryai. Then, someone invited him to Midjourney, and he became obsessed.

The Google engineer who thinks the company's AI has come to life

Midjourney has become one of the most popular AI art generators largely because it allows anyone to freely create new images on command. Using the prompt "/imagine," a user can type in whatever they want to see and the AI will return four newly created images in 60 seconds. The user can also ask the AI to improve, or "upscale," the visual quality with new variations on the same idea.

The start-up, which calls itself "an independent research lab ... expanding the imaginative powers of the human species," operates largely out of a 1-million-follower network on the chat service Discord, with rooms devoted to character creation, environments and "show and tell."

After paying for a corporate account, Allen started generating thousands of images, changing the text prompts with every creation. He experimented with new settings, scenarios and effects. He asked for images in the styles of Leonardo da Vinci and the American psychedelic artist Alex Grey.

The pieces that really caught his attention, though, were what he now calls his "space opera theater" series. He started with a simple mental image - "a woman in a Victorian frilly dress, wearing a space helmet" - and kept fine-tuning the prompts, "using tests to really make an epic scene, like out of a dream." He said he spent 80 hours making more than 900 iterations of the art, adding words like "opulent" and "lavish" to fine tune its tone and feel. He declined to share the full series of words he used to create his art, saying it is his artistic product, and that he intends to publish it later. "If there's one thing you can take ownership of, it's your prompt," he said.

"I was like: Dude," he said. "This is so sick! I want to see more of it! I'm addicted! I'm obsessed!"

When he found images he really liked, he pulled them into Adobe Photoshop to remove visual artifacts. In one image, the central figure was missing a head, so he also painted in a crop of dark, wavy hair. He used another machine-learning tool, Gigapixel AI, to increase the photos' quality and sharpness, then printed the three pieces on canvas - all variations on the French phrase for "space opera theater," which he thought sounded cool - and drove to submit them to the state fair.

When he looked at the pieces, he said, he saw "a supernatural reality ... something we haven't even been able to experience yet, past the great beyond." But the pace of AI art is moving perhaps even faster than the internet. "You're looking at art from a month ago," he added. "In technology terms, that's decades. This piece is antiquated compared to what Midjourney is doing now."

***

The state fair in Pueblo, Colo., was an unlikely place for the writing of a new chapter in art history. The 150-year-old festival, known for its horse and livestock competitions, runs a series of more traditional art competitions, including for homemade dolls, quilts, porcelain art and needlework, as well as for the best canned carrots, medicinal remedies and holiday breads.

Of the 596 entries in the "fine arts" competition, 21 amateur "emerging artists" submitted pieces of "digitally manipulated photography," one of the fair's newest categories. Asked what art materials he had used, Allen told state fair officials only that he used Midjourney - though he did not exactly go into detail, and no one seems to have asked.

One of the judges, Dagny McKinley, an author and art historian who runs a playwright festival in nearby Steamboat Springs, remembers walking past Allen's canvas and being immediately drawn to a piece that felt reminiscent of Renaissance art.

"It had an immediate story: People looking out into another world, everyone with their backs to you, no one facing or engaging with the viewers," she said. "You get interested: What are they seeing?"

McKinley said she did not realize the art was AI-generated but said it wouldn't have changed her judgment anyway; Allen, she said, "had a concept and a vision he brought to reality, and it's really a beautiful piece."

Sebastian Smee, The Washington Post's art critic, said the piece's textures and lighting are reminiscent of Gustave Moreau, a late 19th-century artist, associated with the Decadents, who influenced Edgar Degas and Henri Matisse. (He also recalled a quote from the artist Sol LeWitt, who said, "The idea becomes a machine that makes the art.")

When Allen announced his victory himself on Midjourney's Discord channel, saying he had spent "many weeks of fine tuning and curating," the responses spiraled between muted excitement to outright dread. In a chat board devoted to philosophical debates, one user compared the win to "entering a marathon and driving a Lamborghini to the finish line." Another user wrote that the "stunt" threatened to "get this tool banned and hated even more."

Ready or not, mass video deepfakes are coming

The win also triggered a flood of rage online. A tweet expressing profane dismay at Allen's win has been liked more than 85,000 times. Another person tweeted, "We're watching the death of artistry unfold right before our eyes." Allen said he has also gotten "a lot of very nasty hate" in his online inboxes; on his game company's Instagram profile, one user said he should return his award and "post a public apology before some MAJOR backlash comes your way."

Some of the frustration has come from how the tools were built: a similar tool, Stable Diffusion, was "trained" on 2 billion images taken from the internet, including from personal blogs and amateur-art sites like Flickr and DeviantArt.

Allen, however, dismisses the point as shallow: "Where did you learn how to do your art? You looked at art. Whose art was it? You learned their techniques, you studied their art, you added it to your repertoire," he said.

Allen listed the pieces for $750 each, he said, and two were sold at the fair to unknown buyers, though he now frets that he should have charged far more, given that it could be "essentially a piece of art history." On Discord, some users asked whether he should have been more explicit, to which Allen replied: "Did I have to?"

Despite the online furor, Allen's neighbors seemed more sanguine about adapting to AI. As far as anyone at the Colorado Department of Agriculture can tell, Allen did not break any rules. Pieces for the category are only required to involve "technology as part of the creative or presentation process." Digital filters, color-manipulation tools and the "recombination of images" are all expressly allowed.

No one has filed an official grievance over the result, either, department spokeswoman Olga Robak said, though there has been an unrelated dispute in the fair's goat-shearing contest.

Robak, who studied art history, finds the controversy fascinating. "People put bananas on the wall and called it art," she said. "Even photography was not considered an art form for a long time; people said it was just pushing a button, and now we realize it's about composition, color, light. Who are we to say that AI is not the same way?"

Don't ask if the duct-tape banana is art. Ask if it's any good.

***

Jessica Hair, a 25-year-old receptionist at a doctor's office who won third place in the competition, said she did not feel Allen had acted unfairly and had no hard feelings about his win.

Hair said her "Judge, Jury, Executioner," which depicts a tuxedoed skeleton on a golden throne surrounded by skulls, took 15 hours to create with a stylus on an iPad Pro. But Allen's piece took time, effort and subjective judgment, too, and "how do we qualify what is and isn't art?" she said.

She did wonder, though, if it might have broken the fair's rules requiring all art to have been made by Colorado residents. Would the AI, wherever it exists, qualify?

McKinley, the contest judge, said she understands the frustrations from some artists feeling spurned in their craft, and she believes the festival should consider a category purely devoted to AI art. But she sees such technology as opening up a new world of possibilities for artists - and as something it'd be better to embrace, since it isn't going away.

"It's not going to take away from a beautiful painting or a sculpture you can touch," she said. "It's just one more tool we have to advance what we can create."

Gregory Block, an oil painter in Denver who was not a part of the competition, said he finds it hard to imagine an AI generator supplanting the hundreds of hours - and all the "heart, soul, blood, sweat and tears" - he's invested into his art. But he also thinks back to the artists that first inspired him, who used rudimentary devices like the camera lucida, in the 1800s, to aid their own designs.

"That was thought of as cheating, too," he said. "Yet they used it to make these incredible paintings: anatomically correct figures, beautiful soft lighting. ... Those steps in technology are elemental to our art. Otherwise, we'd still be doing cave paintings with just our hands and blood."

The AI, he said, can imbue the art with a mysterious beauty, made all the more special because it is so hard to understand. But "the soul any of us can find in a piece of artwork, the emotion, the human struggle we identify with in art is always our own."

"It doesn't have to necessarily be created out of a human soul, the artwork itself. It is for us to see and react to," Block said. "We the viewers are, in the end, the ultimate artists. We're the ones creating the world that is coming in through our eyeballs. That world is in our mind."
Nitasha Tiku contributed to this report.

# data[].attributes.attachment_link.title
He used AI to win a fine-arts competition. Was it cheating?

# data[].attributes.attachment_link.description
When Jason Allen submitted his "Théâtre D'opéra Spatial" into the Colorado State Fair's fine arts competition last week, the sumptuous print was an immediate hit. It also marked a new milestone in the growth of artificial intelligence.

# data[].attributes.attachment_link.readable_text
Welcome to AI Ethics Guidelines Global Inventory, a project by AlgorithmWatch that maps frameworks that seek to set out principles of how systems for automated decision-making (ADM) can be developed and implemented ethically (last update in April 2020). The database currently includes 173 guidelines. Learn more about the inventory here or jump right in and browse through our inventory using the new filters and search feature!

Are relevant guidelines missing? Please let us know and use the submission form!

# data[].attributes.attachment_link.title
AI Ethics Guidelines Global Inventory by AlgorithmWatch

# data[].attributes.attachment_link.description
AI Ethics Guidelines Global Inventory, a project by AlgorithmWatch that maps frameworks that seek to set out principles of how systems for automated decision-making (ADM) can be developed and implemented ethically. Browse through the inventory using the new filters and search feature! Last update: M

# data[].attributes.attachment_link.readable_text
Skip to Content

Avenue de l'Europe
F-67075 Strasbourg Cedex
Tel. +33 (0)3 88 41 20 00
www.coe.int

Datavisualisation of AI initiatives Datavisualisation of AI initiatives

Non-exhaustive data collection by the Secretariat. Database in open data: Click here to access to the database

Tutorial Tutorial

This datavisualisation brings together more than 450 documents related to artificial intelligence, coming from national authorities, the private sector, international organisations or multi-stakeholder initiatives.

The "bubbles" on the upper right-hand side allow you to filter according to the source of the documents. Clicking on national authorities will only display information relating to that source, such as distribution by publication date or simply the descriptive list.

On the left side, the rectangles, whose size is proportional to the number of documents, are also clickable. Here, for example, Germany can be selected to "filter" information about this state from the entire graphical representation. The list of documents is automatically updated, and can be accessed directly by clicking on the title.

Clicking on the light blue bubble, which represents the international organisations, it is clear that the Council of Europe, together with the European Union, the OECD and UNESCO, are among the most productive organisations in the field. Clicking on the rectangle representing the Council of Europe updates the entire datavisualisation with data relating to the organisation.

# data[].attributes.attachment_link.description
Skip to Content Avenue de l'Europe F-67075 Strasbourg Cedex Tel. +33 (0)3 88 41 20 00 www.coe.int Datavisualisation of AI initiatives Datavisualisation of AI initiatives Non-exhaustive data collection by the Secretariat. Database in open data: Click here to access to the database Tutorial Tutorial This datavisualisation brings together more than 450 documents related to artificial intelligence, coming from national authorities, the private sector, international organisations or multi-stakeholder initiatives.

# data[].attributes.headline
Regulating AI in Europe. Four problems and four solutions

# data[].attributes.subject
Regulating AI in Europe. Four problems and four solutions

# data[].attributes.headline
AEPD: GDPR compliance of processings that embed Artificial Intelligence An introduction

# data[].attributes.subject
AEPD: GDPR compliance of processings that embed Artificial Intelligence An introduction

# data[].attributes.attachment_link.readable_text
Artificial intelligence and human rights

Artificial Intelligence (AI) raises important and urgent issues. AI is already with us - changing the information that we receive, the choices that we make and the ways in which our societies function. In the coming years it will play an even greater role in how governments and public institutions operate, and in how citizens interact and participate in the democratic process.

AI presents both benefits and risks. It is the role of the Council of Europe to ensure that human rights, democracy and the rule of law are protected and promoted in the digital environment. So, we need to ensure that AI is used to promote and protects our standards. I look forward to the outcome of the work of the Committee on Artificial Intelligence (CAI), mandated by the Committee of Ministers to elaborate a [framework] Convention on the development, design and application of artificial intelligence, based on the Council of Europe's standards and that is conducive to innovation.

This will be complemented by sector-specific work throughout our Organisation.

The Council of Europe has on many occasions demonstrated its ability to pioneer new standards, which have then become global benchmarks. We will address AI in this tradition, in a multistakeholder approach with other international organisations, civil society, business and academia.
Marija Pejčinović Burić
Secretary General of the Council of Europe

# data[].attributes.attachment_link.title
Council of Europe and Artificial Intelligence - Artificial Intelligence - www.coe.int

# data[].attributes.attachment_link.description
Welcome to Council's of Europe Artificial Intelligence website

# data[].attributes.attachment_link.readable_text
As Singapore develops its digital economy, a trusted ecosystem is key - one where organisations can benefit from tech innovations while consumers are confident to adopt and use AI. In the global discourse on AI ethics and governance, Singapore believes that its balanced approach can facilitate innovation, safeguard consumer interests, and serve as a common global reference point.

On 25 May 2022, IMDA/PDPC launched A.I. Verify - the world's first AI Governance Testing Framework and Toolkit for companies that wish to demonstrate responsible AI in an objective and verifiable manner. A.I. Verify - currently a Minimum Viable Product (MVP), aims to promote transparency between companies and their stakeholders.

Developers and owners can verify the claimed performance of their AI systems against a set of principles through standardised tests. A.I. Verify packages a set of open-source testing solutions together, including a set of process checks into a Toolkit for convenient self-assessment. The Toolkit will generate reports for developers, management, and business partners, covering major areas affecting AI performance.

10 companies from different sectors and of different scale, have already tested and/or provided feedback. These companies include - AWS, DBS Bank, Google, Meta, Microsoft, Singapore Airlines, NCS (Part of Singtel Group)/Land Transport Authority, Standard Chartered Bank, UCARE.AI, and X0PA.AI.

We welcome organisations to pilot the MVP. Companies participating in the pilot will have the unique opportunity to:

Gain early access to the MVP and use it to conduct self-testing on their AI systems/models;

Use MVP-generated reports to demonstrate transparency and build trust with their stakeholders; and

Help shape an internationally applicable MVP to reflect industry needs and contribute to international standards development.

Read our invitation here, primer here and press release here.

For background on the development of the MVP, please click here.

Model AI Governance Framework

On 23 January 2019, the PDPC released its first edition of the Model AI Governance Framework (Model Framework) for broader consultation, adoption and feedback. The Model Framework provides detailed and readily-implementable guidance to private sector organisations to address key ethical and governance issues when deploying AI solutions. By explaining how AI systems work, building good data accountability practices, and creating open and transparent communication, the Model Framework aims to promote public understanding and trust in technologies.

On 21 January 2020, the PDPC released the second edition of the Model Framework.

Guiding Principles

Decisions made by AI should be
EXPLAINABLE, TRANSPARENT & FAIR

AI systems should be
HUMAN-CENTRIC

From Principles to Practice

Operations Management

The second edition includes additional considerations (such as robustness and reproducibility) and refines the original Model Framework for greater relevance and usability. For instance, the section on customer relationship management has been expanded to include considerations on interactions and communications with a broader network of stakeholders. The second edition of the Model Framework continues to take a sector- and technology-agnostic approach that can complement sector-specific requirements and guidelines.

Access the second edition of the Model Framework here and the primer here.

Implementation and Self Assessment Guide for Organisations (ISAGO)

Intended as a companion guide to the Model Framework, ISAGO aims to help organisations assess the alignment of their AI governance practices with the Model Framework. It also provides an extensive list of useful industry examples and practices to help organisations implement the Model Framework.

ISAGO is the result of the collaboration with World Economic Forum's Centre for the Fourth Industrial Revolution to drive further AI and data innovation. ISAGO was developed in close consultation with the industry, with contributions from over 60 organisations.

Access the ISAGO here.

Compendium of Use Cases

Complementing the Model Framework and ISAGO is a Compendium of Use Cases (Compendium) that demonstrates how local and international organisations across different sectors and sizes implemented or aligned their AI governance practices with all sections of the Model Framework. The Compendium also illustrates how the featured organisations have effectively put in place accountable AI governance practices and benefitted from the use of AI in their line of business. We hope these real-world use cases will inspire other companies to do the same.

Volume 1

Volume 1 features use cases from Callsign, DBS Bank, HSBC, MSD, Ngee Ann Polytechnic, Omada Health, UCARE.AI and Visa Asia Pacific. Access it here.

Volume 2

Volume 2 contains use cases from the City of Darwin (Australia), Google, Microsoft, Taiger as well as special section on how AI Singapore implemented our Model Framework in its 100 Experiments projects with IBM, RenalTeam, Sompo Asia Holdings and VersaFleet. Access it here.

A Guide to Job Redesign in the Age of AI (Guide)

Under the guidance of the Advisory Council of the Ethical Use of AI and Data, the Infocomm Media Development Authority (IMDA) and the PDPC have collaborated with the Lee Kuan Yew Centre for Innovative Cities (LKYCIC), Singapore University of Technology and Design to launch the Singapore's first guide that helps organisations and employees understand how existing job roles can be redesigned to harness the potential of AI, so that the value of their work is increased.

Launched on 4 December 2020, this Guide provides an industry-agnostic and practical approach to help companies manage AI's impact on employees, and for organisations that are adopting AI to prepare themselves for the digital future.

This Guide provides guidance on practical steps in four areas of job redesign:
Transforming Jobs
The Guide supports IMDA's efforts to build a trusted and progressive AI environment that benefits businesses, employees and consumers. For example, the Model Framework guides organisations to deploy AI responsibly and address consumer concerns. Likewise, the Guide encourages organisations to take a human-centric approach to manage the impact of AI adoption by investing in redesigning jobs and reskilling employees.

Access the Guide here and the primer here.

Adoption and Feedback

We encourage organisations to use the Model Framework, ISAGO and Guide for internal discussion and implementation. Trade associations and chambers, professional bodies, and interest groups are welcome to use this document for their discussions, and adapt it for their own use. The way in which businesses employ AI continues to evolve and so will this living document in the form of future editions.

To this end, we welcome organisations to share with us:

Practical examples that would aid in illustrating section(s) of the Model Framework and Guide; and/or

Experiences in using the Model Framework, ISAGO and Guide, e.g. how easy it is to implement the measures, how the framework can be better improved, or a helpful case of implementation that we may publish as a use case. Your use cases would continue to inspire more companies to implement AI responsibly.

Please email us at corporate@pdpc.gov.sg.

Advisory Council on the Ethical Use of AI and Data (Advisory Council)

The Advisory Council on the Ethical Use of AI & Data was first established in 2018 for the purpose of:

(a) advising the Government on ethical, policy and governance issues arising from the use of data-driven technologies in the private sector; and

(b) supporting the Government in providing general guidance to businesses to minimise ethical, governance and sustainability risks, and to mitigate adverse impact on consumers from the use of data-driven technologies.

The members of the Advisory Council are appointed by the Minister for Communications and Information and supported by a Secretariat.

The Advisory Council members for the term 2022-2025 are:

Chairperson
Mr V.K. Rajah SC, Duxton Hill Chambers (Singapore Group Practice), former Judge of Appeal and the Attorney General of Singapore

Members (in alphabetical order)

1. Mr Blaise Aguera y Arcas, Engineering Fellow, Google Research

2. Ms Kathy Baxter, Principal Architect, Ethical AI Practice, Salesforce

3. Mr Chia Song Hwee, Deputy CEO, Temasek International

4. Mr Andreas Ebert, Worldwide National Technology Officer, Microsoft

5. Ms Kay Firth-Butterfield, Head of AI & Machine Learning and Member of the Executive Committee, WEF

6. Mr Piyush Gupta, CEO, DBS Group

7. Dr Hiroaki Kitano, President & CEO, Sony Computer Science Lab

8. Mr Shameek Kundu, Head of Financial Services and Chief Strategy Officer, TruEra Inc

9. Mr Li Chun, Group CEO, Lazada and Vice President, Alibaba Group

10. Dr Ieva Martinkenaite, Vice President of Telenor Research, Head of AI and Analytics

11. Dr Francesca Rossi, Fellow and AI Ethics Global Leader, IBM

12. Dr Tan Geok Leng, CEO, AIDA Technologies

13. Mr Andrew Wyckoff, Director of the Science, Technology and Innovation Directorate, OECD

# data[].attributes.attachment_link.description
As Singapore develops its digital economy, a trusted ecosystem is key - one where organisations can benefit from tech innovations while consumers are confident to adopt and use AI. In the global discourse on AI ethics and governance, Singapore believes that its balanced approach can facilitate innovation, safeguard consumer interests, and serve as a common global reference point.

# data[].attributes.attachment_link.description
Artificial Intelligence is rife with contradictions. It is a powerful tool that is also surprisingly limited in terms of its current capabilities. And, while it has the potential to improve human existence, at the same time it threatens to deepen social divides and put millions of people out of work.

# data[].attributes.attachment_link.readable_text
Abstract

Artificial intelligence (AI) is a salient but polarizing issue of recent times. Actors around the world are engaged in building a governance regime around it. What exactly the "it" is that is being governed, how, by who, and why-these are all less clear. In this review, we attempt to shine some light on those questions, considering literature on AI, the governance of computing, and regulation and governance more broadly. We take critical stock of the different modalities of the global governance of AI that have been emerging, such as ethical councils, industry governance, contracts and licensing, standards, international agreements, and domestic legislation with extraterritorial impact. Considering these, we examine selected rationales and tensions that underpin them, drawing attention to the interests and ideas driving these different modalities. As these regimes become clearer and more stable, we urge those engaging with or studying the global governance of AI to constantly ask the important question of all global governance regimes: Who benefits?

License

CC-BY Attribution-No Derivatives 4.0 International

Disciplines

Law Privacy Law Science and Technology Law Computer Law Law and Society Arts and Humanities Social and Behavioral Sciences Political Science International Relations Other Political Science Public Affairs, Public Policy and Public Administration Public Policy Science and Technology Policy

# data[].attributes.attachment_link.title
AI and Global Governance: Modalities, Rationales, Tensions

# data[].attributes.attachment_link.description
Artificial intelligence (AI) is a salient but polarizing issue of recent times. Actors around the world are engaged in building a governance regime around it. What exactly the "it" is that is being governed, how, by who, and why-these are all less clear.

# data[].attributes.attachment_link.description
“Amongst the myriad of opinions, one voice stands out as a rational and impartial advocate: Gary Marcus.”
—Candice Clark. Click to read Marcus on AI, by Gary Marcus, a Substack publication with tens of thousands of subscribers.

# data[].attributes.attachment_link.readable_text
With AI suddenly the buzzword that everyone is talking about, there have been several developments at the geopolitical level relating to AI and its regulation - especially in the UK. Yet, there is still a large amount of confusion in this space.

Here at the Oxford Internet Institute, University of Oxford we organised a series of events on AI, regulation, geopolitics, and digital transformation of the public sector this year. In my latest blog, I try to address some of this confusion and present my main thoughts and key takeaways from the OII events and discussions with academics, government employees, and policymakers.

Geopolitical risks of AI

Let's start with the biggest narrative today. Will AI bring about the end of the world? No, AI will not bring about the end of the world. However, AI does improve the overall efficiency and effectiveness of systems. What this means is that actors who do bad things - be it people or countries - can do bad things more efficiently and effectively than ever before. At the country level, this is something we have seen before, and which has been playing out for decades, with the internet. The internet was initially viewed as a tool of liberation [1], but studies have since shown that this is far from the truth and rather the opposite is true. Authoritarian regimes have higher rates of internet penetration than less authoritarian regimes as it improves the ability of the governing powers to control the population - it is an authoritarian ruler's dream come true [2, 3]. Unsurprisingly, countries like Russia and China are trying to take this to their model to the world by effectively utilising existing global internet governance forums to craft a more authoritarian future for the internet [4]. This has not yet happened, but the processes are taking place today. A same fate awaits AI. It is, and will continue to be, essential that the West - collectively - is able to present a unified front against a potential illiberal, authoritarian, and undemocratic future for AI. We must ensure that these systems are imbued with our core values and beliefs.

Regulating AI

This brings us to the next question. Is AI currently unregulated? If you follow the news or discussions in the media and policy circles you would expect the answer to be yes. However, this is not true. The foundations are already there. As we point out in our book chapter on the transnational regulation of AI, "what is being discussed is whether the existing framework needs optimising" [5]. The rules already exist. We have rules that, for example, say stealing intellectual property, causing loss of life, or discriminating against individuals on protected characteristics must not be done and you can be punished if you do otherwise. The questions we must be asking instead are: 1) how we can empower our existing regulators to conduct regulation in their area of expertise in as effective a way as possible? and 2) how can we enable our regulators to develop new skills and capacities to maintain or improve their ability to regulate well in the age of AI. In this regard, the UK's proposed approach to the regulation of AI by "leveraging existing activities and expertise from across the broader economy" is absolutely the way to go about this [6]. The creation of new agencies specifically tasked with the monitoring, inspection, or regulation of AI systems does not make sense - this is especially true in the context of proposals such as global "IAEA" like institutions.

Political power and AI

Is this growing political interest in AI really just about AI? Not exactly. The current interest in AI is representative of a much larger transformation related to the nature of power in the digital age. AI is a power amplifier, it allows for large amounts of information to be managed, digested, and utilised in ways not possible before. The country or government that can utilise AI the best will see large economic growth, labour growth, productivity growth, amongst other clear benefits. Therefore, we can expect to see (and already are seeing) increasing interest at the geopolitical level related to strategic control and development of the infrastructure underpinning AI such as chip manufacturing, data centres for storage and processing, data generation capabilities, space-based internet infrastructure, and submarine cable infrastructure amongst others [7].

Use of AI in the public sector

There are often questions about whether or not AI will transform our governments. The answer is probably yes, but perhaps not in the way most people expect. In order to gain advantages offered by AI, governments will increasingly attempt to integrate AI into - or create new - business processes. However, it is important to understand what increasing usage of AI in and by governments actually means. While at the governmental level there is certainly interest in citizen-facing applications and AI use cases, the biggest net-gains for the public sector will be internally focused. There will be rapidly increasing usage of AI-based systems by governmental agencies - especially in the defence sector, but not only - to improve their own effectiveness and efficiency. However, this will require new investments into digital infrastructure, digital identities, interoperability platforms, and data management systems at a minimum.

The way ahead

Ultimately, AI is a bureaucracy reinforcer, not a bureaucracy dismantler. AI will not make the state or bureaucracy go away, they will only get stronger and more effective. In order to drive this transformation, there will be increased integration between the private and public sectors as governments attempt to maintain a competitive advantage in the AI-industry. AI-systems will be procured, built, and trained by private sector companies, which will be hosted on private sector infrastructure, but will increasingly rely on private and governmentally-held and curated data sources.

AI is already here. It is not possible to turn back time. We must learn to live with it and understand what it can and cannot do.

References:

[1] Diamond, L. (2010). Liberation technology. J. Democracy, 21, 69.

[2] Deibert, R. (2015). Authoritarianism goes global: Cyberspace under siege. Journal of Democracy, 26(3), 64-78.

[3] Rød, E. G., & Weidmann, N. B. (2015). Empowering activists or autocrats? The Internet in authoritarian regimes. Journal of Peace Research, 52(3), 338-351.

[4] Daniëlle Flonk (2021). Emerging illiberal norms: Russia and China as promoters of internet content control. International Affairs, 97(6), 1925-1944.

[5] Dempsey, M., McBride, K., Haataja, M., & Bryson, J. (2022). Transnational Digital Governance and Its Impact on Artificial Intelligence. In The Oxford Handbook of AI Governance. Oxford University Press.

[6] See the white paper on this topic: https://www.gov.uk/government/publications/ai-regulation-a-pro-innovation-approach/white-paper

[7] See this great blog post by Vili Lehdonvirta on the geopolitics of digital infrastructure: https://www.oii.ox.ac.uk/news-events/news/behind-ai-a-massive-infrastructure-is-changing-geopolitics/

About the author

Dr Keegan McBride is a Departmental Lecturer in AI, Government and Policy. He is also Course Director, MSc Social Science of the Internet, Oxford Internet Institute, University of Oxford. His recent works include a co-authored chapter in The Oxford Handbook of AI Governance. Read the chapter, ' Transnational Digital Governance and Its Impact on Artificial Intelligence'.

# data[].attributes.attachment_link.title
OII | AI, Geopolitics, Regulation, and Digital Innovation - What is actually going on right now?

# data[].attributes.attachment_link.description
Let's start with the biggest narrative today. Will AI bring about the end of the world? No, AI will not bring about the end of the world.

# data[].attributes.attachment_link.readable_text
About

Dr. Keegan McBride is an expert on topics such as digital government, digital innovation, the use of AI in the public sector, digital well-being and happiness, and government interoperability and data exchange. He is an active member of the scholarly community, participating in several high-level digital government focused conferences and publishing in leading peer-reviewed digital government focused journals. In his research he aims to develop an understanding about the future trajectory of the state in the digital age by exploring the complex and co-evolutionary relationships between technology, society, and the state.

Outside of these academic interests, Keegan is actively working with NGOs, particularly those that focus on reimagining a better, open, human-centric, and equitable digital future, he can additionally be found consulting and working with governments and private sector organizations on the topic of governmental digitalization. Keegan received an MSc in e-Governance Technologies and Services and completed his doctoral studies in public administration -both at the Tallinn University of Technology in Tallinn, Estonia.

Research Interests

Interoperability, Artificial Intelligence, Artificial Intelligence in the public sector, digital innovation, open government data, digital government, digital transformation, co-creation of public services, procurement of innovation, digital wellbeing.

Areas of Interest for Doctoral Supervision

Positions at the OII

MSc SSI Course Director, January 2023 -

Departmental Research Lecturer in AI, Government, and Policy, September 2022 -

# data[].attributes.attachment_link.description
Keegan McBride is a departmental research lecturer in AI, Government, and Policy at the OII. His research creates insight about the state in the digital age by exploring the interactions between the state, society, and technology.

# data[].attributes.attachment_link.readable_text
Introduction

Impact assessments have received particular attention, on both sides of the Atlantic, as a tool for algorithmic accountability. This article aims to address how a Data Protection Impact Assessment (DPIA) (Article 35) links the GDPR's two approaches to algorithmic accountability-individual rights and systemic governance-and potentially leads to more accountable and explainable algorithms. Examining the GDPR's approach to impact assessments suggests that the scholarship has been getting explanations wrong. Algorithmic explanations should not be understood as static statements but as a circular and multi-layered process. The literature has largely to date focused on what information goes to whom and when; we argue that the impact assessment process plays a crucial role in connecting internal company heuristics and risk mitigation to outward-facing rights and in forming the substance of several different kinds of explanations.

We begin by introducing the algorithmic accountability tools in the GDPR (section 'Algorithmic accountability in the GDPR'). In section 'Individual rights in the GDPR and the multi-layered explanation', we explore the individual rights of data subjects as regards algorithmic decisions. In section 'Collaborative governance in the GDPR', we explain the GDPR's collaborative governance of algorithms. In section 'The DPIA as an algorithmic impact assessment', we explain the requirements of the DPIA under the GDPR. In section 'A model algorithmic impact assessment', we discuss the broader literature on Impact Assessments and how our interpretation of the GDPR's impact assessment requirement in fact leads to a better, more complex understanding of the GDPR's explanations of algorithmic decision-making than a focus on Article 22 alone. We close by calling for what we call a multi-layered approach to explanations, stemming from the impact assessment process.

As a methodological disclaimer: this article gives a relatively large amount of attention to the opinions and guidelines of the Article 29 Working Party, now the European Data Protection Board (EDPB), an advisory body consisting of national Data Protection Authorities. We are aware that these guidelines are not binding and cannot be considered to be the only possible interpretation of the GDPR. However, the GDPR (at Article 70) states that the EDPB is required to issue opinions, guidelines, and recommendations in order to ensure the consistent application of the GDPR. Accordingly, the interpretational activity of the EDPB is not only influential for commentators, but also for the activity of national Data Protection Authorities. Moreover, we understand such opinions and guidelines to be an essential component of the GDPR's regulatory approach, discussed further below. That approach-referred to in scholarship as 'collaborative governance' or 'new governance'-often entails broad or vague binding textual requirements, clarified over time in regulatory guidance or industry best practices or back-and-forth between regulated companies and regulators. To dismiss EDPB guidelines and opinions as non-binding 'soft law' is to overlook the central role this softer law serves in the design of the GDPR as a regulatory system.

Algorithmic accountability in the GDPR

With this background in mind, the next two sections of this article go into more detail on both the individual rights and systemic governance elements of the GDPR's approach to algorithmic accountability, before turning to the role of the DPIA in linking the two facets.

Individual rights in the GDPR and the multi-layered explanation

The core debate has primarily focused on whether or not Article 22 creates an ex post right to explanation of an individual decision made by an automated system. Our view, discussed at length by each of us elsewhere, is that it does. Automated decisions with significant effects must be made 'legible' to individuals, in the sense that individuals must be able to understand enough about the decision-making process to be able to invoke their other rights under the GDPR, including the right to contest a decision. Several of the Member States implementing Article 22(2)b of the GDPR have outlined the Article 22 explanation duties in greater detail.

Existing discussions of the Article 22 right to explanation, however, largely obscure the more complex approach to algorithmic transparency taken by the GDPR as a whole. As we discuss below in section 'Towards a model AIA', our view is that the GDPR's transparency rights are best discussed together as a system. That is, the GDPR, is best understood as establishing a system of multi-layered explanations. Individuals have a right to both a system-wide but detailed description of the logic of an algorithm (Articles 13, 14, 15), and more specific insights on individual decisions taken. We discuss layers of explanations further in this section.

Collaborative governance in the GDPR

The GDPR's approach to preventing bias and discrimination in algorithmic decision-making is another example of collaborative governance in action. Recital 71 tasks companies with preventing 'discriminatory effects on natural persons on the basis of racial or ethnic origin, political opinion, religion or beliefs, trade union membership, genetic or health status or sexual orientation' in profiling and algorithmic decision-making. The GDPR does not lay out how to do this. Instead, the Guidelines suggest that companies check data sets for bias, regularly review the accuracy and relevance of decisions, deploy systems that audit algorithms, and use 'appropriate procedures and measures to prevent errors, inaccuracies or discrimination' on the basis of sensitive data such as race, religion, or health information, deployed on a cyclical basis.

The DPIA as an algorithmic impact assessment

Within this dual system of algorithmic accountability-individual rights accompanied by extensive but collaborative governance of companies' behaviour-the DPIA plays a special role. We claim that as applied to algorithmic decision-making, the DPIA is best understood as a nexus between the GDPR's two approaches to algorithmic accountability. Understanding it in this way allows us to better understand what is or might be required, and to observe the tool's potential shortcomings as implemented in the GDPR.

The Guidelines on ADM interpret the GDPR to mandate DPIAs for all ADM with significant effects. Article 35(3)(a) requires a DPIA for 'a systematic and extensive evaluation of personal aspects relating to natural persons which is based on automated processing ... on which decisions are based that produce legal effects concerning the natural person or similarly significantly affect the natural person' (emphasis added). As Casey, Farhangi, and Vogl have noted, 'demonstrating that a DPIA is not necessary will, in many instances, itself require a DPIA'. We note, too, that at least one Member State, Slovenia, requires algorithmic impact assessments (AIAs) for ADM under Article 22(1) of the GDPR.

In this section, we address the DPIA as an AIA. We identify what the purpose of the DPIA is in the GDPR, and what it must include. Understanding the DPIA's purpose in algorithmic governance both clarifies what the content should be and points to several shortcomings in the current conception of it. The GDPR's DPIA will serve, in the ADM context, as an AIA. It thus may prove to be an example for governments around the world considering using impact assessments as a tool to achieve algorithmic accountability.

What is required in a DPIA?

In this section, we discuss the GDPR's requirements for a generic DPIA, before turning in the next sections to requirements specific to algorithmic decision-making. The GDPR requires DPIAs only under certain circumstances. The GDPR describes a DPIA as 'an assessment of the impact of the envisaged processing operations on the protection of personal data' (Article 35). That assessment, per the text of the GDPR, must include: a description of the 'processing operations' (in this case, the algorithm) and the purpose of the processing; an assessment of the necessity of processing in relation to the purpose; an assessment of the risks to individual rights and freedoms; and importantly, the measures a company will use to address these risks and demonstrate GDPR compliance, including security measures (Article 35(7)) (Recitals 84, 90).

The DPIA must occur before a company implements a system. That is, a company must assess a system and propose risk-mitigation measures, before data processing takes place (Article 35(1)). But the GDPR also envisions iteration. For example, if the risk posed by a system changes, a company must assess whether it is complying with its own Impact Assessment (Article 35(11)). It should also under such circumstances review and possibly revise the DPIA itself.

As for other forms of external oversight, the Guidelines recommend but do not require seeking advice from independent experts, ranging from lawyers and sociologists to data security experts. The GDPR does not generally require most DPIAs to be overseen by a public authority (the Data Protection Authority). But if a risk assessment indicates that processing would result in high risk in the absence of measures taken by the controller to mitigate the risk, then a company must consult with the regulator before processing (Article 36). Thus a company effectively decides itself whether it should be subject to regulatory oversight, as part of the impact assessment process.

What is the purpose of a DPIA, in the context of the GDPR's algorithmic governance?

Understanding the DPIA in this way both clarifies its potential content and leads us to observations about how the DPIA as an AIA might be implemented and even improved. The DPIA is not a perfect AIA. As a tool in the GDPR's overall algorithmic governance regime, however, it has more potential than might initially meet the eye.

How understanding the DPIA's dual role helps clarify its content

The DPIA has two roles: as a tool in the GDPR's systemic (and collaborative) governance regime, and as an element of the GDPR's protection of individual rights. Understanding the DPIA in this way-as a connection between the two regulatory subsystems-lets us better understand how it is meant to function as an AIA, to the extent of further clarifying its content. It also leads us to some insights in the next section ('Towards a model AIA') about the layers of algorithmic explanations produced by, and to be released according to, the GDPR.

The DPIA also, however, has an unexplored role in the GDPR's system of individual rights.

First, the DPIA can serve as a source of material for the much-discussed disclosures to individuals about algorithmic decision-making: the individual notification and access rights. Remember, data subjects have a right to receive 'meaningful information' about the 'logic involved, as well as the significance and the envisaged consequences' of ADM (Articles 13, 14, 15). A DPIA must contain, as mentioned above, 'a systematic description of the envisaged processing operations and the purposes of the processing...' (Article 35(7)). If companies must already internally describe ADM at a systematic level as part of the DPIA process, those internal descriptions could be disclosed to individuals, or at least serve as the basis for these disclosures, in addition to being released to the public in the form of summaries.

Similarly, a DPIA must include an assessment of 'the risks to the rights and freedoms' of individuals, and individuals have a right in the context of ADM to be informed of the 'significance and envisaged consequences' of such decision-making (Articles 35, 13, 14, 15). Again, as a company conducting ADM must conduct a DPIA, it should consider how the information it produces in that process might also feed into or even satisfy the individual rights requirements under the GDPR.

In other words, we might interpret the GDPR provisions on the DPIA as serving as a form of commitment-making to protecting, or even enabling, individual algorithmic due process rights. By characterizing these individual rights as risk-mitigation measures, the Guidelines on ADM both provide a substantive backstop as to what must be included in a DPIA, and task companies with constituting -through the process of performing a DPIA-what these individual rights will look like in practice. Thus the DPIA serves as a means of expanding company commitments, changing company decision-making heuristics to include an assessment of individual due process rights. It simultaneously serves as a collaborative governance mechanism used to involve companies in constituting the substance, in practice, of individual due process rights.

Conceptually, the implications of this are even broader. By characterizing third-party and expert oversight as a form of 'suitable safeguard' or 'suitable measure' to protect individual rights in the face of ADM, the recommendation in the Guidelines on ADM links individual rights protection with collaborative governance techniques. Companies are tasked with coming up with ways to prevent error, bias, discrimination, and other harms to individual rights, and external oversight is imposed over how they choose to address these problems. That external oversight itself is also conceptualized as a crucial aspect of individual rights in the GDPR, standing in for individuals to ensure that they are not subjected to an unfair, arbitrary, discriminatory, or erroneous system.

Shortcomings of the DPIA

This failure could be drastic. The GDPR puts a lot of faith in the behaviour of companies and in the capacity of regulators. As discussed, the GDPR often tasks companies with coming up with the substance of (i) how individual rights will be implemented and (ii) how to address unfairness, biases, and discrimination-related concerns about algorithms. In the absence of public oversight, how can we be sure that this hybrid system of individual rights and collaborative governance is working towards the public good?

Individual notification and access rights could do some of the necessary accountability work for the GDPR's attempts at collaborative governance. This is somewhat more convincing. If companies indeed link their DPIA content to what they disclose to individuals (for example, disclosing the systemic description of processing uncovered during a DPIA to individuals as the 'logic involved' in a decision-making system), then it is likely that these disclosures will make their way to other third parties, who may be able to provide the expertise and oversight over company self-governance. For example, an individual who feels she has been discriminated against might disclose the information she has received about a system's decisions to a civil society group, which could in turn help publicize the story and the information, triggering market mechanisms or regulatory feedback from the public or oversight by external experts. This is, however, a more attenuated way of getting at the same outcome as public disclosure, and risks failing entirely if companies significantly disaggregate the DPIA process from individual disclosure rights.

A model AIA: towards multi-layered explanations

From examining the GDPR's DPIA mechanism generally to discussing the DPIA as an AIA, we now turn to imagining a more ideal DPIA.

We close this article with a call for more work on establishing a model AIA that could serve as a basis for what we call multi-layered explanations of algorithmic decision-making. This will involve interdisciplinary efforts: technologists to assess what risk-mitigation and accountability measures could be implemented, and lawyers and ethicists to think through how to better involve constituents and define problems. It will also involve a deeper exploration of how to link the material created during the DPIA process to the individual disclosures required under the GDPR.

Proposals for AIAs

We now turn to recent proposals for AIAs, which draw to varying degrees on these precursors. We find both common threads and significant differences in the proposals. We also find a significant gap in this literature that our perspective on the GDPR helps to fill.

Lessons for calls for AIAs generally

Our GDPR-specific analysis has implications for proposals for AIAs generally. Our research into the GDPR's version of an AIA suggests that the proposals discussed above have largely missed several important observations.

Thirdly, as part of a larger system of governance, there are unexplored connections between the GDPR's DPIA and its underlying substantive individual rights and substantive principles. It is true that many of the GDPR's individual rights and principles about algorithmic decision-making are articulated in broad, sometimes aspirational, terms. Unlike an EIS, the GDPR's version of the AIA has a substantive backstop in, for example, Recital 71's admonishment that a data controller should minimize the risk of error and prevent discriminatory effects. The oddity is the GDPR's circularity: the AIA helps not just to implement but to constitute both these substantive backstops and the GDPR's individual rights. Thus, there is a substantive backstop to company self-regulation through impact assessments-but it is a moving target, in part given meaning by affected companies themselves.

Towards a model AIA: producing multi-layered explanations

We close with a framework for a model AIA under the GDPR. More research is clearly needed in both the technological and policy space to implement this in practice. But we hope to start that conversation here. Our envisioned model AIA is informed both by our understanding of the GDPR and by our overview of impact assessment proposals above. Like previous AIA proposals, we emphasize stakeholder input, expert oversight, and public disclosure as essential elements of an effective impact assessment. Unlike previous AIA proposals, we deploy our understanding of the GDPR to suggest how the DPIA process informs and connects into the GDPR's system of individual rights, through disclosures that we refer to as multi-layered explanations. Thus, our conclusions inform not only the ongoing policy discussion of impact assessments, but the ongoing debate about individual algorithmic transparency rights in the GDPR.

Elements of a model AIA

As to substantive risk-mitigation measures, different data controllers may have different duties. Article 24(1) of the GDPR states that taking into account the nature, scope, context, and risks of data processing, the controller shall implement appropriate technical and organizational measures to ensure compliance with the GDPR. Accordingly, algorithmic decision-making involving bigger risks for data subjects should entail more safeguards. As discussed, data controllers to a certain extent choose their own algorithmic accountability safeguards: some are required in the GDPR both at Article 22(3) and at Recital 71 (algorithmic auditing, the rights to contest, to have a new decision, to a human in the loop, and to explanation), but these are not closed lists, and the Guidelines on ADM suggest additional techniques. In the case of more intrusive and riskier ADM processes, the data controller should and likely will implement all possible safeguards, including a right to explanation of an individual decision.

To make the Impact Assessment process meaningful, Data Protection Authorities must be willing to spot check and enforce against captured versions of it. While the GDPR does not require regulatory involvement in all DPIAs, DPAs could use the GDPR's broad information-forcing powers to inspect particular companies and check for compliance. This spot-checking might work not just to monitor and improve the efficacy of the process, but to identify substantive problems with algorithmic decision-making. DPAs might over time use what they have learned to establish more concrete best practices or support the establishment of sector-specific codes of conducts around algorithmic fairness, as suggested in the Guidelines on ADM. Several implementing Member States have already put in place substantive backstops around algorithmic decision-making, prohibiting decision-making based on particular factors, or that is discriminatory or biased. Slovenia, as mentioned, couples this substantive prohibition against discrimination with a required impact assessment process. This dual approach of linking impact assessments to substantive prohibitions may help to tether internal company risk mitigation measures to the public good.

Comparing DPIA requirements with algorithmic accountability requirements under the GDPR

The more granular but nonetheless central insight arising from our research is this: the DPIA process can and should inform the substance of the GDPR's individual algorithmic accountability rights. That is, when we take a close look at the substance a company or public agency is required to produce during a DPIA, it maps surprisingly well on to the disclosure requirements of the GDPR.

Article 35(7) GDPR requires that a DPIA should contain:

1. a systematic description of the envisaged processing operations and the purposes of the processing, (...)

2. an assessment of the necessity and proportionality of the processing operations in relation to the purposes

3. an assessment of the risks to the rights and freedoms of data subjects (...); and the measures envisaged to address the risks, including safeguards, security measures and mechanisms to ensure the protection of personal data and to demonstrate compliance with this Regulation taking into account the rights and legitimate interests of data subjects and other persons concerned.

This required content of a DPIA largely maps onto the GDPR's individual algorithmic transparency rights. For example, where Article 35(7) requires a DPIA to contain a 'systematic description of the envisaged processing operations', this could be used as the foundation for the GDPR's disclosure requirement that individuals must be informed of 'meaningful information about the logic involved' in algorithmic processing. The Guidelines on DPIAs interpret the systematic description to include: the nature, scope, context, and purposes of the processing; categories of personal data, recipients, and storage; and a functional description of the processing operation and the assets on which personal data rely. Any or all of this systematic description of processing produced during a DPIA could feed into what is disclosed to individuals regarding 'meaningful information about the logic involved'.

If we compare the GDPR's transparency requirements for ADM with this and other substantive requirements for DPIAs, the similarities are striking (see Table 1).

The Guidelines on ADM only make these parallels more apparent. The Guidelines on ADM once again echo the substantive requirements of a DPIA ( Table 2).

That is, there are numerous similarities between information that a data controller is required to produce during a DPIA and what a data controller is required to release to individuals as part of the GDPR's algorithmic transparency duties. The data controller's duty to systematically describe the processing operations in a DPIA is similar to the algorithmic transparency duty to clarify the categories of personal data used in ADM and how algorithmic profiling is built. The controller's duty to assess the necessity and proportionality of the processing operations in a DPIA is similar to the algorithmic transparency duty to explain the pertinency of personal data used and the relevance of the profiling. The controller's duty to assess the data processing risks and impacts on individuals is similar to the transparency duty to explain the impact of the profiling use in ADM. Lastly, the controller's duty to establish safeguards of individual rights in the case of ADM is similar to the duty to find and describe measures envisaged to address the risks in DPIA.

In other words, in the case of ADM, the outputs of the DPIA process described in the GDPR appear to correspond to algorithmic transparency duties in the GDPR (as interpreted by Article 29 Working Party (now the EDPB)).

Towards multi-layered explanations from an algorithmic DPIA

Our perspective on the DPIA as linking systemic governance to individual rights thus has implications for the GDPR's overall approach to algorithmic explanations. The DPIA process in our view suggests what we call ' multi-layered explanations ' for ADM. These explanations will likely be crafted as part of the DPIA process and should be released either directly to the public or to affected individuals.

With our perspective on the GDPR's two-pronged approach to algorithmic accountability, and our emphasis on the role of the DPIA, we understand there to be more layers: individual explanations, group explanations, and systemic explanations, both internal and external. And unlike Edwards and Veale, we have more optimism that these multi-layer explanations can be grounded either in the text or subtext of the GDPR.

Moreover, there may be an argument for the disclosure of group- or location-based explanations to individuals as part of the GDPR's individual transparency rights regime. That is, even if DPIAs are not required to be made public, and even if companies decide not to disclose to the public what they discover about the impact of algorithmic decision-making on particular groups, they may nonetheless have to do so to impacted individuals under Article 22.

Finally, some scholars have remarked that what is needed is not merely an explanation, but a legal justification of automated decisions taken. The full concept of justification is not easy to address in the data protection framework and is beyond the scope of this article. For the limited scope of this article, however, justifying a decision means not merely explaining the logic and reasoning behind it, but also explaining why it is a correct, lawful, and fair decision, ie that the decision is based on proportional and necessary data processing, using pertinent categories of data and relevant profiling mechanisms.

Again, connecting the DPIA to transparency requirements may clarify what this could mean. Language about the DPIA process suggests that in addition to technical explanations of a model, data controllers should produce justificatory explanations of a system during a DPIA. Under the DPIA process, data controllers must prove the legal proportionality and necessity of the data processing, and thus the legal necessity and proportionality of eventual automated decisions taken (Article 35(7)(d)). This may constitute a form of justification for data use and profiling mechanisms. Similarly, the Guidelines on ADM recommend that data controllers (in order to comply with Articles 13-15) explain the pertinence of categories of data used and the relevance of the profiling mechanism. Assessing whether the data used are pertinent and the profile is relevant for a decision, as well as assessing the necessity and proportionality of the data processing in an ADM system, seems to constitute a type of justification of automated decision systems. The purpose of such assessment is not just transparency about the technology and its processes, but an explanation about the lawfulness, fairness, and legitimacy of certain decisions.

1. they could optimize efforts that would otherwise be spent on two different tasks (the DPIA and disclosure requirements) by taking compliance with DPIA duties (Article 35) and feeding them into transparency duties as imposed by Article 13-15 (and 22) of the GDPR;

2. publicly disclosing (at least some parts) of the DPIA as a basis for explaining automated decisions is considered a best practice recommended in the DPIA framework, in line with the data protection by design principle (Article 25 GDPR);

3. disclosing information about algorithmic data processing to data subjects and collecting their reactions (through, eg the right to contest, to have a new decision, to have human involvement, etc.) could be considered compliant with the duty to seek the view of impacted data subjects (Article 35(9) GDPR), in the continuous cycle of the DPIA framework;

4. in general terms, the dynamic merging of an algorithmic DPIA with multi-layered explanations might be a 'suitable safeguard' to protect fundamental rights and freedoms of individuals both under Article 22(3) and under Article 35(7)(d) of the GDPR;

5. developing an algorithmic DPIA and explanation safeguards in parallel (intrinsically related to the right to contest a decision, right to a human-in-the-loop, etc.) might be the best way to enrich transparency with accountability safeguards and overcome the 'transparency fallacy' through a virtuous cycle of algorithmic auditing and continuous detection/mitigation of unfair effects.

The idea of at least partially merging algorithmic accountability duties with the DPIA process also seems useful considering the most advanced literature on explanations. As discussed above, a multi-layered and multi-step explanation would be a continuous process, not merely a product.

Conclusion

There is a growing literature suggesting that AIAs are a crucial tool in establishing algorithmic accountability. This paper addresses that tool as it is implemented in the GDPR. We find that the GDPR's version of an AIA serves as a central connection between its two approaches to regulating algorithms: individual rights and systemic governance. That framing allowed us to identify both value in and shortcomings of the GDPR's Impact Assessment regime as applied to algorithmic governance.

This analysis, we hope, will have value for other discussions of AIAs beyond the GDPR. In particular, moving from individual transparency rights and governance accountability duties in the field of ADM, we suggest a model of multi-layered explanations drawn from an impact assessment process. Since there are several layers of algorithmic explanation required by the GDPR, we recommend that data controllers disclose a relevant summary of a system, produced in the DPIA process, as a first layer of algorithmic explanation, to be followed by group explanations and more granular, individualized explanations. More research is needed, in particular about how different layers of explanations-systemic explanations, group explanations, and individual explanations-can interact each other, and how technical tools can help in developing an AIA that might be re-used towards GDPR-complying explanations and disclosures.

The authors, in alphabetical order, have contributed equally to this work. This work was funded under Fullbright Schuman Program [to Kaminski], and EU Horizon 2020 Project PANELFIT, Grant Agreement n. 788039. No conflict of interest to declare.

Author notes

# data[].attributes.attachment_link.title
Algorithmic impact assessments under the GDPR: producing multi-layered explanations

# data[].attributes.attachment_link.description
Key PointsPolicymakers, scholars, and commentators are increasingly concerned with the risks of using algorithms for profiling and automated decision-making.Thi

# data[].attributes.attachment_link.title
Evaluating the Social Impact of Generative AI Systems in Systems and Society

# data[].attributes.attachment_link.description
Generative AI systems across modalities, ranging from text, image, audio, and video, have broad social impacts, but there exists no official standard for means of evaluating those impacts and which impacts should be evaluated.

# data[].attributes.attachment_link.readable_text
𝗛𝗮𝗺𝗯𝘂𝗿𝗴 𝗗𝗮𝘁𝗮 𝗣𝗿𝗼𝘁𝗲𝗰𝘁𝗶𝗼𝗻 𝗔𝘂𝘁𝗵𝗼𝗿𝗶𝘁𝘆: 𝗖𝗵𝗲𝗰𝗸𝗹𝗶𝘀𝘁 𝗳𝗼𝗿 𝘁𝗵𝗲 𝘂𝘀𝗲 𝗼𝗳 𝗟𝗟𝗠-𝗯𝗮𝘀𝗲𝗱 𝗰𝗵𝗮𝘁𝗯𝗼𝘁𝘀 (𝘀𝘂𝗰𝗵 𝗮𝘀 𝗖𝗵𝗮𝘁𝗚𝗣𝗧) On November 13, 2023, the DPA in Hamburg published a checklist intended to serve as a guide for companies and authorities on the data protection-compliant use of LLM-based chatbots. Below are some of the guidelines drawn up by the supervisory authority: - Clear and documented internal instructions are to be formulated as to whether and under what conditions which tools can be used. Specific examples of permitted and prohibited usage scenarios help to clarify this. - Professional chatbot accounts should be made available. Employees should not create an account independently and using private data. If use in a professional context is desired, professional accounts should also be made available. If possible, these work accounts should not contain the names of individual employees. It is advisable to use an e-mail address created specifically for this purpose. - As a general rule, if the provider of a chatbot has allowed the chatbot to use data for its own purposes in the terms and conditions, no personal data may be transmitted to the AI. This applies to any information that could be used to identify customers, business partners or other third parties, as well as data relating to the company's own employees. - Entries that could possibly be related to specific persons should be avoided. It is not sufficient to remove names and addresses from the entry. - The option to reject the use of data for training purposes should be used. Manufacturers of AI models often use all the data entered for further training of their AI. Private individuals and employees of other companies can then "request" this content. - Decisions with legal effect should generally only be made by humans. Otherwise, the requirements of Art. 22 GDPR must be observed. If an LLM-based chatbot develops suggestions that are accepted by employees, they must ensure that they have an actual scope for decision-making. - It is also advisable to regulate other aspects such as the protection of copyrights or trade secrets. Guidelines (German): https://lnkd.in/eBkpe9ey#DSGVO #AI #KI #dataprotection #GDPR

# data[].attributes.attachment_link.title
Dr. Carlo Piltz on LinkedIn: Checkliste zum Einsatz LLM-basierter Chatbots

# data[].attributes.attachment_link.description
𝗛𝗮𝗺𝗯𝘂𝗿𝗴 𝗗𝗮𝘁𝗮 𝗣𝗿𝗼𝘁𝗲𝗰𝘁𝗶𝗼𝗻 𝗔𝘂𝘁𝗵𝗼𝗿𝗶𝘁𝘆: 𝗖𝗵𝗲𝗰𝗸𝗹𝗶𝘀𝘁 𝗳𝗼𝗿 𝘁𝗵𝗲 𝘂𝘀𝗲 𝗼𝗳 𝗟𝗟𝗠-𝗯𝗮𝘀𝗲𝗱 𝗰𝗵𝗮𝘁𝗯𝗼𝘁𝘀 (𝘀𝘂𝗰𝗵 𝗮𝘀...