# data[].attributes.attachment_link.readable_text
Tackling the COVID-19 pandemic has become one of the world's biggest challenges, with governments and private actors joining forces in finding innovative solutions to accelerate the post-pandemic recovery. The European Union (EU) has adopted the Recovery and Resilience Facility that aims to mitigate the social and economic impact of the pandemic through large-scale financial support of 672.5 billion EUR for investment and reforms.

In the healthcare sector, the EU has launched an ambitious plan to respond to the effects of the coronavirus, namely EU4Health 2021-2027, with a total investment of 5.1 billion EUR. One of its core elements would be to advance digital transformation within healthcare systems, through specialized funding schemes and by boosting the EU's readiness to overcome cross-border health threats. The EU is leading the international response to the COVID-19 pandemic and the launch of the Rome Declaration by the G20 leaders and other states, during the Global Health Summit on May 21 st is a milestone on that path Some of the principles of the declaration emphasize the need for an increased use of healthtech, facilitating data sharing or know-how transfers to boost digital transformation within the healthcare sector. As the study "The rise of digital health technologies during the pandemic", conducted by the European Parliament Research Service shows, the crisis had a remarkable contribution in accelerating the transition towards a digitalized healthcare system. However, there is still a risk of widening the "digital divide" through a rapid technological advancement that is not yet accessible to all social groups.

After more than one year of implementing digital solutions to offer a swift and targeted response to the pandemic, how can we assess the impact of different emerging technologies in transforming digital healthcare? Was there an overpromise of adopting such technologies to tackle the pandemic, or do the use-cases show the opposite? And what is the potential for scaling and building a digitally-powered healthcare system?

The technological trends of the healthcare sector are the following:

The role of automation in the healthcare sector evolved exponentially during the COVID-19 pandemic, as providers of medical services increased the use of automation within their work processes. Due to the extraordinary pressure generated by the pandemic, the healthcare sector faced challenges that hindered or extensively delayed the workflows of routine medical procedures such as maintaining and handling millions of registers, removing errors from health data, exchanging patient data and ensuring interoperability etc., which will remain with us for a while.

Robotic Process Automation (RPA) and Intelligent Automation (IA) proved to be vital in delivering efficiency, supporting organizations all over the world to keep their operations going during the global health crisis. Especially in the healthcare system, RPA and IA technologies add various benefits for work efficiency and accuracy, while allowing medical staff to focus on higher value tasks. Thus, the use of automation in the healthcare sector presents advantages such as:

Handling impressive volumes of patient requests;

Reducing healthcare data errors;

Providing better conditions for patients, including management of their files or improved communication;

Improving ordering and billing;

Supporting early disease detection;

Increasing precision for diagnosis.

Enterprise automation company UiPath strives to cure healthcare inefficiencies with automation by reducing burdensome tasks that clutter up healthcare systems, improving workflows, offering patients single-point access to their information (including medical histories, billing information, medical appointments etc.) and reducing supply chain costs. Some of the use cases of the RPA solutions implemented in the healthcare sector include: vaccine management (including regulatory reporting, vaccine scheduling, and mass vaccination appointments or supply chain management), automating the process of a leading US healthcare service provider or streamlining high volume processes (claims processing, data entry, reconciliation for government healthcare schemes) for one of the largest hospital networks in North India;

The United Kingdom National Health Service (NHS) considers applying intelligent automation to core clinical processes as a top priority, given the fact that it can reduce the burden of employees and be cost-effective at the same time.

Robots became essential in providing a response to overcome the effects of the pandemic. They don't operate autonomously in all cases, as they require human supervision and they can perform simple and repetitive tasks. Especially during the COVID-19 crisis, the deployment of robots increased to help with monitoring patients, delivering medication and supplies, but also sanitizing clinical spaces and being extremely useful for frontline medical workers.

One important distinction should be made with regard to robotics versus traditional forms of automation. While robotics refers to designing, creating and using robots that will perform a specific task, automation involves physical machines and computer software to perform tasks which are typically done by humans. Moreover, RPA is totally different from robotics, as the technology uses software robots to perform actions on an application's user interface and it's extremely efficient in automating rule-based tasks.

Telepresence of robots deployed in hospitals was incredibly useful in allowing doctors who were home based or in quarantine continue their activity, being able to offer support and treatment to patients in emergency rooms and lowering the risks of additional infections;

Zipline's autonomous drones are delivering blood products and medication during the pandemic. The company acts as a centralized distribution network for COVID-19 supplies in Ghana and Rwanda and also tries to expand it in the United States, by working with the Federal Aviation Administration.

Artificial Intelligence (AI) solutions are being used in complementarity with data analysis to make predictions on the spread of the pandemic, but it can also determine aspects related to how patients are being treated. AI algorithms have the capacity of analyzing a sheer amount of data to provide early diagnosis, accurate and personalized medical treatment, and to determine a medical framework for prevention and optimal disease management.

There are still multiple ethical concerns on the use of AI in healthcare, part of them being addressed in the first regulatory proposal on AI at the EU level - Artificial Intelligence Act. Given the fact that AI systems used for medical devices might present a high risk for consumers' fundamental rights, there will be specific requirements for ensuring compliance with the new obligations proposed in the AI Act (which has to be adopted through the EU ordinary procedure).

Healthcare systems need to prove their resilience while also achieving a solid framework for critical infrastructure. This will be particularly important for ensuring data security and safety, especially as medical data is considered sensitive data under data protection rules such as General Data Protection Regulation (GDPR). Thus, it is vital to provide reliable mechanisms against potential data breaches, data loss and cybersecurity threats, including cyberattacks, ransomware, phishing, or socially engineered malware.

As the healthcare industry is highly regulated, to protect patients' privacy and security, several solutions need to be available to achieve robust healthcare systems, such as: backup storage of data, cloud services and encryption to prevent data misuses.

5. Monitoring COVID-19 with digital technologies

Undoubtedly, technology acted as a catalyst for healthcare innovation. The possibility to monitor and track the spread of the pandemic with digital technologies enabled governments and companies alike to implement resourceful strategies to mitigate the negative effects of coronavirus.

South Korea, one of the global innovation and technology front runners, implemented a government-coordinated containment and mitigation strategy relying on technology, using contact tracing apps, facial recognition tools, security cameras tools and global positioning system (GPS) to monitor developments of new cases of COVID-19, but also to identifying and isolating infections early. These methods allowed South Korea to remain among the countries with the lowest per-capita mortality rates worldwide;

Google and Apple joined forces to develop the Exposure Notifications System, which enables users to find out if they have been exposed to someone having COVID-19 and there are already reports that show how this "contact tracing tech has saved lives of between 4,200 and 8,700 users in the UK";

6. Telemedicine (virtual care) & teleconsultations

The spread of the pandemic incentivized a swift transition to a digitally-enabled way of providing healthcare services, with numerous benefits for both patients and doctors. Many countries worldwide relied on telemedicine and teleconsultations to continue delivering healthcare services remotely, limiting the medical staff exposure to the virus. Thus, telemedicine has proved to have multiple benefits, including its convenience, affordability and capability of high-quality outcomes.

In the United States, there are initiatives meant to increase the number of telehealth visits such as "decisions to cover them by Medicare, Medicaid and many private insurers, some of whom have even waived co-pays and deductibles for some visits, especially for ones related to COVID-19 symptoms";

In Europe, the teleconsultations market is doubling or tripling every year, while the global telehealth market is expected to reach over 80 billion dollars in 2021 and to grow up to 218.5 billion dollars by 2025. As well, a study conducted by Ernst & Young (EY) shows that the "use of telephone and video exploded from 20% to over 80%", which encourages "the vast majority of physicians (81%) (...) to accelerate their introduction of new digital technologies".

Technological advancement is a cornerstone in reshaping our economies and societies in the aftermath of the COVID-19 pandemic. There are already remarkable contributions by several technologies delivering innovation and improvements that will transform the healthcare sector and enable the digital transition. From automation to AI, from cybersecurity to data analytics, as well as other new digital tools, we will see the healthcare industry transform and reinvent itself gradually as it embraces the opportunities brought by digital technologies.

Cezara Panait is the Head of Digital Policy at the think-tank Europuls - Centre of European Expertise, where she is leading the research and policy activity on emerging technologies, including automation, artificial intelligence and digital platforms. She frequently publishes articles and moderates high-level debates with policy-makers on these topics.

# data[].attributes.attachment_link.title
Six Technologies Bringing Digital Transformation in Healthcare - The European Business Review

# data[].attributes.attachment_link.description
By Cezara Panait Tackling the COVID-19 pandemic has become one of the world's biggest challenges, with governments and private actors joining forces in finding innovative solutions to accelerate the post-pandemic recovery. The European Union (EU) has adopted the Recovery and Resilience Facility that aims to mitigate the social and economic impact of the pandemic through ...

# data[].attributes.attachment_link.readable_text
Tehnologiile emergente avansează rapid, așa că este nevoie de reguli pentru acestea. Uniunea Europeană propune primul cadru legal din istorie pentru inteligența artificială. Cum este abordată reglementarea AI la nivel global?

Inteligența artificială (AI) e deja prezentă în viața de zi cu zi și poate ajunge chiar să influențeze comportamentul social și pe cel din mediul online. „Technology and Democracy: Understanding the influence of online technologies on political behaviour and decision-making", ec.europa.eu Sistemele AI realizează analize și predicții pe baza unor seturi de date complexe, astfel încât să poată conduce la decizii relevante, într-un timp foarte scurt. Acest lucru îmbunătățește semnificativ eficiența și productivitatea muncii și tinde la obiectivul de a face predicții imparțiale, spre deosebire de oameni, care au întotdeauna o urmă de subiectivitate în deciziile luate. Totuși, și algoritmii AI pot replica anumite stereotipuri sau prejudecăți existente în societate. Se creează astfel riscul ca decizia automată să încalce drepturile utilizatorilor (non-discriminarea, libertatea de exprimare, accesul la informații, dreptul la viață privată și protecția datelor etc.).

crearea de deepfakes „Deepfakes and the New AI-Generated Fake Media Creation-Detection Arms Race", scientificamerican.com sunt doar câteva exemple în care utilizarea AI-ului poate duce la încălcarea unor drepturi.

Ce prevede viitorul Regulament european pentru inteligența artificială?

Înainte de a stabili noile obligații pentru statele membre, UE a încercat să determine care sunt riscurile pe care le prezintă diverse categorii de sisteme AI pentru drepturile fundamentale ale utilizatorilor. Sistemele au fost categorisite în mai multe graduri de risc.

CITEȘTE ȘI: Etică și AI. Putem avea încredere în roboți?

Aplicațiile AI care prezintă un risc inacceptabil pentru libertatea sau integritatea fizică a persoanelor vor fi interzise. În această categorie intră aplicații care pot manipula, în mod subliminal, comportamentul utilizatorului pentru a-i afecta libertatea de decizie (de la jucării care încurajează copiii, prin sisteme audio inteligente, să aibă un comportament periculos până la sistemul de acordare a scorului social de către guverne). De asemenea, identificarea biometrică în spații publice în timp real va fi interzisă, cu anumite excepții limitate.

O serie întreagă de aplicații prezintă un risc crescut. În această categorie intră sistemele de transport care pot pune în pericol viața și sănătatea persoanelor - spre exemplu, mașinile fără șofer care folosesc AI și, în ipoteza unui potențial accident, ar trebui să ia o decizie care poate salva sau pune în pericol viața persoanelor. De asemenea, este deja dovedit că procedurile de angajare și recrutare automate, care folosesc algoritmi AI pentru analizarea CV-urilor și ierarhizarea candidaților, au discriminat anumite categorii de candidați pe baza sexului, rasei, vârstei, etniei sau chiar a statutului social, deci vor intra și ele în categoria de risc crescut.

Risc crescut au și serviciile esențiale private și publice, precum cele care analizează acordarea unui credit bancar și îl pot refuza pe baza unui scorul social - și în această situație a fost identificată tendința algoritmilor AI de a discrimina. Acest lucru e deja interzis de Articolul 22 din Regulamentul General privind Protecția Datelor, care interzice aplicarea unor decizii automatizate, deși nu menționează implicit termenul de inteligență artificială. „Știai că inteligența artificială este reglementată în România de 19 ani?", ; „Cum impactează GDPR noul regulament AI" ,

De asemenea, utilizarea inteligenței artificiale în sistemul de justiție poate pune în pericol drepturile și libertățile fundamentale, precum și folosirea în procedurile de migrație, azil și control al granițelor prin verificarea automată a autenticității documentelor.

Chiar și sistemele AI care au un risc limitat (de exemplu, asistenții virtuali de tip chatbot) vor avea obligații specifice de transparență, astfel încât utilizatorii să știe că interacționează cu un robot și să poată lua decizii în consecință.

Propunerea legislativă nu va reglementa folosirea aplicațiilor cu un risc minim pentru utilizatori, precum jocurile video care folosesc inteligența artificială sau filtrele care detectează mesajele spam, categorie unde se vor încadra majoritatea sistemelor de AI, întrucât prezintă pericole minore sau nu implică astfel de riscuri.

Deși nu este infailibilă, modalitatea de a institui obligații și interdicții pe baza încadrării sistemelor AI în categorii de risc pare a fi cea mai bună soluție. Aceasta a fost avută în vedere încă de la elaborarea documentului strategic „Conturarea viitorului digital al Europei: Comisia prezintă Strategia privind datele și Strategia privind inteligența artificială" Cartea albă privind inteligența artificială (EU White Paper on AI) , ec.europa.eu din februarie 2020.

Pe lângă intervenția legislativă orizontală, care completează normele deja existente din sfera protecției consumatorului, a respectării datelor personale și a politicilor privind non-discriminarea și egalitatea de gen, Actul prevede și elaborarea unor angajamente voluntare sau coduri de conduită. Companiile vor putea adopta măsurile de prevenire a efectelor negative ale utilizării AI-urilor din aceste coduri sau angajamente, astfel încât să se creeze un cadru flexibil și adaptabil.

Această propunerea de a implementa mecanisme de (instrumente cvasi-legale, cu o forță juridică scăzută) sau chiar angajamente voluntare din partea actorilor privați este susținută și la nivel global, spre exemplu, în Principiile directoare ale ONU privind mediul de afaceri și drepturile omului. „Guiding Principles On Business And Human Rights", (PDF) Aceste principii prevăd măsuri prin care companiile pot conduce la respectarea drepturilor omului, spre exemplu prin efectuarea evaluărilor de impact pentru produsele pe care le realizează, obligații de diligență etc. Astfel, aplicarea unor asemenea coduri de conduită în privința sistemelor AI poate avea un real efect pozitiv în mediul digital.

Care sunt provocările propunerii UE de reglementare a sistemelor AI?

Întotdeauna, progresul tehnologic a funcționat astfel: mai întâi au apărut noi produse și servicii și abia ulterior a fost resimțită necesitatea reglementării acestora. Așa se întâmplă și în contextul actual, al unui mediu digital extrem de dinamic și inovator, în care există o varietate foarte mare de aplicații care folosesc AI. Astfel, pe de o parte există un pol al certitudinii juridice, care impune standarde ridicate de protecție pentru utilizatori și obligații pentru companii, iar pe de altă parte se regăsește dorința mediului privat de a inova, de a aduce noi soluții tehnologice în prim-plan și de a revoluționa sectorul digital.

Propunerea UE are în vedere crearea unui spațiu deschis de testare a aplicațiilor AI înainte de a fi puse la dispoziția utilizatorilor, într-un mediu denumit regulatory sandbox , care le-ar putea oferi, în special start-up-urilor și întreprinderilor mici și mijlocii, posibilitatea de a vedea cum interacționează sistemele create de ei, cum se poate asigura calitatea și relevanța datelor folosite de algoritmi și care este impactul asupra drepturilor omului. În condițiile adoptării propunerii, în anii următori, va fi esențială găsirea unui echilibru între stabilirea unui cadru juridic oportun pentru sectorul tehnologic și libertatea de a inova și de a experimenta pentru companii. Având în vedere evoluțiile rapide din acest domeniu, este necesar să existe atât flexibilitate, cât și o adaptabilitate a regulilor, care să nu constrângă avansul tehnologic prin suprareglementare.

prin prezentarea informațiilor într-un fel care să conducă la influențarea comportamentului lor. Felul în care sistemele AI vor putea influența atitudinea online a utilizatorilor necesită o supraveghere mult mai atentă, pentru a minimiza interferențele cu libertatea de gândire și de opinie, accesul la informații sau dreptul la viață privată al utilizatorilor.

Și împărțirea sistemelor AI în categorii de riscuri pentru drepturile și siguranța persoanelor poate fi problematică, întrucât aplicațiile pot avea utilizări diferite, iar evaluarea efectelor acestora în societate nu sunt întotdeauna lesne de identificat. Sistemele AI vor fi clasificate în funcție de scopul asumat, însă vor exista de multe ori și implicații sociale care nu au fost avute în vedere la momentul dezvoltării produsului. Mai mult decât atât, în timp, aplicațiile pot căpăta noi valențe, astfel că o monitorizare constantă a impactului acestora va fi necesară pentru încadrarea în grupele de risc.

Care este statusul global al cursei pentru reglementarea AI?

Suveranitatea digitală și în special leadership-ul privind sistemele AI sunt dorite atât de UE, cât și de SUA sau China, iar abordările celor trei mari puteri din acest domeniu sunt cât se poate de diferite în ceea ce privește cercetarea și dezvoltarea în domeniul AI.

se observă că în clasamentul cercetării în domeniul AI conduc SUA, urmate de China și abia apoi de UE. Un studiu realizat de Center for Data Innovation la începutul acestui an confirmă aceeași ordine, cu mențiunea că în viitor este foarte posibil ca China să înlocuiască SUA ca lider mondial. UE pare să rămână în aceeași poziție ca în prezent. „Who Is Winning the AI Race: China, the EU, or the United States?", datainnovation.org

În aceste condiții, discrepanțele majore în abordările acestor mari puteri în ce privește sistemele AI devin și mai importante. Spre exemplu, în China se folosesc sisteme de acordare a scorului de credit social „China Social Credit System, Punishments and Rewards Explained", businessinsider.com care ar putea să le restricționeze cetățenilor accesul la împrumuturi bancare, transport sau educație pe baza monitorizării comportamentului lor. În totală opoziție, regulamentul UE dorește să interzică în mod expres orice posibilitate de a aplica un astfel de scor de credit social.

dorește să accelereze cercetarea și dezvoltarea de sisteme AI în beneficiul economiei și al societății, punând la dispoziție investiții substanțiale în acest sens. SUA a susținut oficial perspectiva europeană a promovării AI-ului „de încredere" , termen enunțat pentru prima oară enunțat de un grup independent de experți europeni în elaborarea (trustworthy) Orientărilor în materie de etică pentru o inteligență artificială fiabilă. Aceste orientări au evidențiat o serie de principii precum: supraveghere, robustețe tehnică și siguranță, confidențialitate și guvernanța datelor, transparență, diversitate și non-discriminare, bunăstare socială și responsabilitate. Ulterior enunțării lor inițial ca principii etice, aceste reguli au fost introduse drept obligații juridice pentru sistemele AI cu risc înalt, așa cum sunt ele categorisite de propunerea europeană de Regulament pentru AI.

punând accent pe rolul tehnologiilor emergente în politicile de securitate și apărare. Nu în ultimul rând, Organizația pentru Cooperare și Dezvoltare Economică a propus un set de principii de bază pentru AI „OECD Principles on AI", oecd.org care să conducă la îmbunătățirea condițiilor economice și sociale cu ajutorul tehnologiei.

De asemenea, urmează să fie disponibile și fonduri special alocate pentru dezvoltarea tehnologiilor emergente, inclusiv inteligența artificială, prin Planul Național de Redresare și Reziliență (PNRR).

Un moment-cheie pentru reglementarea AI

Inteligența artificială are un rol din ce în ce mai important în activitățile umane, în interacțiunile online și în luarea unor decizii care pot afecta în mod considerabil comportamentul uman. Ne aflăm într-un moment-cheie în care se trasează liniile directoare care vor ghida următoarele decade în ceea ce privește felul în care sistemele care utilizează AI, dezvoltate de companii private sau de state, vor trebui să se conformeze unor standarde legale și etice care să asigure respectul drepturilor și valorilor fundamentale.

Aceste reguli vor trebui să creeze un echilibru între inovație și reglementare și să încurajeze contribuția tehnologiilor emergente la dezvoltarea economică și socială. Cum abordările sunt diferite la nivel global, este important ca această cursă pentru reglementarea AI să seteze cele mai bune măsuri, unele care vor revoluționa impactul acestei tehnologii în societate.

Text de
Cezara Panait

# data[].attributes.attachment_link.title
Cursa globală pentru reglementarea AI - Mindcraft Stories

# data[].attributes.attachment_link.description
Inteligența artificială (AI) e deja prezentă în viața de zi cu zi și poate ajunge chiar să influențeze comportamentul social și pe cel din mediul online. „Technology and Democracy: Understanding the influence of online technologies on political behaviour and decision-making", ec.europa.eu Sistemele AI realizează analize și predicții pe baza unor seturi de date complexe, astfel încât să poată conduce la decizii relevante, într-un timp foarte scurt.

# data[].attributes.attachment_link.description
Artificial intelligence is already a part of our daily lives. When we post pictures to social media, search online or ask questions from chatbots, we're interacting with AI. Authorities, such as cities, rely on AI for providing public services. And governments are seeking solutions to global problems by using algorithmically produced knowledge.

# data[].attributes.attachment_link.readable_text
In spring 2018, MinnaLearn and the University of Helsinki came together with the aim of helping people to be empowered, not threatened, by artificial intelligence. Together, they built the Elements of AI to teach the basics of AI to people from a wide range of backgrounds.

# data[].attributes.attachment_link.title
A free online introduction to artificial intelligence for non-experts

# data[].attributes.attachment_link.description
Learn more about MinnaLearn's and the University of Helsinki's AI course - no programming or complicated math required.

# data[].attributes.attachment_link.readable_text
Skip to Content

Avenue de l'Europe
F-67075 Strasbourg Cedex
Tel. +33 (0)3 88 41 20 00
www.coe.int

Datavisualisation of AI initiatives Datavisualisation of AI initiatives

Non-exhaustive data collection by the Secretariat. Database in open data: Click here to access to the database

Tutorial Tutorial

This datavisualisation brings together more than 450 documents related to artificial intelligence, coming from national authorities, the private sector, international organisations or multi-stakeholder initiatives.

The "bubbles" on the upper right-hand side allow you to filter according to the source of the documents. Clicking on national authorities will only display information relating to that source, such as distribution by publication date or simply the descriptive list.

On the left side, the rectangles, whose size is proportional to the number of documents, are also clickable. Here, for example, Germany can be selected to "filter" information about this state from the entire graphical representation. The list of documents is automatically updated, and can be accessed directly by clicking on the title.

Clicking on the light blue bubble, which represents the international organisations, it is clear that the Council of Europe, together with the European Union, the OECD and UNESCO, are among the most productive organisations in the field. Clicking on the rectangle representing the Council of Europe updates the entire datavisualisation with data relating to the organisation.

# data[].attributes.attachment_link.description
Skip to Content Avenue de l'Europe F-67075 Strasbourg Cedex Tel. +33 (0)3 88 41 20 00 www.coe.int Datavisualisation of AI initiatives Datavisualisation of AI initiatives Non-exhaustive data collection by the Secretariat. Database in open data: Click here to access to the database Tutorial Tutorial This datavisualisation brings together more than 450 documents related to artificial intelligence, coming from national authorities, the private sector, international organisations or multi-stakeholder initiatives.

# data[].attributes.attachment_link.readable_text
Welcome to AI Ethics Guidelines Global Inventory, a project by AlgorithmWatch that maps frameworks that seek to set out principles of how systems for automated decision-making (ADM) can be developed and implemented ethically (last update in April 2020). The database currently includes 173 guidelines. Learn more about the inventory here or jump right in and browse through our inventory using the new filters and search feature!

Are relevant guidelines missing? Please let us know and use the submission form!

# data[].attributes.attachment_link.title
AI Ethics Guidelines Global Inventory by AlgorithmWatch

# data[].attributes.attachment_link.description
AI Ethics Guidelines Global Inventory, a project by AlgorithmWatch that maps frameworks that seek to set out principles of how systems for automated decision-making (ADM) can be developed and implemented ethically. Browse through the inventory using the new filters and search feature! Last update: M

# data[].attributes.attachment_link.description
OECD.AI leverages live data from partners to show trends about where, how and at what rate AI is being developed, used and funded, and in which sectors.

# data[].attributes.attachment_link.readable_text
On a spring afternoon in 2014, Brisha Borden was running late to pick up her god-sister from school when she spotted an unlocked kid's blue Huffy bicycle and a silver Razor scooter. Borden and a friend grabbed the bike and scooter and tried to ride them down the street in the Fort Lauderdale suburb of Coral Springs.

Just as the 18-year-old girls were realizing they were too big for the tiny conveyances - which belonged to a 6-year-old boy - a woman came running after them saying, "That's my kid's stuff." Borden and her friend immediately dropped the bike and scooter and walked away.

But it was too late - a neighbor who witnessed the heist had already called the police. Borden and her friend were arrested and charged with burglary and petty theft for the items, which were valued at a total of $80.

Subscribe to the Series

Machine Bias: Investigating the algorithms that control our lives.

Compare their crime with a similar one: The previous summer, 41-year-old Vernon Prater was picked up for shoplifting $86.35 worth of tools from a nearby Home Depot store.

Prater was the more seasoned criminal. He had already been convicted of armed robbery and attempted armed robbery, for which he served five years in prison, in addition to another armed robbery charge. Borden had a record, too, but it was for misdemeanors committed when she was a juvenile.

Yet something odd happened when Borden and Prater were booked into jail: A computer program spat out a score predicting the likelihood of each committing a future crime. Borden - who is black - was rated a high risk. Prater - who is white - was rated a low risk.

Two years later, we know the computer algorithm got it exactly backward. Borden has not been charged with any new crimes. Prater is serving an eight-year prison term for subsequently breaking into a warehouse and stealing thousands of dollars' worth of electronics.

Scores like this - known as risk assessments - are increasingly common in courtrooms across the nation. They are used to inform decisions about who can be set free at every stage of the criminal justice system, from assigning bond amounts - as is the case in Fort Lauderdale - to even more fundamental decisions about defendants' freedom. In Arizona, Colorado, Delaware, Kentucky, Louisiana, Oklahoma, Virginia, Washington and Wisconsin, the results of such assessments are given to judges during criminal sentencing.

Rating a defendant's risk of future crime is often done in conjunction with an evaluation of a defendant's rehabilitation needs. The Justice Department's National Institute of Corrections now encourages the use of such combined assessments at every stage of the criminal justice process. And a landmark sentencing reform bill currently pending in Congress would mandate the use of such assessments in federal prisons.

In 2014, then U.S. Attorney General Eric Holder warned that the risk scores might be injecting bias into the courts. He called for the U.S. Sentencing Commission to study their use. "Although these measures were crafted with the best of intentions, I am concerned that they inadvertently undermine our efforts to ensure individualized and equal justice," he said, adding, "they may exacerbate unwarranted and unjust disparities that are already far too common in our criminal justice system and in our society."

The sentencing commission did not, however, launch a study of risk scores. So ProPublica did, as part of a larger examination of the powerful, largely hidden effect of algorithms in American life.

We obtained the risk scores assigned to more than 7,000 people arrested in Broward County, Florida, in 2013 and 2014 and checked to see how many were charged with new crimes over the next two years, the same benchmark used by the creators of the algorithm.

The score proved remarkably unreliable in forecasting violent crime: Only 20 percent of the people predicted to commit violent crimes actually went on to do so.

When a full range of crimes were taken into account - including misdemeanors such as driving with an expired license - the algorithm was somewhat more accurate than a coin flip. Of those deemed likely to re-offend, 61 percent were arrested for any subsequent crimes within two years.

We also turned up significant racial disparities, just as Holder feared. In forecasting who would re-offend, the algorithm made mistakes with black and white defendants at roughly the same rate but in very different ways.

The formula was particularly likely to falsely flag black defendants as future criminals, wrongly labeling them this way at almost twice the rate as white defendants.

White defendants were mislabeled as low risk more often than black defendants.

Could this disparity be explained by defendants' prior crimes or the type of crimes they were arrested for? No. We ran a statistical test that isolated the effect of race from criminal history and recidivism, as well as from defendants' age and gender. Black defendants were still 77 percent more likely to be pegged as at higher risk of committing a future violent crime and 45 percent more likely to be predicted to commit a future crime of any kind. ( Read our analysis.)

The algorithm used to create the Florida risk scores is a product of a for-profit company, Northpointe. The company disputes our analysis.

In a letter, it criticized ProPublica's methodology and defended the accuracy of its test: "Northpointe does not agree that the results of your analysis, or the claims being made based upon that analysis, are correct or that they accurately reflect the outcomes from the application of the model."

Northpointe's software is among the most widely used assessment tools in the country. The company does not publicly disclose the calculations used to arrive at defendants' risk scores, so it is not possible for either defendants or the public to see what might be driving the disparity. (On Sunday, Northpointe gave ProPublica the basics of its future-crime formula - which includes factors such as education levels, and whether a defendant has a job. It did not share the specific calculations, which it said are proprietary.)

Northpointe's core product is a set of scores derived from 137 questions that are either answered by defendants or pulled from criminal records. Race is not one of the questions. The survey asks defendants such things as: "Was one of your parents ever sent to jail or prison?" "How many of your friends/acquaintances are taking drugs illegally?" and "How often did you get in fights while at school?" The questionnaire also asks people to agree or disagree with statements such as "A hungry person has a right to steal" and "If people make me angry or lose my temper, I can be dangerous."

The appeal of risk scores is obvious: The United States locks up far more people than any other country, a disproportionate number of them black. For more than two centuries, the key decisions in the legal process, from pretrial release to sentencing to parole, have been in the hands of human beings guided by their instincts and personal biases.

If computers could accurately predict which defendants were likely to commit new crimes, the criminal justice system could be fairer and more selective about who is incarcerated and for how long. The trick, of course, is to make sure the computer gets it right. If it's wrong in one direction, a dangerous criminal could go free. If it's wrong in another direction, it could result in someone unfairly receiving a harsher sentence or waiting longer for parole than is appropriate.

The first time Paul Zilly heard of his score - and realized how much was riding on it - was during his sentencing hearing on Feb. 15, 2013, in court in Barron County, Wisconsin. Zilly had been convicted of stealing a push lawnmower and some tools. The prosecutor recommended a year in county jail and follow-up supervision that could help Zilly with "staying on the right path." His lawyer agreed to a plea deal.

But Judge James Babler had seen Zilly's scores. Northpointe's software had rated Zilly as a high risk for future violent crime and a medium risk for general recidivism. "When I look at the risk assessment," Babler said in court, "it is about as bad as it could be."

Then Babler overturned the plea deal that had been agreed on by the prosecution and defense and imposed two years in state prison and three years of supervision.

Criminologists have long tried to predict which criminals are more dangerous before deciding whether they should be released. Race, nationality and skin color were often used in making such predictions until about the 1970s, when it became politically unacceptable, according to a survey of risk assessment tools by Columbia University law professor Bernard Harcourt.

In the 1980s, as a crime wave engulfed the nation, lawmakers made it much harder for judges and parole boards to exercise discretion in making such decisions. States and the federal government began instituting mandatory sentences and, in some cases, abolished parole, making it less important to evaluate individual offenders.

But as states struggle to pay for swelling prison and jail populations, forecasting criminal risk has made a comeback.

Dozens of risk assessments are being used across the nation - some created by for-profit companies such as Northpointe and others by nonprofit organizations. (One tool being used in states including Kentucky and Arizona, called the Public Safety Assessment, was developed by the Laura and John Arnold Foundation, which also is a funder of ProPublica.)

There have been few independent studies of these criminal risk assessments. In 2013, researchers Sarah Desmarais and Jay Singh examined 19 different risk methodologies used in the United States and found that "in most cases, validity had only been examined in one or two studies" and that "frequently, those investigations were completed by the same people who developed the instrument."

Their analysis of the research through 2012 found that the tools "were moderate at best in terms of predictive validity," Desmarais said in an interview. And she could not find any substantial set of studies conducted in the United States that examined whether risk scores were racially biased. "The data do not exist," she said.

Since then, there have been some attempts to explore racial disparities in risk scores. One 2016 study examined the validity of a risk assessment tool, not Northpointe's, used to make probation decisions for about 35,000 federal convicts. The researchers, Jennifer Skeem at University of California, Berkeley, and Christopher T. Lowenkamp from the Administrative Office of the U.S. Courts, found that blacks did get a higher average score but concluded the differences were not attributable to bias.

The increasing use of risk scores is controversial and has garnered media coverage, including articles by the Associated Press, and the Marshall Project and FiveThirtyEight last year.

Most modern risk tools were originally designed to provide judges with insight into the types of treatment that an individual might need - from drug treatment to mental health counseling.

"What it tells the judge is that if I put you on probation, I'm going to need to give you a lot of services or you're probably going to fail," said Edward Latessa, a University of Cincinnati professor who is the author of a risk assessment tool that is used in Ohio and several other states.

But being judged ineligible for alternative treatment - particularly during a sentencing hearing - can translate into incarceration. Defendants rarely have an opportunity to challenge their assessments. The results are usually shared with the defendant's attorney, but the calculations that transformed the underlying data into a score are rarely revealed.

"Risk assessments should be impermissible unless both parties get to see all the data that go into them," said Christopher Slobogin, director of the criminal justice program at Vanderbilt Law School. "It should be an open, full-court adversarial proceeding."

Black Defendants' Risk Scores

White Defendants' Risk Scores

These charts show that scores for white defendants were skewed toward lower-risk categories. Scores for black defendants were not. (Source: ProPublica analysis of data from Broward County, Fla.)

Proponents of risk scores argue they can be used to reduce the rate of incarceration. In 2002, Virginia became one of the first states to begin using a risk assessment tool in the sentencing of nonviolent felony offenders statewide. In 2014, Virginia judges using the tool sent nearly half of those defendants to alternatives to prison, according to a state sentencing commission report. Since 2005, the state's prison population growth has slowed to 5 percent from a rate of 31 percent the previous decade.

In some jurisdictions, such as Napa County, California, the probation department uses risk assessments to suggest to the judge an appropriate probation or treatment plan for individuals being sentenced. Napa County Superior Court Judge Mark Boessenecker said he finds the recommendations helpful. "We have a dearth of good treatment programs, so filling a slot in a program with someone who doesn't need it is foolish," he said.

However, Boessenecker, who trains other judges around the state in evidence-based sentencing, cautions his colleagues that the score doesn't necessarily reveal whether a person is dangerous or if they should go to prison.

"A guy who has molested a small child every day for a year could still come out as a low risk because he probably has a job," Boessenecker said. "Meanwhile, a drunk guy will look high risk because he's homeless. These risk factors don't tell you whether the guy ought to go to prison or not; the risk factors tell you more about what the probation conditions ought to be."

"I'm surprised [my risk score] is so low. I spent five years in state prison in Massachusetts." (Josh Ritchie for ProPublica)

Sometimes, the scores make little sense even to defendants.

James Rivelli, a 54-year old Hollywood, Florida, man, was arrested two years ago for shoplifting seven boxes of Crest Whitestrips from a CVS drugstore. Despite a criminal record that included aggravated assault, multiple thefts and felony drug trafficking, the Northpointe algorithm classified him as being at a low risk of reoffending.

"I am surprised it is so low," Rivelli said when told by a reporter he had been rated a 3 out of a possible 10. "I spent five years in state prison in Massachusetts. But I guess they don't count that here in Broward County." In fact, criminal records from across the nation are supposed to be included in risk assessments.

Less than a year later, he was charged with two felony counts for shoplifting about $1,000 worth of tools from Home Depot. He said his crimes were fueled by drug addiction and that he is now sober.

Northpointe was founded in 1989 by Tim Brennan, then a professor of statistics at the University of Colorado, and Dave Wells, who was running a corrections program in Traverse City, Michigan.

Wells had built a prisoner classification system for his jail. "It was a beautiful piece of work," Brennan said in an interview conducted before ProPublica had completed its analysis. Brennan and Wells shared a love for what Brennan called "quantitative taxonomy" - the measurement of personality traits such as intelligence, extroversion and introversion. The two decided to build a risk assessment score for the corrections industry.

Brennan wanted to improve on a leading risk assessment score, the LSI, or Level of Service Inventory, which had been developed in Canada. "I found a fair amount of weakness in the LSI," Brennan said. He wanted a tool that addressed the major theories about the causes of crime.

Brennan and Wells named their product the Correctional Offender Management Profiling for Alternative Sanctions, or COMPAS. It assesses not just risk but also nearly two dozen so-called "criminogenic needs" that relate to the major theories of criminality, including "criminal personality," "social isolation," "substance abuse" and "residence/stability." Defendants are ranked low, medium or high risk in each category.

As often happens with risk assessment tools, many jurisdictions have adopted Northpointe's software before rigorously testing whether it works. New York State, for instance, started using the tool to assess people on probation in a pilot project in 2001 and rolled it out to the rest of the state's probation departments - except New York City - by 2010. The state didn't publish a comprehensive statistical evaluation of the tool until 2012. The study of more than 16,000 probationers found the tool was 71 percent accurate, but it did not evaluate racial differences.

A spokeswoman for the New York state division of criminal justice services said the study did not examine race because it only sought to test whether the tool had been properly calibrated to fit New York's probation population. She also said judges in nearly all New York counties are given defendants' Northpointe assessments during sentencing.

In 2009, Brennan and two colleagues published a validation study that found that Northpointe's risk of recidivism score had an accuracy rate of 68 percent in a sample of 2,328 people. Their study also found that the score was slightly less predictive for black men than white men - 67 percent versus 69 percent. It did not examine racial disparities beyond that, including whether some groups were more likely to be wrongly labeled higher risk.

Brennan said it is difficult to construct a score that doesn't include items that can be correlated with race - such as poverty, joblessness and social marginalization. "If those are omitted from your risk assessment, accuracy goes down," he said.

In 2011, Brennan and Wells sold Northpointe to Toronto-based conglomerate Constellation Software for an undisclosed sum.

Wisconsin has been among the most eager and expansive users of Northpointe's risk assessment tool in sentencing decisions. In 2012, the Wisconsin Department of Corrections launched the use of the software throughout the state. It is used at each step in the prison system, from sentencing to parole.

In a 2012 presentation, corrections official Jared Hoy described the system as a " giant correctional pinball machine " in which correctional officers could use the scores at every "decision point."

Wisconsin has not yet completed a statistical validation study of the tool and has not said when one might be released. State corrections officials declined repeated requests to comment for this article.

Some Wisconsin counties use other risk assessment tools at arrest to determine if a defendant is too risky for pretrial release. Once a defendant is convicted of a felony anywhere in the state, the Department of Corrections attaches Northpointe's assessment to the confidential presentence report given to judges, according to Hoy's presentation.

In theory, judges are not supposed to give longer sentences to defendants with higher risk scores. Rather, they are supposed to use the tests primarily to determine which defendants are eligible for probation or treatment programs.

Prediction Fails Differently for Black Defendants

Overall, Northpointe's assessment tool correctly predicts recidivism 61 percent of the time. But blacks are almost twice as likely as whites to be labeled a higher risk but not actually re-offend. It makes the opposite mistake among whites: They are much more likely than blacks to be labeled lower risk but go on to commit other crimes. (Source: ProPublica analysis of data from Broward County, Fla.)

But judges have cited scores in their sentencing decisions. In August 2013, Judge Scott Horne in La Crosse County, Wisconsin, declared that defendant Eric Loomis had been "identified, through the COMPAS assessment, as an individual who is at high risk to the community." The judge then imposed a sentence of eight years and six months in prison.

Loomis, who was charged with driving a stolen vehicle and fleeing from police, is challenging the use of the score at sentencing as a violation of his due process rights. The state has defended Horne's use of the score with the argument that judges can consider the score in addition to other factors. It has also stopped including scores in presentencing reports until the state Supreme Court decides the case.

"The risk score alone should not determine the sentence of an offender," Wisconsin Assistant Attorney General Christine Remington said last month during state Supreme Court arguments in the Loomis case. "We don't want courts to say, this person in front of me is a 10 on COMPAS as far as risk, and therefore I'm going to give him the maximum sentence."

That is almost exactly what happened to Zilly, the 48-year-old construction worker sent to prison for stealing a push lawnmower and some tools he intended to sell for parts. Zilly has long struggled with a meth habit. In 2012, he had been working toward recovery with the help of a Christian pastor when he relapsed and committed the thefts.

After Zilly was scored as a high risk for violent recidivism and sent to prison, a public defender appealed the sentence and called the score's creator, Brennan, as a witness.

Brennan testified that he didn't design his software to be used in sentencing. "I wanted to stay away from the courts," Brennan said, explaining that his focus was on reducing crime rather than punishment. "But as time went on I started realizing that so many decisions are made, you know, in the courts. So I gradually softened on whether this could be used in the courts or not."

"Not that I'm innocent, but I just believe people do change." (Stephen Maturen for ProPublica)

Still, Brennan testified, "I don't like the idea myself of COMPAS being the sole evidence that a decision would be based upon."

After Brennan's testimony, Judge Babler reduced Zilly's sentence, from two years in prison to 18 months. "Had I not had the COMPAS, I believe it would likely be that I would have given one year, six months," the judge said at an appeals hearing on Nov. 14, 2013.

Zilly said the score didn't take into account all the changes he was making in his life - his conversion to Christianity, his struggle to quit using drugs and his efforts to be more available for his son. "Not that I'm innocent, but I just believe people do change."

Florida's Broward County, where Brisha Borden stole the Huffy bike and was scored as high risk, does not use risk assessments in sentencing. "We don't think the [risk assessment] factors have any bearing on a sentence," said David Scharf, executive director of community programs for the Broward County Sheriff's Office in Fort Lauderdale.

Broward County has, however, adopted the score in pretrial hearings, in the hope of addressing jail overcrowding. A court-appointed monitor has overseen Broward County's jails since 1994 as a result of the settlement of a lawsuit brought by inmates in the 1970s. Even now, years later, the Broward County jail system is often more than 85 percent full, Scharf said.

In 2008, the sheriff's office decided that instead of building another jail, it would begin using Northpointe's risk scores to help identify which defendants were low risk enough to be released on bail pending trial. Since then, nearly everyone arrested in Broward has been scored soon after being booked. (People charged with murder and other capital crimes are not scored because they are not eligible for pretrial release.)

The scores are provided to the judges who decide which defendants can be released from jail. "My feeling is that if they don't need them to be in jail, let's get them out of there," Scharf said.

Scharf said the county chose Northpointe's software over other tools because it was easy to use and produced "simple yet effective charts and graphs for judicial review." He said the system costs about $22,000 a year.

In 2010, researchers at Florida State University examined the use of Northpointe's system in Broward County over a 12-month period and concluded that its predictive accuracy was "equivalent" in assessing defendants of different races. Like others, they did not examine whether different races were classified differently as low or high risk.

Scharf said the county would review ProPublica's findings. "We'll really look at them up close," he said.

Broward County Judge John Hurley, who oversees most of the pretrial release hearings, said the scores were helpful when he was a new judge, but now that he has experience he prefers to rely on his own judgment. "I haven't relied on COMPAS in a couple years," he said.

Hurley said he relies on factors including a person's prior criminal record, the type of crime committed, ties to the community, and their history of failing to appear at court proceedings.

ProPublica's analysis reveals that higher Northpointe scores are slightly correlated with longer pretrial incarceration in Broward County. But there are many reasons that could be true other than judges being swayed by the scores - people with higher risk scores may also be poorer and have difficulty paying bond, for example.

Most crimes are presented to the judge with a recommended bond amount, but he or she can adjust the amount. Hurley said he often releases first-time or low-level offenders without any bond at all.

However, in the case of Borden and her friend Sade Jones, the teenage girls who stole a kid's bike and scooter, Hurley raised the bond amount for each girl from the recommended $0 to $1,000 each.

Hurley said he has no recollection of the case and cannot recall if the scores influenced his decision.

Sade Jones, who had never been arrested before, was rated a medium risk. (Josh Ritchie for ProPublica)

The girls spent two nights in jail before being released on bond.

"We literally sat there and cried" the whole time they were in jail, Jones recalled. The girls were kept in the same cell. Otherwise, Jones said, "I would have gone crazy." Borden declined repeated requests to comment for this article.

Jones, who had never been arrested before, was rated a medium risk. She completed probation and got the felony burglary charge reduced to misdemeanor trespassing, but she has still struggled to find work.

"I went to McDonald's and a dollar store, and they all said no because of my background," she said. "It's all kind of difficult and unnecessary."

Julia Angwin is a senior reporter at ProPublica. From 2000 to 2013, she was a reporter at The Wall Street Journal, where she led a privacy investigative team that was a finalist for a Pulitzer Prize in Explanatory Reporting in 2011 and won a Gerald Loeb Award in 2010.

Lauren Kirchner is a senior reporting fellow at ProPublica. Surya Mattu is a contributing researcher. Design and production by Rob Weychert and David Sleight.

# data[].attributes.attachment_link.description
There's software used across the country to predict future criminals. And it's biased against blacks.

# data[].attributes.attachment_link.readable_text
How should we fairly balance this trade-off? There's no universal answer, but in the 1760s, the English judge William Blackstone wrote, "It is better that ten guilty persons escape than that one innocent suffer."

Blackstone's ratio is still highly influential in the US today. So let's use it for inspiration.

Move the threshold to where the "released but rearrested" percentage is roughly 10 times the "needlessly jailed" percentage.

You can already see two problems with using an algorithm like COMPAS. The first is that better prediction can always help reduce error rates across the board, but it can never eliminate them entirely. No matter how much data we collect, two people who look the same to the algorithm can always end up making different choices.

The second problem is that even if you follow COMPAS's recommendations consistently, someone-a human-has to first decide where the "high risk" threshold should lie, whether by using Blackstone's ratio or something else. That depends on all kinds of considerations-political, economic, and social.

Now we'll come to a third problem. This is where our explorations of fairness start to get interesting. How do the error rates compare across different groups? Are there certain types of people who are more likely to get needlessly detained?

Let's see what our data looks like when we consider the defendants' race.

Now move each threshold to see how it affects black and white defendants differently.

Race is an example of a protected class in the US, which means discrimination on that basis is illegal. Other protected classes include gender, age, and disability.

Now that we've separated black and white defendants, we've discovered that even though race isn't used to calculate the COMPAS risk scores, the scores have different error rates for the two groups. At the default COMPAS threshold between 7 and 8, 16% of black defendants who don't get rearrested have been needlessly jailed, while the same is true for only 7% of white defendants. That doesn't seem fair at all! This is exactly what ProPublica highlighted in its investigation.

Okay, so let's fix this.

Move each threshold so white and black defendants are needlessly jailed at roughly the same rate.
(There are a number of solutions. We've picked one, but you can try to find others.)
We tried to reach Blackstone's ratio again, so we arrived at the following solution: white defendants have a threshold between 6 and 7, while black defendants have a threshold between 8 and 9. Now roughly 9% of both black and white defendants who don't get rearrested are needlessly jailed, while 75% of those who do are rearrested after spending no time in jail. Good work! Your algorithm seems much fairer than COMPAS now.

But wait-is it? In the process of matching the error rates between races, we lost something important: our thresholds for each group are in different places, so our risk scores mean different things for white and black defendants.

White defendants get jailed for a risk score of 7, but black defendants get released for the same score. This, once again, doesn't seem fair. Two people with the same risk score have the same probability of being rearrested, so shouldn't they receive the same treatment? In the US, using different thresholds for different races may also raise complicated legal issues with the 14th Amendment, the equal protection clause of the Constitution.

So let's try this one more time with a single threshold shared between both groups.

Move the threshold again so white and black defendants are needlessly jailed at the same rate.

If you're getting frustrated, there's good reason. There is no solution.

We gave you two definitions of fairness: keep the error rates comparable between groups, and treat people with the same risk scores in the same way. Both of these definitions are totally defensible! But satisfying both at the same time is impossible.

The reason is that black and white defendants are rearrested at different rates. Whereas 52% of black defendants were rearrested in our Broward County data, only 39% of white defendants were. There's a similar difference in many jurisdictions across the US, in part because of the country's history of police disproportionately targeting minorities (as we previously mentioned).

Predictions reflect the data used to make them-whether by algorithm or not. If black defendants are arrested at a higher rate than white defendants in the real world, they will have a higher rate of predicted arrest as well. This means they will also have higher risk scores on average, and a larger percentage of them will be labeled high-risk- both correctly and incorrectly. This is true no matter what algorithm is used, as long as it's designed so that each risk score means the same thing regardless of race.

This strange conflict of fairness definitions isn't just limited to risk assessment algorithms in the criminal legal system. The same sorts of paradoxes hold true for credit scoring, insurance, and hiring algorithms. In any context where an automated decision-making system must allocate resources or punishments among multiple groups that have different outcomes, different definitions of fairness will inevitably turn out to be mutually exclusive.

There is no algorithm that can fix this; this isn't even an algorithmic problem, really. Human judges are currently making the same sorts of forced trade-offs-and have done so throughout history.

But here's what an algorithm has changed. Though judges may not always be transparent about how they choose between different notions of fairness, people can contest their decisions. In contrast, COMPAS, which is made by the private company Northpointe, is a trade secret that cannot be publicly reviewed or interrogated. Defendants can no longer question its outcomes, and government agencies lose the ability to scrutinize the decision-making process. There is no more public accountability.

So what should regulators do? The proposed Algorithmic Accountability Act of 2019 is an example of a good start, says Andrew Selbst, a law professor at the University of California who specializes in AI and the law. The bill, which seeks to regulate bias in automated decision-making systems, has two notable features that serve as a template for future legislation. First, it would require companies to audit their machine-learning systems for bias and discrimination in an "impact assessment." Second, it doesn't specify a definition of fairness.

"With an impact assessment, you're being very transparent about how you as a company are approaching the fairness question," Selbst says. That brings public accountability back into the debate. Because "fairness means different things in different contexts," he adds, avoiding a specific definition allows for that flexibility.

But whether algorithms should be used to arbitrate fairness in the first place is a complicated question. Machine-learning algorithms are trained on "data produced through histories of exclusion and discrimination," writes Ruha Benjamin, an associate professor at Princeton University, in her book Race After Technology. Risk assessment tools are no different. The greater question about using them-or any algorithms used to rank people-is whether they reduce existing inequities or make them worse.

Selbst recommends proceeding with caution: "Whenever you turn philosophical notions of fairness into mathematical expressions, they lose their nuance, their flexibility, their malleability," he says. "That's not to say that some of the efficiencies of doing so won't eventually be worthwhile. I just have my doubts."
Words and code by Karen Hao and Jonathan Stray. Design advising from Emily Luong and Emily Caulfield. Editing by Niall Firth and Gideon Lichfield. Special thanks to Rashida Richardson from AI Now, Mutale Nkonde from Berkman Klein Center, and William Isaac from DeepMind for their review and consultation.
Correction: A previous version of the article linked to information about a risk assessment tool different from COMPAS. It has been removed to avoid confusion.

# data[].attributes.attachment_link.title
Can you make AI fairer than a judge? Play our courtroom algorithm game

# data[].attributes.attachment_link.description
As a child, you develop a sense of what "fairness" means. It's a concept that you learn early on as you come to terms with the world around you. Something either feels fair or it doesn't. But increasingly, algorithms have begun to arbitrate fairness for us.

# data[].attributes.attachment_link.readable_text
The way we approach Artificial Intelligence (AI) will define the world we live in the future. To help building a resilient Europe for the Digital Decade, people and businesses should be able to enjoy the benefits of AI while feeling safe and protected.

The European AI Strategy aims at making the EU a world-class hub for AI and ensuring that AI is human-centric and trustworthy. Such an objective translates into the European approach to excellence and trust through concrete rules and actions.

In April 2021, the Commission presented its AI package, including:

A European approach to excellence in AI

Fostering excellence in AI will strengthen Europe's potential to compete globally.

The EU will achieve this by:

1. enabling the development and uptake of AI in the EU;

2. making the EU the place where AI thrives from the lab to the market;

3. ensuring that AI works for people and is a force for good in society;

4. building strategic leadership in high-impact sectors.

The Commission and Member States agreed to boost excellence in AI by joining forces on policy and investments. The 2021 review of the Coordinated Plan on AI outlines a vision to accelerate, act, and align priorities with the current European and global AI landscape and bring AI strategy into action.

Maximising resources and coordinating investments is a critical component of AI excellence. Through the Horizon Europe and Digital Europe programmes, the Commission plans to invest €1 billion per year in AI. It will mobilise additional investments from the private sector and the Member States in order to reach an annual investment volume of €20 billion over the course of the digital decade.

The Recovery and Resilience Facility makes €134 billion available for digital. This will be a game-changer, allowing Europe to amplify its ambitions and become a global leader in developing cutting-edge, trustworthy AI.

Access to high quality data is an essential factor in building high performance, robust AI systems. Initiatives such as the EU Cybersecurity Strategy, the Digital Services Act and the Digital Markets Act, and the Data Governance Act provide the right infrastructure for building such systems.

A European approach to trust in AI

Building trustworthy AI will create a safe and innovation-friendly environment for users, developers and deployers.

The Commission has proposed 3 inter-related legal initiatives that will contribute to building trustworthy AI:

1. a European legal framework for AI to address fundamental rights and safety risks specific to the AI systems;

2. a civil liability framework - adapting liability rules to the digital age and AI;

3. a revision of sectoral safety legislation (e.g. Machinery Regulation, General Product Safety Directive).

European proposal for a legal framework on AI

The Commission aims to address the risks generated by specific uses of AI through a set of complementary, proportionate and flexible rules. These rules will also provide Europe with a leading role in setting the global gold standard.

This framework gives AI developers, deployers and users the clarity they need by intervening only in those cases that existing national and EU legislations do not cover. The legal framework for AI proposes a clear, easy to understand approach, based on four different levels of risk: unacceptable risk, high risk, limited risk, and minimal risk.

Important milestones

# data[].attributes.attachment_link.title
A European approach to artificial intelligence

# data[].attributes.attachment_link.description
The way we approach Artificial Intelligence (AI) will define the world we live in the future. To help building a resilient Europe for the Digital Decade, people and businesses should be able to enjoy the benefits of AI while feeling safe and protected.