A critical review of the EU’s ‘Ethics Guidelines for Trustworthy AI’ – TNW

Europe has some of the most progressive, human-centric artificial intelligence governance policies in the world. Compared to the heavy-handed government oversight in China or the Wild West-style anything goes approach in the US, the EUs strategy is designed to stoke academic and corporate innovation while also protecting private citizens from harm and overreach. But that doesnt mean its perfect.

In 2018, the European Commission began its European AI Alliance initiative. The alliance exists so that various stakeholders can weigh-in and be heard as the EU considers its ongoing policies governing the development and deployment of AI technologies.

Since 2018, more than 6,000 stakeholders have participated in the dialogue through various venues, including online forums and in-person events.

Subscribe to our newsletter now for a weekly recap of our favorite AI stories in your inbox.

The commentary, concerns, and advice provided by those stakeholders has been considered by the EUs High-level expert group on artificial intelligence, who ultimately created four key documents that work as the basis for the EUs policy discussions on AI:

1. Ethics Guidelines for Trustworthy AI

2. Policy and Investment Recommendations for Trustworthy AI

3. Assessment List for Trustworthy AI

4. Sectoral Considerations on the Policy and Investment Recommendations

This article focuses on item number one: the EUs Ethics Guidelines for Trustworthy AI.

Published in 2019, this document lays out the barebones ethical concerns and best practices for the EU. While I wouldnt exactly call it a living document, it is supported by a continuously updated reporting system via the European AI Alliance initiative.

The Ethics Guidelines for Trustworthy AI provides a set of 7 key requirements that AI systems should meet in order to be deemed trustworthy.

Per the document:

AI systems should empower human beings, allowing them to make informed decisions and fostering their fundamental rights. At the same time, proper oversight mechanisms need to be ensured, which can be achieved through human-in-the-loop, human-on-the-loop, and human-in-command approaches.

Neurals rating: poor

Human-in-the-loop, human-on-the-loop, and human-in-command are all wildly subjective approaches to AI governance that almost always rely on marketing strategies, corporate jargon, and disingenuous approaches to discussing how AI models work in order to appear efficacious.

Essentially, the human in the loop myth involves the idea that an AI system is safe as long as a human is ultimately responsible for pushing the button or authorizing the execution of a machine learning function that could potentially have an adverse effect on humans.

The problem: Human-in-the-loop relies on competent humans at every level of the decision-making process to ensure fairness. Unfortunately, studies show that humans are easily manipulated by machines.

Were also prone to ignore warnings whenever they become routine.

Think about it, whens the last time you read all the fine print on a website before agreeing to the terms presented? How often do you ignore the check engine light on your car or the time for an update alert on software when its still functioning properly?

Automating programs or services that affect human outcomes under the pretense that having a human in the loop is enough to prevent misalignment or misuse is, in this authors opinion, a feckless approach to regulation that gives businesses carte blanche to development harmful models as long as they tack on a human-in-the-loop requirement for usage.

As an example of what could go wrong, ProPublicas award-winning Machine Bias article laid bare the propensity for the human-in-the-loop paradigm to cause additional bias by demonstrating how AI used to recommend criminal sentences can perpetuate and amplify racism.

A solution: the EU should do away with the idea of creating proper oversight mechanisms and instead focus on creating policies that regulate the use and deployment of black box AI systems to prevent them from deployment in situations where human outcomes might be affected unless theres a human authority who can be held ultimately responsible.

Per the document:

AI systems need to be resilient and secure. They need to be safe, ensuring a fall back plan in case something goes wrong, as well as being accurate, reliable and reproducible. That is the only way to ensure that also unintentional harm can be minimized and prevented.

Neurals rating : needs work.

Without a definition of safe, the whole statement is fluff. Furthermore, accuracy is a malleable term in the AI world that almost always refers to arbitrary benchmarks that do not translate beyond laboratories.

A solution: the EU should set a bare minimum requirement that AI models deployed in Europe with the potential to affect human outcomes must demonstrate equality. An AI model that achieves lower reliability or accuracy on tasks involving minorities should be considered neither safe nor reliable.

Per the document:

Besides ensuring full respect for privacy and data protection, adequate data governance mechanisms must also be ensured, taking into account the quality and integrity of the data, and ensuring legitimised access to data.

Neurals rating: good, but could be better.

Luckily, the General Data Protection Regulation (GDPR) does most of the heavy lifting here. However, the terms quality and integrity are highly subjective as is the term legitimised access.

A solution: the EU should define a standard where data must be obtained with consent and verified by humans to ensure the databases used to train models contain only data that is properly-labeled and used with the permission of the person or group who generated it.

Per the document:

The data, system and AI business models should be transparent. Traceability mechanisms can help achieving this. Moreover, AI systems and their decisions should be explained in a manner adapted to the stakeholder concerned. Humans need to be aware that they are interacting with an AI system, and must be informed of the systems capabilities and limitations.

Neurals rating: this is hot garbage.

Only a small percentage of AI models lend themselves to transparency. The majority of AI models in production today are black box systems that, by the very nature of their architecture, produce outputs using far too many steps of abstraction, deduction, or conflation for a human to parse.

In other words, a given AI system might use billions of different parameters to produce an output. In order to understand why it produced that particular outcome instead of a different one, wed have to review each of those parameters step-by-step so that we could come to the exact same conclusion as the machine.

A solution: the EU should adopt a strict policy preventing the deployment of opaque or black box artificial intelligence systems that produce outputs that could affect human outcomes unless a designated human authority can be held fully accountable for unintended negative outcomes.

Per the document:

Unfair bias must be avoided, as it could could have multiple negative implications, from the marginalization of vulnerable groups, to the exacerbation of prejudice and discrimination. Fostering diversity, AI systems should be accessible to all, regardless of any disability, and involve relevant stakeholders throughout their entire life circle.

Neurals rating: poor.

In order for AI models to involve relevant stakeholders throughout their entire life circle theyd need to be trained on data sourced from diverse sources and developed by teams of diverse people. The reality is that STEM is dominated by white, straight, cis-males and there are myriad peer-reviewed studies demonstrating how that simple, demonstrable fact makes it almost impossible to produce many types of AI models without bias.

A solution: unless the EU has a method by which to solve the lack of minorities in STEM, it should instead focus on creating policies that prevent businesses and individuals from deploying AI models that produce different outcomes for minorities.

Per the document:

AI systems should benefit all human beings, including future generations. It must hence be ensured that they are sustainable and environmentally friendly. Moreover, they should take into account the environment, including other living beings, and their social and societal impact should be carefully considered.

Neurals rating: great. No notes!

Per the document:

Mechanisms should be put in place to ensure responsibility and accountability for AI systems and their outcomes. Auditability, which enables the assessment of algorithms, data and design processes plays a key role therein, especially in critical applications. Moreover, adequate an accessible redress should be ensured.

Neurals rating: good, but could be better.

Theres currently no political consensus as to whos responsible when AI goes wrong. If the EUs airport facial recognition systems, for example, mistakenly identify a passenger and the resulting inquiry causes them financial harm (they miss their flight and any opportunities stemming from their travel) or unnecessary mental anguish, theres nobody who can be held responsible for the mistake.

The employees following procedure based on the AIs flagging of a potential threat are just doing their jobs. And the developers who trained the systems are typically beyond reproach once their models go into production.

A solution: the EU should create a policy that specifically dictates that humans must always be held accountable when an AI system causes an unintended or erroneous outcome for another human. The EUs current policy and strategy encourages a blame the algorithm approach that benefits corporate interests more than citizen rights.

While the above commentary may be harsh, I believe the EUs AI strategy is a light leading the way. However, its obvious that the EUs desire to compete with the Silicon Valley innovation market in the AI sector has pushed the bar for human-centric technology a little further towards corporate interests than the unions other technology policy initiatives have.

The EU wouldnt sign off on an aircraft that was mathematically proven to crash more often if Black persons, women, or queer persons were passengers than it did when white men were onboard. It shouldnt allow AI developers to get away with deploying models that function that way either.

Read the original:

A critical review of the EU's 'Ethics Guidelines for Trustworthy AI' - TNW