The Problem With COVID-19 Artificial Intelligence Solutions and How to Fix Them – Stanford Social Innovation Review

(Photo by Engin Akyurt/Unsplash)

Private and public entities around the world, particularly in the health care and governance sectors, are developing and deploying a range of artificial intelligence (AI) systems in emergency response to COVID-19. Some of these systems work to track and predict its spread; others support medical response or help maintain social control. Indeed, AI systems can reduce strain on overwhelmed health care systems; help save lives by quickly diagnosing patients, and assessing health declines or progress; and limit the viruss spread.

But theres a problem: The algorithms driving these systems are human creations, and as such, they are subject to biases that can deepen societal inequities and pose risks to businesses and society more broadly. In this article, we look at data on the pandemic, share two recent applications of AI, and suggest a number of ways nonprofit and business leaders can help ensure that they develop, manage, and use transformative AI equitably and responsibly.

Using techinical frameworks, such as machine learning, AI systems use algorithms to make inferences from data about people. This includes demographic attributes, preferences, and likely future behaviors. To effectively serve a range of populations, AI systems must learn to make associations based on massive amounts of data that accurately reflect information across identities. However, the data they rely on is often rife with social and cultural biases. Data might not exist for certain populations, may exist but be poor quality for certain groups, and/or reflect inequities in society. As a result, algorithms can make inaccurate predictions and perpetuate social stereotypes and biases.

Unfortunately, much of the data about COVID-19 that the US Center for Disease Control and Prevention (CDC) and others are collecting and tracking is incomplete and biased. COVID-19 infection rates, for example, have been subject to a vast undercount, by a factor of 50 or more. Medical data is reflecting only a subset of the populationin many cases, the affluent, white communities who have ready access to limited tests and expensive medical procedures. But there are other important data gaps too:

Some of the AI systems created to support COVID-19 medical response help diagnose and detect COVID-19 through basic online screening or analyzing chest images. Others, such as the forthcoming version of eCART, can help predict COVID-specific outcomes and inform clinical decisions. This is particularly useful for medical volunteers without pulmonary training, who must assess patients conditions and decide who needs help first. AI tech may also prove helpful in the search for a COVID-19 vaccine and other therapies.

However, the data gaps we mentioned earlier have major implications for medical AI systems and AI-enhanced vaccine trials. People react differently to viruses, vaccines, and treatments, as previous outbreaks like SARS and Ebola have illustrated. Data available on COVID-19 outside the United States, for example, shows that men and women face different fatality rates, and a recent research paper found that women patients admitted to the Wuhan Union Hospital had higher levels of COVID-19 antibodies than men. Given systemic inequities that worsen health outcomes for certain racial and ethnic groups, its equally important to understand COVID-19 health outcomes for different identities, as well as the intersectional implications.

Algorithms that dont account for existing inequities risk making inaccurate predictionsor worse. In 2019, a study found that the widely used Optum algorithm, which used health-care spending as a proxy to measure need, was biased against black Americans. It didnt account for discrimination or lack of access, both of which lead to lower spending on health care by black Americans. Amid the COVID-19 crisis, AI systems that inform limited-resource allocations (such as who to put on a ventilator) must be careful not to inadvertently prioritize certain identities over others. While developers aim to make algorithms race-blind by excluding race as a metric, this can ignore or hide rather than preventdiscrimination. For example, algorithms that inform clinical decisions may use proxies such as preexisting conditions. Diabetes is a preexisting condition linked to higher rates of COVID-19, and it has a higher incidence for black Americans. If an algorithm uses preexisting conditions but is blind to race, it can result in disproportionately prioritizing white Americans over black Americans.

While some firms adhere to rigorous testingconducting large validation studies prior to releasing products, for examplenot all firms are thorough. Further, the decision-making processes of most AI algorithms are not transparent. This opens the door to inaccurate or discriminatory predictions for certain demographics, and thus poses immense risks to the individuals and practitioners using them.

Another recent application of AI is contact tracing, or tracking people who have come into contact with the virus to help contain it. By tracking user information such as health and location, and using AI-powered facial recognition, these tools can enforce social distancing and inform citizens of contact with positive cases. In China, users are assigned a coronavirus score, which impacts their access to public transportation, work, and school. And US government officials have begun raising the possibility of mass surveillance, collecting anonymized, aggregate user location data from tech giants like Facebook and Google to map the spread of COVID-19.

But surveillance tools have ethical implicationsagain, particularly for marginalized populations. Using AI to decide who leaves their home could lead to a form of COVID-19 redlining, subjecting certain communities to greater enforcements. This calls to mind another AI model that results in higher surveillance of poor communities of color: predictive policing. In the United States, risk-assessment algorithms use criminal history information, but dont take into account deep-rooted racial bias in the policing system, that black Americans are arrested more often for smaller crimes and that neighborhoods with high concentration of black Americans are more heavily patrolled. Black Americans end up overrepresented in the data, which then links to racially biased policing outcomes. Similarly, communities impacted by proposed surveillance systems would likely be poorer communities of color harder hit by COVID-19 for a variety of reasons linked to historical inequities and discrimination.

It is not clear how or how long government agencies or other entities will use these types of AI tools. In China, tracking could stick around after the crisis, allowing Beijing authorities to monitor religious minorities, political dissidents, and other marginalized communities with a history of being over-surveilled. And although data collection in the United States will initially be anonymized and aggregated, theres potential for misuse and de-anonymization in the future.

Various AI systems are proving incredibly valuable to tackling the pandemic, and others hold immense promise. But leaders must take care to develop, manage, and use this technology responsibly and equitably; the risks of discrimination and deepening inequality are simply unacceptable. Here are five actions to take now:

1. Demand transparency and explanation of the AI system. First and foremost, leaders need to hold themselves accountable. Particularly with AI systems targeting medical response, its important that decision makers understand which groups are represented in the datasets and what the quality of that data is across different groups. Tools such as Datasheets for Datasets are useful for tracking information on dataset creators; the composition, sampling, and labeling process; and intended uses. Leaders whose organizations develop AI systems should also ask questions like: Whose opinions, priorities, and expertise are included in development, and whose are left out?

2. Join and promote multidisciplinary ethics working groups or councils to inform response to COVID-19. This is already happening in Germany and can provide useful insights into how to respond to COVID-19, including using AI. Working groups are a way to bring together social scientists, philosophers, community leaders, and technical teams to discuss potential bias concerns and fairness tradeoffs, as well as solutions.

3. Build partnerships to fill health-data gaps in ways that protect and empower local communities. Nonprofits and universities are especially well-positioned to work with disenfranchised communities and form community research partnerships. In San Francisco, for example, a coalition of citywide Latinx organizations partnered with UCSF to form a COVID-19 task force. The coalition launched a project that tested nearly 3,000 residents in predominantly Latinx neighborhoods to better understand how the virus spreads. The task force and its local volunteers integrated concerns of community members and provided extensive support services to people who tested positive.

4. Advance research and innovation while emphasizing diversity and inclusion. Only a handful of tech companies and elite university labs develop most large-scale AI systems, and developers tend to be white, affluent, technically oriented, and male. Given that AI isnt neutral and that technologies are a product of the context in which they are created, these systems often fail to meet the needs of different communities. Research initiatives like the recently launched Digital Transformation Institute, a collaborative effort to bring together tech companies and US research universities to fight COVID-19, must emphasize inclusion and justice (alongside innovation and efficiency), and prioritize multi-disciplinary and diverse teams. They can and should take advantage of tools like an AI Fairness Checklist in designing solutions.

5. Resist the urge to prioritize efficiency at the cost of justice and equity. Leaders should rise to the challenge of not compromising justice and equity. In some cases, the question is not how best to develop or deploy an AI system, but whether the AI system should be built or used at all.

As the pandemic continues to severely impact individuals, communities, and economies, nonprofit and business leaders must respond quicklybut not at the cost of heightening discrimination and inequality in the communities hardest hit by the pandemic. AI can help us improve medical response and minimize the spread of COVID-19, but using it wisely requires equity-fluent leadership and a long-term view. As Prashant Warier, CEO and co-founder of the AI company Qure.ai, put it, Once people start using our algorithms, they never stop.

Read more from the original source:
The Problem With COVID-19 Artificial Intelligence Solutions and How to Fix Them - Stanford Social Innovation Review