Explanation methods that help users understand and trust machine-learning models often describe how much certain features used in the model contribute to its prediction. For example, if a model predicts a patients risk of developing cardiac disease, a physician might want to know how strongly the patients heart rate data influences that prediction.
But if those features are so complex or convoluted that the user cant understand them, does the explanation method do any good?
MIT researchers are striving to improve the interpretability of features so decision makers will be more comfortable using the outputs of machine-learning models. Drawing on years of field work, they developed a taxonomy to help developers craft features that will be easier for their target audience to understand.
We found that out in the real world, even though we were using state-of-the-art ways of explaining machine-learning models, there is still a lot of confusion stemming from the features, not from the model itself, says Alexandra Zytek, an electrical engineering and computer science PhD student and lead author of a paper introducing the taxonomy.
To build the taxonomy, the researchers defined properties that make features interpretable for five types of users, from artificial intelligence experts to the people affected by a machine-learning models prediction. They also offer instructions for how model creators can transform features into formats that will be easier for a layperson to comprehend.
They hope their work will inspire model builders to consider using interpretable features from the beginning of the development process, rather than trying to work backward and focus on explainability after the fact.
MIT co-authors include Dongyu Liu, a postdoc; visiting professor Laure Berti-quille, research director at IRD; and senior author Kalyan Veeramachaneni, principal research scientist in the Laboratory for Information and Decision Systems (LIDS) and leader of the Data to AI group. They are joined by Ignacio Arnaldo, a principal data scientist at Corelight. The research is published in the June edition of the Association for Computing Machinery Special Interest Group on Knowledge Discovery and Data Minings peer-reviewed Explorations Newsletter.
Real-world lessons
Features are input variables that are fed to machine-learning models; they are usually drawn from the columns in a dataset. Data scientists typically select and handcraft features for the model, and they mainly focus on ensuring features are developed to improve model accuracy, not on whether a decision-maker can understand them, Veeramachaneni explains.
For several years, he and his team have worked with decision makers to identify machine-learning usability challenges. These domain experts, most of whom lack machine-learning knowledge, often dont trust models because they dont understand the features that influence predictions.
For one project, they partnered with clinicians in a hospital ICU who used machine learning to predict the risk a patient will face complications after cardiac surgery. Some features were presented as aggregated values, like the trend of a patients heart rate over time. While features coded this way were model ready (the model could process the data), clinicians didnt understand how they were computed. They would rather see how these aggregated features relate to original values, so they could identify anomalies in a patients heart rate, Liu says.
By contrast, a group of learning scientists preferred features that were aggregated. Instead of having a feature like number of posts a student made on discussion forums they would rather have related features grouped together and labeled with terms they understood, like participation.
With interpretability, one size doesnt fit all. When you go from area to area, there are different needs. And interpretability itself has many levels, Veeramachaneni says.
The idea that one size doesnt fit all is key to the researchers taxonomy. They define properties that can make features more or less interpretable for different decision makers and outline which properties are likely most important to specific users.
For instance, machine-learning developers might focus on having features that are compatible with the model and predictive, meaning they are expected to improve the models performance.
On the other hand, decision makers with no machine-learning experience might be better served by features that are human-worded, meaning they are described in a way that is natural for users, and understandable, meaning they refer to real-world metrics users can reason about.
The taxonomy says, if you are making interpretable features, to what level are they interpretable? You may not need all levels, depending on the type of domain experts you are working with, Zytek says.
Putting interpretability first
The researchers also outline feature engineering techniques a developer can employ to make features more interpretable for a specific audience.
Feature engineering is a process in which data scientists transform data into a format machine-learning models can process, using techniques like aggregating data or normalizing values. Most models also cant process categorical data unless they are converted to a numerical code. These transformations are often nearly impossible for laypeople to unpack.
Creating interpretable features might involve undoing some of that encoding, Zytek says. For instance, a common feature engineering technique organizes spans of data so they all contain the same number of years. To make these features more interpretable, one could group age ranges using human terms, like infant, toddler, child, and teen. Or rather than using a transformed feature like average pulse rate, an interpretable feature might simply be the actual pulse rate data, Liu adds.
In a lot of domains, the tradeoff between interpretable features and model accuracy is actually very small. When we were working with child welfare screeners, for example, we retrained the model using only features that met our definitions for interpretability, and the performance decrease was almost negligible, Zytek says.
Building off this work, the researchers are developing a system that enables a model developer to handle complicated feature transformations in a more efficient manner, to create human-centered explanations for machine-learning models. This new system will also convert algorithms designed to explain model-ready datasets into formats that can be understood by decision makers.
View original post here:
Building explainability into the components of machine-learning models - MIT News
- What is Artificial Intelligence (AI)? - Definition from ... [Last Updated On: June 12th, 2016] [Originally Added On: June 12th, 2016]
- Artificial Intelligence | Neuro AI [Last Updated On: June 12th, 2016] [Originally Added On: June 12th, 2016]
- Association for the Advancement of Artificial Intelligence [Last Updated On: June 13th, 2016] [Originally Added On: June 13th, 2016]
- A.I. Artificial Intelligence - Wikipedia, the free ... [Last Updated On: June 17th, 2016] [Originally Added On: June 17th, 2016]
- Artificial Intelligence - The New York Times [Last Updated On: June 17th, 2016] [Originally Added On: June 17th, 2016]
- Intro to Artificial Intelligence Course and Training ... [Last Updated On: June 28th, 2016] [Originally Added On: June 28th, 2016]
- Artificial Intelligence | Neuro AI [Last Updated On: July 1st, 2016] [Originally Added On: July 1st, 2016]
- What is Artificial Intelligence (AI)? Webopedia Definition [Last Updated On: July 1st, 2016] [Originally Added On: July 1st, 2016]
- Intro to Artificial Intelligence Course and Training Online ... [Last Updated On: July 5th, 2016] [Originally Added On: July 5th, 2016]
- Artificial Intelligence News -- ScienceDaily [Last Updated On: September 16th, 2016] [Originally Added On: September 16th, 2016]
- Artificial intelligence positioned to be a game-changer - CBS ... [Last Updated On: October 13th, 2016] [Originally Added On: October 13th, 2016]
- Artificial Intelligence: A Modern Approach - amazon.com [Last Updated On: October 31st, 2016] [Originally Added On: October 31st, 2016]
- Artificial Intelligence - IndiaBIX [Last Updated On: November 23rd, 2016] [Originally Added On: November 23rd, 2016]
- The Non-Technical Guide to Machine Learning & Artificial ... [Last Updated On: November 23rd, 2016] [Originally Added On: November 23rd, 2016]
- Artificial Intelligence - Graduate Schools of Science ... [Last Updated On: November 23rd, 2016] [Originally Added On: November 23rd, 2016]
- Artificial Intelligence in Medicine: An Introduction [Last Updated On: November 23rd, 2016] [Originally Added On: November 23rd, 2016]
- What does artificial intelligence mean? - Definitions.net [Last Updated On: November 23rd, 2016] [Originally Added On: November 23rd, 2016]
- Artificial Intelligence Lockheed Martin [Last Updated On: November 23rd, 2016] [Originally Added On: November 23rd, 2016]
- Artificial Intelligence Course - Computer Science at CCSU [Last Updated On: November 23rd, 2016] [Originally Added On: November 23rd, 2016]
- FREE Artificial Intelligence Essay - Example Essays [Last Updated On: November 23rd, 2016] [Originally Added On: November 23rd, 2016]
- Elon Musk's artificial intelligence group signs Microsoft ... [Last Updated On: November 23rd, 2016] [Originally Added On: November 23rd, 2016]
- Real FX - Slotless Racing with Artificial Intelligence [Last Updated On: November 23rd, 2016] [Originally Added On: November 23rd, 2016]
- Artificial Intelligence: What It Is and How It Really Works [Last Updated On: January 4th, 2017] [Originally Added On: January 4th, 2017]
- Artificial Intelligence Market Size and Forecast by 2024 [Last Updated On: January 4th, 2017] [Originally Added On: January 4th, 2017]
- Algorithm-Driven Design: How Artificial Intelligence Is ... [Last Updated On: January 4th, 2017] [Originally Added On: January 4th, 2017]
- 9 Development in Artificial Intelligence | Funding a ... [Last Updated On: January 4th, 2017] [Originally Added On: January 4th, 2017]
- Artificial Intelligence Tops Humans in Poker Battle What's the Big Deal? - PokerNews.com [Last Updated On: February 6th, 2017] [Originally Added On: February 6th, 2017]
- Is AI a Threat to Christianity? - The Atlantic [Last Updated On: February 6th, 2017] [Originally Added On: February 6th, 2017]
- Allow mathematicians to pierce artificial intelligence frontiers - Livemint [Last Updated On: February 6th, 2017] [Originally Added On: February 6th, 2017]
- Montreal sees its future in smart sensors, artificial intelligence (with video) - Computerworld [Last Updated On: February 6th, 2017] [Originally Added On: February 6th, 2017]
- Silicon Valley Hedge Fund Takes On Wall Street With AI Trader - Bloomberg [Last Updated On: February 6th, 2017] [Originally Added On: February 6th, 2017]
- The Observer view on artificial intelligence - The Guardian [Last Updated On: February 6th, 2017] [Originally Added On: February 6th, 2017]
- Artificial Intelligence Is Coming Whether You Like It Or Not - Mother Jones [Last Updated On: February 6th, 2017] [Originally Added On: February 6th, 2017]
- RealDoll Creating Artificial Intelligence System, Robotic Sex Dolls ... - Breitbart News [Last Updated On: February 7th, 2017] [Originally Added On: February 7th, 2017]
- Forget lessons, these smart skis are loaded with artificial intelligence - Mashable [Last Updated On: February 7th, 2017] [Originally Added On: February 7th, 2017]
- Artificial Intelligence Correctly Predicted the Patriots' 34-28 Super ... - Digital Trends [Last Updated On: February 7th, 2017] [Originally Added On: February 7th, 2017]
- Why C-Levels Need To Think About eLearning And Artificial Intelligence - Forbes [Last Updated On: February 7th, 2017] [Originally Added On: February 7th, 2017]
- Artificial Intelligence-Driven Robots: More Brains Than Brawn - Forbes [Last Updated On: February 7th, 2017] [Originally Added On: February 7th, 2017]
- Artificial intelligence: How to build the business case - ZDNet [Last Updated On: February 7th, 2017] [Originally Added On: February 7th, 2017]
- What 'social artificial intelligence' means for marketers - VentureBeat [Last Updated On: February 7th, 2017] [Originally Added On: February 7th, 2017]
- Actress Kristen Stewart's Research Paper On Artificial Intelligence: A Critical Evaluation - Forbes [Last Updated On: February 7th, 2017] [Originally Added On: February 7th, 2017]
- Baidu cut its healthcare business to concentrate on artificial intelligence - Asia Times [Last Updated On: February 9th, 2017] [Originally Added On: February 9th, 2017]
- Google Android Wear 2.0 update puts artificial intelligence inside your wristwatch - The Sun [Last Updated On: February 9th, 2017] [Originally Added On: February 9th, 2017]
- How criminals use Artificial Intelligence and Machine Learning - BetaNews [Last Updated On: February 9th, 2017] [Originally Added On: February 9th, 2017]
- In the Labs: Connected vehicles in Ohio, artificial intelligence in Illinois and Massachusetts - Network World [Last Updated On: February 9th, 2017] [Originally Added On: February 9th, 2017]
- Keeping an eye on artificial intelligence - The National Business Review [Last Updated On: February 10th, 2017] [Originally Added On: February 10th, 2017]
- Actors, teachers, therapists think your job is safe from artificial intelligence? Think again - The Guardian [Last Updated On: February 10th, 2017] [Originally Added On: February 10th, 2017]
- Wells Fargo Innovation Group to Focus on Artificial Intelligence, Payments and APIs - Wall Street Journal (blog) [Last Updated On: February 10th, 2017] [Originally Added On: February 10th, 2017]
- SAP aims to step up its artificial intelligence, machine learning game as S/4HANA hits public cloud - ZDNet [Last Updated On: February 10th, 2017] [Originally Added On: February 10th, 2017]
- Artificial Intelligence Is Coming To Police Bodycams, Raising Privacy Concerns - Forbes [Last Updated On: February 10th, 2017] [Originally Added On: February 10th, 2017]
- Nvidia Beats Earnings Estimates As Its Artificial Intelligence Business Keeps On Booming - Forbes [Last Updated On: February 10th, 2017] [Originally Added On: February 10th, 2017]
- Could Artificial Intelligence Ever Become A Threat To Humanity? - Forbes [Last Updated On: February 10th, 2017] [Originally Added On: February 10th, 2017]
- Artificial intuition will supersede artificial intelligence, experts say - Network World [Last Updated On: February 11th, 2017] [Originally Added On: February 11th, 2017]
- The Peril of Inaction with Artificial Intelligence - Gigaom [Last Updated On: February 11th, 2017] [Originally Added On: February 11th, 2017]
- TASER International Bringing Artificial Intelligence to Law Enforcement - Motley Fool [Last Updated On: February 11th, 2017] [Originally Added On: February 11th, 2017]
- LG G6 teasers emphasize battery life, artificial intelligence - CNET [Last Updated On: February 11th, 2017] [Originally Added On: February 11th, 2017]
- Wells Fargo sets up artificial intelligence team in tech push - Reuters [Last Updated On: February 11th, 2017] [Originally Added On: February 11th, 2017]
- Ford spending $1 billion on self-driving artificial intelligence - CNET [Last Updated On: February 11th, 2017] [Originally Added On: February 11th, 2017]
- Artificial Intelligence in Business Process Automation - Nanalyze [Last Updated On: February 12th, 2017] [Originally Added On: February 12th, 2017]
- An artificial intelligence gamble that paid off - Minneapolis Star Tribune [Last Updated On: February 12th, 2017] [Originally Added On: February 12th, 2017]
- Ford to Invest $1 Billion in Artificial Intelligence Start-Up - New York Times [Last Updated On: February 12th, 2017] [Originally Added On: February 12th, 2017]
- Wells Fargo Pushes Into Artificial Intelligence - Fortune [Last Updated On: February 13th, 2017] [Originally Added On: February 13th, 2017]
- Artificial intelligence predictions surpass reality - UT The Daily Texan [Last Updated On: February 13th, 2017] [Originally Added On: February 13th, 2017]
- Creating artificial intelligence-driven technology products is almost like unleashing the Frankenstein's monster - Economic Times (blog) [Last Updated On: February 13th, 2017] [Originally Added On: February 13th, 2017]
- Inside Intel Corporation's Artificial Intelligence Strategy - Motley Fool [Last Updated On: February 13th, 2017] [Originally Added On: February 13th, 2017]
- The artificial intelligence revolutionising healthcare - Irish Times [Last Updated On: February 13th, 2017] [Originally Added On: February 13th, 2017]
- Ford Announces Investment in Artificial Intelligence Company Argo AI - Motor Trend [Last Updated On: February 13th, 2017] [Originally Added On: February 13th, 2017]
- Ford Invests $1-Billion in Artificial Intelligence - AutoGuide.com [Last Updated On: February 13th, 2017] [Originally Added On: February 13th, 2017]
- Salesforce adds some artificial intelligence to customer service products - TechCrunch [Last Updated On: February 13th, 2017] [Originally Added On: February 13th, 2017]
- No hype, just fact: Artificial intelligence in simple business terms - ZDNet [Last Updated On: February 15th, 2017] [Originally Added On: February 15th, 2017]
- Artificial Intelligence and The Confusion of Our Age - Patheos (blog) [Last Updated On: February 15th, 2017] [Originally Added On: February 15th, 2017]
- How Artificial Intelligence Startups Struck Gold - Entrepreneur [Last Updated On: February 15th, 2017] [Originally Added On: February 15th, 2017]
- Terrifyingly, Google's Artificial Intelligence acts aggressive when cornered - Chron.com [Last Updated On: February 15th, 2017] [Originally Added On: February 15th, 2017]
- This Startup Has Developed A New Artificial Intelligence That Can (Sometimes) Beat Google - Forbes [Last Updated On: February 15th, 2017] [Originally Added On: February 15th, 2017]
- RPI artificial intelligence expert looks at Westworld - Albany Times Union [Last Updated On: February 15th, 2017] [Originally Added On: February 15th, 2017]
- Google's DeepMind artificial intelligence becomes 'highly aggressive' when stressed. Skynet, anyone? - Mirror.co.uk [Last Updated On: February 15th, 2017] [Originally Added On: February 15th, 2017]
- Artificial Intelligence Enters The Classroom - News One [Last Updated On: February 15th, 2017] [Originally Added On: February 15th, 2017]
- John Pisarek Talks Artificial Intelligence - Customer Think [Last Updated On: February 15th, 2017] [Originally Added On: February 15th, 2017]
- Can Artificial Intelligence Predict Earthquakes? - Scientific American [Last Updated On: February 15th, 2017] [Originally Added On: February 15th, 2017]
- Artificial Intelligence Is Becoming A Major Disruptive Force In Banks' Finance Departments - Forbes [Last Updated On: February 15th, 2017] [Originally Added On: February 15th, 2017]