WASHINGTON A pair of artificial intelligence projects from U.S. Army researchers are easing communication barriers that limit the relationship between AI systems and soldiers.

The artificial intelligence projects are designed to support ongoing efforts for the Armys next-generation combat vehicle modernization priority, which includes a focus on autonomous vehicles and AI-enabled platforms.

The first project, named the Joint Understanding and Dialogue Interface, or JUDI, is an AI system that can understand the intent of a soldier when that individual gives a robot verbal instructions. The second project, Transparent Multi-Modal Crew Interface Designs, is meant to give soldiers a better understanding of why AI systems make decisions.

Were attacking a similar problem from opposite ends, said Brandon Perelman, a research psychologist at the Army Research Laboratory who worked on Transparent Multi-Modal Crew Interface Designs.

The JUDI system will improve soldiers situational awareness when working with robots because it will transform that relationship from a heads down, hands full to a heads up, hands free interaction, according to Matthew Marge, a computer scientist at the lab. Simply put, this means that soldiers will be more aware of their surroundings, he said.

Natural language AI systems are available on the commercial market, but the JUDI system requires a level of awareness in a physical environment that isnt matched by commercial products. Commercial systems can understand what a person is saying and take instructions, but they dont know what is going on in the surrounding area. For the Army, the autonomous system needs to know that.

You want a robot to be able to process what youre saying and ground it to the immediate physical context, Marge said. So that means the robot has to not only interpret the speech, but also have a good idea of where it is [in] the world, the mapping of its surroundings, and how it represents those surroundings in a way that can relate to what the soldier is saying.

Researchers looked into how soldiers speak to robots, and how robots talk back. In prior research, Marge found that humans speak to technology in much simpler, direct language; but when talking to other people, they usually talk about a course of action and the steps involved. However, those studies were done in a safe environment, and not a stressful one similar to combat, during which a soldiers language could be different. Thats an area where Marge knows the Army must perform more research.

Sign up for the C4ISRNET newsletter about future battlefield technologies.

(please select a country) United States United Kingdom Afghanistan Albania Algeria American Samoa Andorra Angola Anguilla Antarctica Antigua and Barbuda Argentina Armenia Aruba Australia Austria Azerbaijan Bahamas Bahrain Bangladesh Barbados Belarus Belgium Belize Benin Bermuda Bhutan Bolivia Bosnia and Herzegovina Botswana Bouvet Island Brazil British Indian Ocean Territory Brunei Darussalam Bulgaria Burkina Faso Burundi Cambodia Cameroon Canada Cape Verde Cayman Islands Central African Republic Chad Chile China Christmas Island Cocos (Keeling) Islands Colombia Comoros Congo Congo, The Democratic Republic of The Cook Islands Costa Rica Cote D'ivoire Croatia Cuba Cyprus Czech Republic Denmark Djibouti Dominica Dominican Republic Ecuador Egypt El Salvador Equatorial Guinea Eritrea Estonia Ethiopia Falkland Islands (Malvinas) Faroe Islands Fiji Finland France French Guiana French Polynesia French Southern Territories Gabon Gambia Georgia Germany Ghana Gibraltar Greece Greenland Grenada Guadeloupe Guam Guatemala Guinea Guinea-bissau Guyana Haiti Heard Island and Mcdonald Islands Holy See (Vatican City State) Honduras Hong Kong Hungary Iceland India Indonesia Iran, Islamic Republic of Iraq Ireland Israel Italy Jamaica Japan Jordan Kazakhstan Kenya Kiribati Korea, Democratic People's Republic of Korea, Republic of Kuwait Kyrgyzstan Lao People's Democratic Republic Latvia Lebanon Lesotho Liberia Libyan Arab Jamahiriya Liechtenstein Lithuania Luxembourg Macao Macedonia, The Former Yugoslav Republic of Madagascar Malawi Malaysia Maldives Mali Malta Marshall Islands Martinique Mauritania Mauritius Mayotte Mexico Micronesia, Federated States of Moldova, Republic of Monaco Mongolia Montserrat Morocco Mozambique Myanmar Namibia Nauru Nepal Netherlands Netherlands Antilles New Caledonia New Zealand Nicaragua Niger Nigeria Niue Norfolk Island Northern Mariana Islands Norway Oman Pakistan Palau Palestinian Territory, Occupied Panama Papua New Guinea Paraguay Peru Philippines Pitcairn Poland Portugal Puerto Rico Qatar Reunion Romania Russian Federation Rwanda Saint Helena Saint Kitts and Nevis Saint Lucia Saint Pierre and Miquelon Saint Vincent and The Grenadines Samoa San Marino Sao Tome and Principe Saudi Arabia Senegal Serbia and Montenegro Seychelles Sierra Leone Singapore Slovakia Slovenia Solomon Islands Somalia South Africa South Georgia and The South Sandwich Islands Spain Sri Lanka Sudan Suriname Svalbard and Jan Mayen Swaziland Sweden Switzerland Syrian Arab Republic Taiwan, Province of China Tajikistan Tanzania, United Republic of Thailand Timor-leste Togo Tokelau Tonga Trinidad and Tobago Tunisia Turkey Turkmenistan Turks and Caicos Islands Tuvalu Uganda Ukraine United Arab Emirates United Kingdom United States United States Minor Outlying Islands Uruguay Uzbekistan Vanuatu Venezuela Viet Nam Virgin Islands, British Virgin Islands, U.S. Wallis and Futuna Western Sahara Yemen Zambia Zimbabwe

Subscribe

By giving us your email, you are opting in to the C4ISRNET Daily Brief.

When a soldier is under pressure, we dont want to have any limit on the range of words or phrases they have to memorize to speak to the robot, Marge said. So from the beginning, we are taking an approach of so-called natural language. We dont want to impose any restrictions on what a soldier might say to a robot.

JUDIs ability to determine a soldiers intent or what Army researchers define as whatever the soldier wants JUDI to do is based on an algorithm that tries to match the verbal instructions with existing data. The algorithm finds an instruction from its training data with the highest overlap and sends it to the robot as a command.

The JUDI system, Marge said, is scheduled for field testing in September. JUDI was developed with help from researchers at the University of Southern Californias Institute for Creative Technologies.

The Transparent Multi-Modal Crew Interface Designs is tackling the AI-human interaction from the other side.

Were looking at ways of improving the ability of AI to communicate information to the soldier to show the soldier what its thinking and what its doing so its more predictable and trustworthy, Perelman said. Because we know that if soldiers dont understand why the AI is doing something and it fails, theyre not going to trust it. And if they dont trust it, theyre not going to use it.

Mission planning is the one area where the Transparent Multi-Modal Crew Interface Designs may prove useful. Perelman compared the program to driving down the highway while a navigation app responds to changes along a route. A driver may want to stay on the highway for the sake of convenience not having to steer through extra turns even if it takes a few minutes longer.

You can imagine a situation during mission planning, for example, where an AI proposes a number of courses of action that you could take, and if its not able to accurately communicate how its coming up with those decisions, then the soldier is really not going to be able to understand and accurately calculate the trade-offs that its taking into account, Perelman said.

He added that through lab testing, the team improved soldiers ability to predict the AIs future mobility actions by 60 percent and allowed the soldiers to decide between multiple courses of actions 40 percent quicker.

The program has transitioned over to the Army Combat Capabilities Development Commands Ground Vehicle System Centers Crew Optimization and Augmentation Technologies program. Thats where it will take part in Mission Enabler Technologies-Demonstrators phase 2 field testing.

Excerpt from:

Two US Army projects seek to improve comms between soldiers and AI - C4ISRNet