Aerial image of endangered Southern Resident killer whales in K pod. The image was obtained using a remotely piloted octocopter drone that was flown during health research by Dr. John Durban and Dr. Holly Fearnbach. (Vulcan Image)

Being an orca isnt easy. Despite a lack of natural predators, these amazing mammals face many serious threats most of them brought about by their human neighbors. Understanding the pressures we put on killer whale populations is critical to the environmental policy decisions that will hopefully contribute to their ongoing survival.

Fortunately, marine mammal researchers like Holly Fearnbach of Sealife Response + Rehab + Research (SR3) and John Durban of Oregon State University are working hard to regularly monitor the condition of the Salish Seas southern resident killer whale population (SKRW). Identified as J pod, K pod and L pod, these orca communities have migrated through the Salish Sea for millennia. Unfortunately, in recent years their numbers have dwindled to only 75 whales, with one new calf born in 2021. This is the lowest population figure for the SRKW in 30 years.

For more than a decade, Fearnbach and Durban have flown photographic surveys to capture aerial images of the orcas. Starting in 2008, image surveys were performed using manned helicopter flights. Then beginning in 2014, the team transitioned to unmanned drones.

As the remote-controlled drone flies 100 feet or more above the whales, images are captured of each of the pod members, either individually or in groups. Since the drone is also equipped with a laser altimeter, the exact distance is known making calculations of the whales dimensions very accurate. The images are then analyzed in whats called a photogrammetric health assessment. This assessment helps determine each whales physical condition, including any evidence of pregnancy or significant weight loss due to malnourishment.

As a research tool, the drone is very cost effective and it allows us to do our research very noninvasively, Fearnbach said. When we do detect health declines in individuals, were able to provide management agencies with these quantitative health metrics.

But while the image collection stage is relatively inexpensive, processing the data has been costly and time-consuming. Each flight can capture 2,000 images with tens of thousands of images captured for each survey. Following the drone work, it typically takes about six months to manually complete the analysis on each seasons batch of images.

Obviously, half a year is a very long time if youre starving or pregnant, which is one reason why SR3s new partnership with Vulcan is so important. Working together, the organizations developed a new approach to process the data more rapidly. The Aquatic Mammal Photogrammetry Tool (AMPT) uses machine learning and an end-user tool to accelerate the laborious process, dramatically shortening the time needed to analyze, identify and categorize all of the images.

Applying machine learning techniques to the problem has already yielded huge results, reducing a six-month process to just six weeks with room for further improvements. Machine learning is a branch of computing that can improve its performance through experience and use of data. The faster turnaround time will make it possible to more quickly identify whales of concern and provide health metrics to management groups to allow for adaptive decision making, according to Vulcan.

Were trying to make and leave the world a better place, primarily through ocean health and conservation, said Sam McKennoch, machine learning team manager at Vulcan. We got connected with SR3 and realized this was a great use case, where they have a large amount of existing data and needed help automating their workflows.

AMPT is based on four different machine learning models. First, the orca detector identifies those images that have orcas in them and places a box around each whale. The next ML model fully outlines the orcas body, a process known in the machine learning field as semantic segmentation. After that comes the landmark detector which detects the rostrum (or snout) of the whale, the dorsal fins, blowhole, shape of the eye patches, fluke notch and so forth. This allows the software to measure and calculate the shape and proportions of various parts of the body.

Of particular interest is whether the whales facial fat deposits are so low they result in indentations of the head that marine biologists refer to as peanut head. This only appears when the orca has lost a significant amount of body fat and is in danger of starvation.

Finally, the fourth machine learning model is the identifier. The shape of the gray saddle patch behind the whales dorsal fin is as unique as a fingerprint, allowing each of the individuals in the pod to be identified.

There are a lot of different kinds of information needed for this kind of automation. Fortunately, Vulcan has been able to leverage some of SR3s prior manual work to bootstrap their machine learning models.

We really wanted to understand their pain points and how we could provide them the tools they needed, rather than the tools we might want to give them, McKennoch said.

As successful as AMPT has been, theres a lot of knowledge and information that has yet to be incorporated into its machine learning models. As a result, theres still the need to have users in-the-loop in a semi-supervised way for some of the ML processing. The interface speeds up user input and standardizes measurements made by different users.

McKennoch believes there will be gains with each batch they process for several cycles to come. Because of this, they hope to continue to improve performance in terms of accuracy, workflow and compute time to the point that the entire process eventually takes days, instead of weeks or months.

This is very important because AMPT will provide information that guides policy decisions at many levels. Human impact on the orcas environment is not diminishing and if anything, is increasing. Overfishing is reducing food sources, particularly chinook salmon, the orcas preferred meal. Commercial shipping and recreational boats continue to cause injury and their excessive noise interferes with the orcas ability to hunt salmon. Toxic chemicals from stormwater runoff and other pollution damage the marine mammals health. Ongoing monitoring of each individual whale will be critical to maintaining their wellbeing and the health of the local marine ecosystem.

Vulcan plans to open-source AMPT, giving it a life of its own in the marine mammal research community. McKennoch said they hope to extend the tool so it can be used for other killer whale populations, different large whales, and in time, possibly smaller dolphins and harbor seals.

Read more:
Can machine learning help save the whales? How PNW researchers use tech tools to monitor orcas - GeekWire