Ask Knut — a system for finding and identifying ice

The app is developed using artificial intelligence and machine learning — as well as a wide range of photos of ice formations, including sea ice and icebergs. Did you think ice was ice? Think again. There are actually several hundred different types of ice formations. 

Associate Professor Ekaterina Kim at the Department of Marine Technology plays a central role in this project, called “Ask Knut”. 

“A photo of an ice sheet actually reveals quite a lot,” she says. Among other things, it can tell you what time of year the photo was taken, and the condition and thickness of the ice, which tells you something about what type of ice it is. The colour, for example, indicates how old or new the ice is.

She says there are hundreds of different names for types of ice, each unique and indicative of the conditions under which the ice was formed: pancake ice, pinnacled iceberg, brash ice, sloping iceberg, dirty ice, frost flowers, etc.

Arctic research benefits from AI
In recent years, Ekaterina Kim has worked with artificial intelligence (AI) and deep learning.  This project was born from two strong research communities at NTNU: SAMCoT and AMOS (Roger Skjetne and others), who are collaborating with Theoharis Theoharis with the Department of Computer Science on computer vision and deep learning.

Plenty of scientists and researchers are already addressing challenges associated with the Arctic, but this field has not yet caught the attention of artificial intelligence experts. She teamed up with PhD candidate Ole-Magnus Pedersen and summer intern Nabil Panchi, and the team started looking into whether it would be possible to classify ice structures observed in the Arctic. The goal is to develop a system people on board vessels can use to discover and identify different types of floating ice, to prevent accidents and injuries.

To create recognition, Ekaterina Kim and Ole-Magnus Pedersen have fed the algorithm they have developed with hundreds of images of ice combined with descriptions of this particular ice formation extremely many times. This is how the machine can learn the differences.

Major changes
Sea ice is changing at a more rapid pace today than it did even just five years ago. And the rate at which things are changing means that our current warning systems and technology can’t quite keep up  The team behind “Ask Knut” hope to develop a simple, user-friendly system that can be installed on a ship’s bridge, continually scanning and filming the surface, classifying any and all ice it identifies.  The people on the bridge can then determine whether this particular piece of sea ice is dangerous or harmless.  The goal is for the system to be able to identify different types of ice even in fog and inclement weather. The app will also be able to recognize different types of ice photographed by ordinary users.

In order to build recognition, Ekaterina, Ole-Magnus and Nabil fed the algorithm they developed with hundreds of photos of ice, along with descriptions of this particular type of ice formation again and again and again. This teaches the machine to recognize the differences.

“In a research perspective, it’s important to monitor glaciers and icebergs closely. It’s important to monitor ice melting — and fresh water quality. With this app, it will be possible to observe, record and classify ice from day to day over shorter or longer periods of time. In time, we hope to licence the technology to interested parties.”

Prosjekter og nyheter