AI turns infrared images taken in total darkness into full colour

The black-and-white images provided by night-vision cameras can be colourised using AI, but it must always be trained on similar images and is unlikely to ever work on unfamiliar general scenes

By Matthew Sparkes

Night-vision cameras convert infrared light – outside the spectrum visible to humans – into visible light so we can “see in the dark”. But this infrared information only allows a black-and-white image to be constructed. Now, AI can colourise these images for a more natural feel.

Andrew Browne at the University of California, Irvine, and his colleagues used a camera that can detect both visible light and part of the infrared spectrum to take 140 images of different faces. The team then trained a neural network to spot correlations between the way objects appeared in infrared and their colour in the visible spectrum. Once trained, this AI could predict the visible colouring from pure infrared images, even those originally taken in total darkness.

Browne believes the approach could become extremely accurate over time, although the results are already difficult to distinguish from genuine colour images. “I think this technology could be used for precise colour evaluation if the amount and variety of data used to train the neural network is sufficiently large to increase accuracy,” he says.

But he concedes that the scope of this project is limited to images of faces, and the AI is unlikely to ever be able to colourise any image without having been trained on similar types of images.

Adrian Hilton at the University of Surrey, UK, says that AI is the ideal solution to spotting any correlations between what is observed in the visible spectrum and what can be picked up in infrared. However, he adds that the AI’s choice of colours will always be a best guess rather than an accurate deduction based on evidence.

“Human faces are, of course, a very constrained group of objects, if you like. It doesn’t immediately translate to colouring a general scene,” he says. “As it stands at the moment, if you apply the method trained on faces to another scene, it probably wouldn’t work, it probably wouldn’t do anything sensible.”

Hilton also says that the same AI trained to colourise images of fruit from infrared images alone would always be fooled by a random blue banana, for instance, as it would have learned context from training data that included multiple images of yellow bananas.

Journal reference: PLoS One, DOI: 10.1371/journal.pone.0265185