Artificial Intelligence Restores Mutilated Rembrandt Painting The Night Watch – ARTnews

One of Rembrandts finest works, Militia Company of District II under the Command of Captain Frans Banninck Cocq (better known as The Night Watch) from 1642, is a prime representation of Dutch Golden Age painting. But the painting was greatly disfigured after the artists death, when it was moved from its original location at the Arquebusiers Guild Hall to Amsterdams City Hall in 1715. City officials wanted to place it in a gallery between two doors, but the painting was too big to fit. Instead of finding another location, they cut large panels from the sides as well as some sections from the top and bottom. The fragments were lost after removal.

Now, centuries later, the painting has been made complete through the use of artificial intelligence. The Rijksmuseum in the Netherlands has owned The Night Watch since it opened in 1885 and considers it one of the best-known paintings in its collection. In 2019, the museum embarked on a multi-year, multi-million-dollar restoration project, referred to as Operation Night Watch, to recover the painting. The effort marks the 26th restoration of the work over the span of its history.

In the beginning, restoring The Night Watch to its original size hadnt been considered until the eminent Rembrandt scholar Erst van der Wetering suggested it in a letter to the museum, noting that the composition would change dramatically. The museum tapped its senior scientist, Rob Erdmann, to head the effort using three primary tools: the remaining preserved section of the original painting, a 17th-century copy of the original painting attributed to Gerrit Lundens that had been made before the cuts, and AI technology.

About the decision to use AI to reconstruct the missing pieces instead of commissioning an artist to repaint the work, Erdmann told ARTnews, Theres nothing wrong with having an artist recreate [the missing pieces] by looking at the small copy, but then wed see the hand of the artist there. Instead, we wanted to see if we could do this without the hand of an artist. That meant turning to artificial intelligence.

AI was used to solve a set of specific problems, the first of which was that the copy made by Lundens is one-fifth the size of the original, which measures almost 12 feet in length. The other issue was that Lundens painted in a different style than Rembrandt, which raised the question of how the missing pieces could be restored to an approximation of how Rembrandt would have painted them. Erdmann created three separate neural networks, a type of machine learning technology that trains computers to learn how to do specific tasks to address the problems.

The first [neural network] was responsible for identifying shared details. It found more than 10,000 details in common between The Night Watch and Lundenss copy. For the second, Erdmann said, Once you have all of these details, everything had to be warped into place, essentially by tinkering with the pieces by scoot[ing one part] a little bit to the left and making another section of the painting 2 percent bigger, and rotat[ing another] by four degrees. This way all the details would be perfectly aligned to serve as inputs to the third and final stage. Thats when we sent the third neural network to art school.

Erdmann made a test for the neural network, similar to flashcards, by splitting up the painting into thousands of tiles and placing matching tiles from both the original and the copy side-by-side. The AI then had to create an approximation of those tiles in the style of Rembrandt. Erdmann graded the approximationsand if it painted in the style of Lundens, it failed. After the program ran millions of times, the AI was ready to reproduce tiles from the Lundens copy in the style of Rembrandt.

The AIs reproduction was printed onto canvas and lightly varnished, and then the reproduced panels were attached to the frame of The Night Watch over top the fragmented original. The reconstructed panels do not touch Rembrandts original painting and will be taken down in three months out of respect for the Old Master. It already felt to me like it was quite bold to put these computer reconstructions next to Rembrandt, Erdmann said.

As for the original painting by Rembrandt, it may receive conservation treatment depending on the conclusions of the research being conducted as part of Operation Night Watch. The painting has sustained damaged that may warrant additional interventions. In 1975, the painting was slashed several times, and, in 1990, it was splashed with acid.

The reconstructed painting went on view at the Rijksmuseum on Wednesday and will remain into September.

Continued here:

Artificial Intelligence Restores Mutilated Rembrandt Painting The Night Watch - ARTnews