Mathematics in art

One really exciting component of the application‑side work that I do is the collaboration with art historians and conservators on art restoration and also analysis and interpretation of art, where we are looking at digitized versions of whatever art piece we’re examining. When I look at a piece of art, it’s definitely not like in the movie A Beautiful Mind where I only see numbers. What I see actually entices the curiosity of what I can’t see or what has been lost.

In mathematical terms, you can phrase this as an inverse problem. You take a measurement – a digital photograph, in our case – of an image. In scientific imaging, we don’t just take a photograph; we might take a more sophisticated image, like multispectral imaging that captures wavelengths beyond the visible light spectrum, such as infrared imaging, or we can also perform X‑ray imaging on a painting. I look at this data and want to understand, based on it, what the original image, or the image at various moments in time, looked like before. This is an inverse problem. It’s about having measurements but being interested in something that is hidden in them, and it’s about inverting – hence “inverse” – the forward process, which is the history that has happened to this art piece.

In the illuminations that accompany the text, you see Adam and Eve in Paradise, how they live there, how Eve eats the apple, and how they are expelled. When the manuscript was created, Adam and Eve were painted as naked, as you know from the biblical story – in Paradise they are without sin. Later on, the book came into the possession of a different owner who was not happy with a children’s book featuring naked figures, so another artist was commissioned to overpaint Adam and Eve with skirts.

We then combined the color photograph – with its colors, textures and so on – with the structural outline of Eve and Adam’s bodies from the infrared image, and inpainted the colors into the area of the skirts following the structures provided by the infrared data.

Thinking about this, the question arises: is mathematics applied to art also an art in itself, an art of finding the right methodology for the restoration? I would say yes. Mathematics is a formalized language with grammar that tells you what you can and cannot do, but within those constraints you have the freedom to put ideas together in whatever way you want. To some extent, mathematics applied anywhere – not just in art – is an art in itself, in finding the right models.

One of the first partial differential equations developed by Marcelo Bertalmio and colleagues was modelled after talking to museum artists: they look at the hole – the damaged part in the picture – observe structures that arrive there, such as color edges, and propagate them inward. This is almost a recipe for writing down a transport‑type equation, which Marcelo and colleagues did.

The beautiful thing about art and mathematics is that they are mutually inspiring. The art historians particularly appreciate using mathematical equations like partial differential equations, because they provide an objective way to handle inpainting, which is very ambiguous. You can never go back in time and prove that what you have done is correct; you can only make an educated guess. These equations help you do that, and you can explain how they work in a formalized way that the art historians can understand and debate.

The inpainting result depends on the modeling assumptions that you put into the differential equation or whatever mathematical model you use to solve the inpainting. A funny example I like to show is that these modeling assumptions, or any prior conceptions you might have, influence how you restore images.

A Spanish woman near Zaragoza, a frequent visitor to a church, saw a heavily damaged portrait of Jesus Christ. The portrait had been in the church for more than 100 years and was damaged, though you could still make out the main characteristics. She felt so awful about this that she took up the task of restoring the portrait herself – and she did.

Her restored version went through the news a lot a couple of years ago. The original portrait is titled Ecce Homo; the new version she created was jokingly called Ecce Mono. It looks like a monkey rather than Jesus Christ. Of course, what you think is good or not is somewhat subjective, but most of us will agree that it is no longer close to the portrait we started with.

This illustrates that in image restoration and inpainting – or inverse problems in imaging in general – everything depends on the prior information you put in and how closely you stick to the data, to the evidence – in this case, the damaged Jesus Christ painting or fresco.

Editor’s note: This article has been faithfully transcribed from the original interview filmed with the author, and carefully edited and proofread. Edit date: 2025