A recent study has shown that in coming years global labour markets will undergo major transformations. This will be due to the so called “Fourth Industrial Revolution”, which is caused by companies looking into the possibilities of machines and algorithms. The study prognosticates that in 2022, 42 percent of all jobs will be performed by machines, while at this moment this percentage is 29. Jobs that are most likely to disappear are those consisting mainly of paperwork and those of the less educated. The study states that jobs based on human skills such as creativity will remain indispensable, and therefore will continue to be executed by real people. However, algorithms are constantly being updated and the introduction of machine learning gives machines more handles to understand and penetrate the concept of art. This raises the question whether it is true that jobs in the arts sector will survive this fourth industrial revolution. This question is addressed by exploring the capabilities of computers.

Analyzing

When you look at a painting there is a great chance that you are capable of making several inferences about it. For example, you can deduce the period, style or artist. A requirement for this is the ability to recognize color, composition, texture, perspective, subject matter and other visual effects. Could a computer algorithm perform these classification tasks as easily as a human? Ahmed Elgammal from the Rutgers university asked himself this question in 2016 and with his colleagues began to explore this using MATLAB®, Machine Learning ToolboxTM and a database containing thousands of paintings from the past six centuries. Their main interest was testing whether the AI algorithm was capable of identify influential paintings and measure creativity.
While an algorithm that can extract features is fairly common in computer vision, the machine learning techniques are a greater challenge. In this research, distance metric learning techniques were applied to weigh the importance of certain features and thereby improve the algorithm. Using this technique, the algorithms’ accuracy in classifying paintings was 60 percent. This is much lower than an art historian, but it performs extremely better than an ordinary human being.
Next, a working algorithm was developed to answer the question if influence could be tested. Elgammal and his team hypothesized that objects and symbols in a painting are reliable features to test influence. They trained the algorithm using Google images to identify specific objects and then tested it on a database consisting of more than 1700 paintings from 66 different artists over a time span of 550 years. Again, 60% of the influences recognized by art historians were identified. An example are the two paintings below. At first they seem fairly different. However, the algorithm can recognize similarities, such as the perspective, the chair, the group of men, a window and a heater, all in similar locations on the painting.

The issue of creativity was a completely different one. First of all, creativity was defined as being unlike proceeding paintings but similar to succeeding ones. By doing this it was possible to transform the database into a network, where vertices are paintings and edges represent similarities between paintings. Deducing which painting is creative from such a graph is similar to a [expand title=”network centrality problem”] The problem of finding the most important node in a network, for example by counting the number of times a node acts as a bridge along the shortest path between two other nodes  [/expand]
and therefore MATLAB could be used to give a creativity score to each painting. It was found that this algorithm was capable of extracting highly influential painters from the dataset, such as Leonardo da Vinci, Johannes Vermeer, Claude Monet and Piet Mondriaan.
Ahmed Elgammal concluded from his research that their algorithms are capable of classifying art and recognizing influential work, and although they are not yet as precise as a human being, they could be excellent tools to navigate through the great amount of art which has been produced over the years.

Abstract art

It is now clear that computers are able to recognize certain features in a painting or picture and are capable of recognizing similarities even though objects might not look the same. Does this mean that computers, as in Plato’s theory of Forms, have a certain ‘Form’ for each object and can recognize which objects in a painting are an imitation of this ‘Form’? In late 2017 Tom White, lecturer in computational design at the University of Wellington, looked into this, by creating art that shows how algorithms see the world. White developed a series of algorithms which transformed pictures from a database on which machine vision algorithms are trained into abstract shapes. Then the abstract shapes were put through the algorithm again to see if it could still recognize the object depicted. If this was not the case the image was adjusted slightly until the algorithm did recognize the object. In this way White says he is capable of ‘reverse-engineering’ how a computer sees objects. For example, below one can see the depicting of an electrical fan and a cello.

White admits that the art is not a product of some highly intelligent AI machine. He says that his role in setting the starting parameters such as colours and thickness of lines and the rejection of ‘paintings’ that he thinks are not pleasing for the eye is very important. He also states that his art is made to give insight in how algorithms work. If one looks at the painting of the cello above, you can see that there is another figure behind the cello. With a humans’ understanding of the world one can deduce that this figure is the cellist holding his instrument. As computers have no prior knowledge of instruments or the concept of music, the algorithm learned itself that a cello consists of these two shapes. This shows that although these machines can be very clever, intelligence is a completely different attribute. If the algorithm is trained on ‘wrong’ data its output will never be correct.
From this kind of research one could conclude that computers are more and more becoming creative actors. Although this algorithm is only looking for the core of an object and how this should be depicted, it is essentially looking for patterns and replicating them. This is a process which can be compared to the way humans learn how to create art. To quote Karthik Kalyanaraman, curator responsible for the exhibition of Tom White’s art: “I think it is foolish to say ‘well it’s not really creative because it doesn’t have consciousness’ ”. This raises the question whether we are approaching a future where machines and algorithms can be seen as artists.

Future

The next step in the development of artificial intelligent art occured in late 2018. During a Christie’s auction the painting Portrait of Edmond Belamy was sold. A portrait of a man that seems to be unfinished. The facial features are a little bit blurry and the edges are still mainly left blank. But the most peculiar thing is the artist’s signature, which reads:

   

Which reveals that this painting was made by an algorithm, defined by the formula above. When this painting went under the hammer it sold for an astronomically high amount of $432,500 (This is not only due to the look of the painting, it is also caused by the fact that algorithm art is a new concept, which is highly valued in the art sector). The painting is created by Obvious, a collective from Paris that is exploring the interface between art and artificial intelligence. The method they use is known under the name GAN, which stands for ‘generative adversarial network’. It’s an algorithm consisting of two parts, a generator and a discriminator. A database consisting of 15,000 paintings were put into the computer and are analyzed. Based on these paintings the generator creates a new image. Then the newly generated image is put through the discriminator. This algorithm tries determines which image is the machine-made one. If the discriminator fails in identifying the machine-made image the process is successful. However, since a human is more sensitive to the facial features one could say that it is not extremely realistic and that the algorithm is easily fooled.

Conclusion

Summarizing, it can be seen that the abilities of algorithms have grown substantially through the years. Not only is it now possible for a machine to classify art and recognize different objects in a painting, they are also capable of creating art themselves. Starting with abstract pictures the skills have increased and even portraits are being made nowadays. It has to be said that this art is made by looking at other work and trying to recreate this. However, these techniques have also been used by the human counterparts of these ‘artists’. It can still be argued whether or not these images could be classified as art. Solving this argument could be done with some kind of visual Turing test: ask humans if they can tell the difference between human-made and machine-made art. It has been found that humans find it increasingly difficult to distinguish these paintings.
However, maybe this question is not important. Art is meant to express imagination, ideas or skill intended to be appreciated for their beauty or emotional power. More and more people are inspired by art made by a machine. To conclude, it is extremely likely that artificial intelligence will change the world of art. But for now it must be seen as an extension, a new way for artists to express their ideas.

---

Dit artikel is geschreven door Jochem Hak