How the Eyes Perceive a Picture
Have you ever wondered how the eye reads a picture? Why, looking at a photograph, we so often feel that certain parts of the image pull the eye so strongly that it is impossible to tear ourselves away and focus on other details? As an attempt to answer this question, a group of psychologists and physiologists in the sixties created a theory of visual perception. The theory was developed further: at present there are at least three mathematical frameworks that make it possible to model the movement of the pupil as it reads a picture, and to relate that movement to the concentration of attention on particular parts of the image.
Two or three years ago I was deeply involved in modelling attention during the viewing of images, and the other day I was asked to show how such a little program works. I rummaged in a dusty corner for the archives of the source code, unpacked them, started compiling, decided to fix a couple of bugs in the algorithms and… got carried away! I present to you the fruit of two days’ effort: several pictures and two different ways of modelling how a person perceives a picture.
The pictures are presented as triptychs. The first panel is the original picture. The middle panel is a heat map. The more intense the green glow, the higher the probability that this region will attract your attention. The last picture is the dynamics of the gaze. This model shows how the gaze glides across the image, and where it may go next. The gaze moves more readily from lighter parts toward darker ones, and the reverse — to shift the focus of attention from darker parts to lighter ones, some effort will probably be required.
When looking at the pictures, one needs to understand that the mathematical framework which models the focusing of human attention does not take into account the psychological aspects of perception — for example, ones like these or ones like these. What the pictures show is how the human gaze moves if it does not discover any recognisable images in the picture.
The first picture shows how a typical photographic subject of the “some object in the middle of the frame” type is perceived. It is especially interesting how the gaze strives toward the centre, yet does not quite reach it. The gaze seems to wander, caressing with light touches the central area in which the object is inscribed. In the picture that shows the dynamics, all the compositional features are beautifully visible in the form of secondary foci of attention and a vector that draws the gaze upward.
And here is an ordinary landscape. Notice that if you look from the bottom of the picture, the gaze is drawn toward the trunks of the trees, whereas if you look from the trees upward, or from the top downward, then in the middle of the sky a “potential well” is clearly visible, into which the gaze involuntarily slides.
A little about web pages. Which parts of a page, do you think, are the most attractive to attention? What is the most important thing to show? Advertising, of course!
A landscape with pretensions to composition. And one immediately sees how that composition is off — if only those people were shifted down and to the right by exactly one square, the golden section would have been observed! As it is, attention concentrates between the edge of the picture and the silhouettes of the people.
And I will show this photo only to demonstrate how automatic analysis starts to lag because of the psychological aspects of perception. When looking at a photograph, people pick out faces and unconsciously pay more attention to them. The car, in this case, is also perceived as the muzzle of some outlandish beast. If we ran eye tracking on this picture, we would notice that the maximum of attention would fall “on the forehead” of the Chrysler and on the person’s face. This, by the way, applies to the next picture as well.
Our “everything” — the Mona Lisa. Let us forget that faces attract the gaze and look at how the painting is perceived if one considers it as a whole. The heat map will give us nothing here, but the dynamics reveal the most interesting things! It turns out that to the right of the face there is a square pointing at the nose, with sides proportional to the golden section. What is more, if one looks closely at the quadrilateral polygons covering the eyes, it turns out that their sides almost correspond to the harmonic series (the discrepancies fit comfortably within the algorithm’s margin of error)! After that, just try not to envy Leonardo’s knowledge of geometry…
And now let us move on to my favourite Impressionists. The heat map immediately shows that either the algorithm is lagging, or the painting is too complex to perceive. Is this not why so many people find Van Gogh so hard to take in? A heap of extraneous centres of attention, creating a mesh of almost white noise… Against that background it is so difficult to catch the flowers in detail; only the pot, perhaps, lends itself to thoughtful viewing. But the visualisation of the dynamics changes everything! It turns out that if one steps back from the details of the brushstrokes and perceives the painting from a distance, a clear diagonal axis comes into view, with an entry point on the table and an explosive finish in the form of the flowers! The painting is not static; the painting is alive!