Vision Modelling. Part One. An Excursion into the Eye
1 Excursion into the eye — 2 Perception — 3 Geometry of vision — 4 Eye tracking — 5 How to catch the gaze — 6 Modelling eye tracking
To understand how a person perceives an image, one has to begin with the organ of vision — the eye. An important point for what follows, besides the anatomical structure of the eye, is the limitations of the eye’s resolving power, which I shall describe here. If all of this is already familiar to you, you may simply skim the highlighted passages and go straight on to the second part.
Anatomy of the eye
So then, you all no doubt remember how the eye is built from the biology textbook, but here I shall tell you several genuinely surprising things that, for some reason, are not taught at school. First, though, I will still recall the structure of the eye (the illustrations are taken from David Hubel’s book Eye, Brain, and Vision):
The eye is held in the orbit by a group of six muscles; they also turn it up and down, right and left, should it be necessary to “look askance”.
The stream of light that falls on the retina passes through the cornea (it provides about 70% of the refraction of light), then through the pupil, which is the analogue of a camera’s diaphragm and is controlled by a group of radial and circular muscles that change its size, and then enters the lens, which provides the final focusing of visible objects. The lens is a gelatinous cushion that is compressed by radial muscles. When compressed, the lens changes its shape, thereby changing the degree of refraction of light and the focal distance.
All of this, in fact, is necessary in order to create a projection of the visible world on the retina. The retina is a part of the brain that separated from it at early stages of development, yet remains firmly connected to it by the bundle of the optic nerve:
The retina performs the primary processing of the incoming image before it enters the visual cortex of the brain.
The retina itself consists of three layers of nerve cells, while the photosensors that perceive light (the rods and cones) form the third, outer layer on its rear surface:
Thus, in order to reach the sensors, light first passes through two layers of nerve cells. On their rear side the photoreceptors are covered with melanin (a black pigment), which plays the same role as the blackening of the inner body of a camera. If there were no blackening, the light, having passed through the layer of rods and cones, would travel farther toward the brain, reflect from it and come back, spoiling our picture of the world in every possible way.
Photoreceptors
There are two kinds of photoreceptors:
- Rods — receptors sensitive to very weak illumination (mainly in the yellow-green part of the spectrum) and able to perceive very fine differences in luminance; therefore at high levels of illumination they become useless and are responsible for night, scotopic vision;
- Cones — receptors that require far more light in order to work, perceive stronger gradations in illumination and are responsible for daytime, photopic vision.
Since the discussion here is about the analysis of images on a computer monitor, I shall say no more about scotopic vision and the rods, and shall concentrate all attention on photopic vision and the cones.
Now then, cones are divided into three types according to the kind of pigment they contain, and each of these types is responsible for the perception of its own spectral band. Conventionally the types of cones are called “blue”, “green” and “red”, but in reality the spectrum they perceive goes beyond the bounds of those colours. Therefore many parts of the spectrum — yellow, for example — are perceived by two kinds of cones at once, and the combined sensation from the excitation of the “red” and “green” cones creates in us the sensation of yellow.
(more accurate graphs of the distribution of cone sensitivity can be found here )
Cones of all three kinds are present in roughly equal numbers, but since in the region of the yellow spectrum the number of cones that perceive it turns out to be twice as large, yellow seems far more intense than blue. The same is true of certain other colours (see the diagram given above).
“Pixels” of the retina
All photoreceptors are united by ganglion cells into blocks called receptive fields — each ganglion cell is matched with one receptive field. A single photoreceptor may belong to several receptive fields at once, and so the fields of two neighbouring ganglion cells overlap by 70–80%. In the simplest case, a receptive field resembles the analogue of a pixel of the light-sensitive matrix in a camera. But it is not as simple as that!
First, receptive fields differ in size depending on the region of the retina: thus in the region of the fovea — the zone of the retina that corresponds to the area of greatest visual acuity — a receptive field measures 1–2 mm (which corresponds to 2–3 minutes of arc), and toward the periphery it already reaches as much as 5 mm!
Second, the “pixels” of receptive fields differ from the pixels of a camera matrix in that there is a division among them — some respond to an “on” signal, others to an “off”. That is, some fields respond exclusively to a change in illumination from darker to lighter, others — vice versa.
Third, the fields differ in size depending on their kinds (they are conventionally labelled P fields and M fields). M fields are larger and have a high speed of response, but because of their size the picture they transmit will be of far poorer resolution than the picture transmitted by P fields. P fields are smaller, so the picture they transmit is more precise, but their speed of response is not great. Thus M fields register motion and answer the question “where is the object (the perception of motion and depth)”, while P fields deal with colour, shape and the details of the visual display, or answer the question “what are the objects like (colour vision and acuity)”. M fields deal with soft contrast, and P fields with sharp contrast. The longer the gaze remains in one place, the greater the role of the P fields becomes.
Fourth, not everything about these “pixels” is so simple: if the pixels of a photo matrix register exclusively a change of colour, receptive fields can, in addition, respond to lines, stripes, and various rectangular segments with sharp edges! That is, besides information that there is a change of colour or illumination, these “upgraded pixels” also transmit the information that a straight line is being observed, information about its length (so long that it runs out of sight, or the end of the segment is visible) and even about the direction (the angle of inclination) of that straight line!
1 Excursion into the eye — 2 Perception — 3 Geometry of vision — 4 Eye tracking — 5 How to catch the gaze — 6 Modelling eye tracking