Cat Vision Model

Looking at the World Through a Cat’s Eyes

Have you ever wanted to see the world through a cat’s eyes? Not metaphorically, but literally: with its colours, its field of view, its way of highlighting motion. About seventeen years ago I wrote quite a lot about modelling human vision, colour perception and colour blindness. There were articles on vision modelling, normalising an image to perception and an eye-tracking model. Everything then revolved around humans: how the eye builds an image, how a colour channel breaks, how one might try to model it on a computer.

A cat is a convenient and interesting next step. Its vision is well studied, differs quite strongly from human vision, yet remains recognisable. You can take an ordinary video from a phone and try to show how roughly the same scene might look to a cat.

How cat vision is organised

A cat is a dichromat. It has two types of cones with sensitivity peaks roughly in the blue region (about 450–460 nm) and in the yellow-green (about 550–560 nm). Red and green do not exist for it as separate colours: they collapse into a single yellow-green channel. This is quite close to what I once modelled for protanopia and deuteranopia, except that here the “loss” of red-green discrimination is the norm, not a defect. Yellow is also noticeably desaturated. As a result a cat’s world is not grey, but neither is it as colourful as ours: blues remain blue, while everything that is red-green for us shifts into yellowish-greyish tones.

The field of view is wider than a human’s. A cat’s horizontal angle of view is noticeably larger — roughly 200–270° against our 180–220°. The eyes are set further apart and not quite in the same plane, so the binocular field is stretched mainly horizontally. Vertically the advantage is less pronounced. There is also a visual streak — a zone of heightened acuity in the central horizontal band of the retina. Top and bottom periphery are noticeably worse: detail there blurs. In addition cats are relatively farsighted in the near zone — roughly up to 80 cm everything is softer than for a human with normal accommodation.

And separately — about motion. Humans have their own psychological peculiarities of colour perception (there was once an article on the power of colour), while cats have their own curious peculiarities of perception. Cats (and their visual cortex) are especially good at picking out objects that move horizontally. It is not simply that “they see motion”. The horizontal component of speed is higher priority for them. A fast horizontal object literally pops out of the background. The background itself can remain relatively calm.

All of this together gives a fairly clear picture of what is worth trying to model: colour, an expanded horizontal field, vertical periphery, and a special attitude to horizontal motion.

Colour model

When I once modelled colour blindness (on colour blindness and modelling colour perception, how colour-blind people see the world), the basic idea was simple: look at how the contribution of the cones changes and recompute colour in a space where one of the channels is weakened or absent. With a cat the logic is similar, only there are two channels from the start.

In the program colour correction is done first, before any blurring. RGB is converted into a simplified dichromatic representation: the blue channel and a yellow-green mixture are taken (red and green collapse), after which the saturation of yellow is reduced. This is not a full spectrophotometric model based on Govardovskii curves, but the principle is the same as in the old experiments with protanopia/deuteranopia: we do not “paint the picture yellow”, we recompute it as if the observer had different photoreceptors. As a result red and green objects become closer to each other in colour, blues remain relatively independent, and overall saturation drops.

Angles of view and the ellipse

A cat’s expanded field of view is primarily horizontal. When we look at a flat picture (a video frame) we do not have a true 200-degree view. So the ellipsoidal distortion here is a way of adapting a wider binocular field to a rectangular frame.

The horizontal is compressed more strongly, the vertical less so. This is not a spherical fisheye where the radius is the same in all directions. An ellipse results: the image is “squeezed” more noticeably at the sides than at the top and bottom. Visually this gives a sense of a wider gaze, although we are still looking at an ordinary rectangle. In the application this is done with a single UV-coordinate transform in the shader — without overlaying several different distortions on top of each other.

Motion and attention

The most interesting part is probably here. A cat does not simply “see motion”. Its visual system (and further — the brain) reacts especially well to the horizontal component. To show this, two things are done in the model.

First, when fast horizontal motion relative to the background is detected, playback slows down. When nothing in particular is moving in the frame, the video runs a little faster than normal (about 1.15×). When a fast object appears, the speed drops (to roughly 0.85×). This is not “slow-motion for beauty”, but an attempt to give the viewer the same feeling that a cat apparently gets: a fast horizontal object becomes more noticeable because time around it is, as it were, stretched.

Second, the object itself is highlighted by sharpness. The background and slower objects are blurred, while the fastest horizontally remains the sharpest. The blur is Gaussian; its strength depends on the speed of the fastest object in the frame. Slower moving things receive intermediate sharpness. The motion mask does not disappear instantly either: it decays along a logistic curve over roughly 150 ms, so the object does not “switch off” abruptly.

Detection is relative to the background. The camera may turn, the background shifts, and this shift is first estimated and compensated. What remains after compensation is considered object motion. It is precisely from the horizontal component of this residual that both the slowdown and the sharpness mask are built.

Vertical periphery is also blurred a little more strongly (visual streak): the top and bottom of the frame are softer than the central band. This is expressed more weakly than the motion effect, but it is present.

Why no neural network

There was an attempt to do motion and depth estimation via TensorFlow.js. The idea looked attractive: take a ready-made depth or optical-flow model and obtain a more accurate map. In practice two things became clear. Large models that gave an acceptable result are too heavy for a web page — inconvenient to send to the client and they noticeably slow processing. Small ones that could still be squeezed in gave too noisy and unstable a result on ordinary phone video. In the end the classical path remained: estimation of the global camera shift, residual and mask. It is simpler, more predictable and does not require loading hundreds of megabytes of weights.

How it is assembled

Everything runs in the browser, on the client. The video is loaded, normalised, then frame by frame goes into a WebGL shader. In the shader colour comes first, then ellipsoidal distortion, Gaussian blur of the background and vertical, and mixing according to the motion mask. The result is recorded and played back already with dynamic speed, and the original video’s sound runs at the same speed.

There is no magic. Motion is calculated classically: estimation of the global camera shift, residual, mask. Colour is a simplified channel recomputation. Distortion is a single geometric transform. But together this already gives the feeling that the picture looks at the world a little differently.

Scientific references

• Clark D.L., Clark R.A. (2016). Neutral point testing of color vision in the domestic cat. Experimental Eye Research, 153, 23–26. DOI: 10.1016/j.exer.2016.10.002. Behavioural confirmation of dichromacy with a neutral point near 505 nm.
• Daw N.W., Pearlman A.L. (1970). Cat colour vision: evidence for more than one cone process. The Journal of Physiology, 211(1), 125–137. DOI: 10.1113/jphysiol.1970.sp009270. Electrophysiological evidence for two cone types.
• Loop M.S., Millican C.L., Thomas S.R. (1987). Photopic spectral sensitivity of the cat. The Journal of Physiology, 382, 537–553. DOI: 10.1113/jphysiol.1987.sp016383. Spectral sensitivity under photopic conditions (peaks ~454 and 561 nm).
• McIlwain J.T. (1983). Representation of the visual streak in visuotopic maps of the cat’s superior colliculus. Vision Research, 23(5), 507–516. DOI: 10.1016/0042-6989(83)90125-6. Representation of the visual streak.
• Ringo J., Wolbarsht M.L., Wagner H.G., Crocker R., Amthor F. (1977). Trichromatic vision in the cat. Science, 198(4318), 753–755. DOI: 10.1126/science.910161. Earlier work suggesting trichromacy (later refined in favour of dichromacy).

Author: ymik