The image above, “A Sunday Afternoon on the Island of La Grande Jatte,” was painted in 1884 by French artist Georges Seurat. The black lines crisscrossing it are not the work of a toddler wreaking havoc with a permanent marker, but that of neuroscientist Robert Wurtz of the National Eye Institute in the US. Ten years ago, he asked a colleague to look at the painting while wearing a contact lens–like contraption that recorded the colleague’s eye movements. These were then translated into the graffiti you see here.
Art lovers may cringe, yet it is likely that Seurat would have been intrigued by this augmentation of his work. The movement Seurat kick-started with this painting — Neo-Impressionism — drew inspiration from the scientific study of how our vision works. Particularly influential was the pioneering research of Hermann von Helmholtz, a German physician, physicist and philosopher and author of a seminal 1867 book, Handbook of Physiological Optics, on the way we perceive depth, color and motion.
One of the questions that occupied Helmholtz, and quite possibly Seurat, is why we don’t perceive the constant eye movements we make when we are scanning our surroundings (or a painted representation of them). Consider that the lines above were drawn in just three minutes. If we saw all those movements as we made them, our view of the world would be a blur of constant motion. As Wurtz and his Italian colleagues Paola Binda and Maria Concetta Morrone explain in two articles in the Annual Review of Vision Science, there’s a lot we know about why that doesn’t happen — and more yet to learn.
The Secrets of Seeing
Beginning with the basics: The only things we can ever hope to see are those that send or reflect light toward our eyes, where it might end up hitting the retina, a layer of nervous tissue that covers the back two-thirds of the inner eyeball. There, the complex image of whatever we are looking at is first translated into activity of individual light-sensitive photoreceptor cells. This pattern is then transmitted to a variety of neurons in the retina that specifically respond to certain colors, shapes, orientations, movements or contrasts. The signals they produce are sent up to the brain through the optic nerve, where they are interpreted and put back together in a progression of specialized areas in the visual cortex.
Yet to transmit all the information that reaches our retina at the resolution we are used to would require an optic nerve with roughly the diameter of an elephant’s trunk. Since that would be rather unwieldy, only one tiny area of the retina — called the fovea — provides this kind of resolution. So in order to grant all the interesting features of our environment their moment in the foveal spotlight, we move our eyes around — a lot — in darts that scientists call saccades. (French for “jerks,” the word was coined in 1879 by French ophthalmologist Émile Javal.) Saccades are guided by what we are paying attention to, even though we are often blissfully unaware of them.
There are a number of reasons why these movements don’t transform our view of the world into a blur of motion. One is that the most distinct things in our field of view may render us blind to other stimuli that are fleeting and faint: Objects that are in clear sight when our eyes don’t move are likely to make a more vivid impression than the blur in between. Scientists refer to this phenomenon as visual masking, and it is thought to be very common in real-life situations where a lot is going on at the same time.
If scientists set up experiments in a way that avoids this visual masking, it reveals that our brains can perceive the less noticeable things. This can be done, Morrone explains, by showing people nothing but very faint and short-lived visual stimuli on an otherwise empty background. Under these conditions, surprising things may happen. When researchers create a motion very similar to what we should normally perceive when we make a saccade, by rapidly moving a mirror around in front of people’s eyes, those people do report seeing movement — and they often find it rather disturbing. Since we do not notice our constant saccades, this suggests that the brain specifically suppresses the signals that reach our retina while a saccadic eye movement is in process. And indeed, experiments have shown that if something appears during a saccade, we may miss it entirely.
But suppression does not adequately explain why the image in our mind’s eye is so stable. If we were to see our surroundings from one angle, then see nothing, and then suddenly see it from another angle, that would still be unsettling. Instead, as Wurtz and others have shown, a kind of remapping happens even before we move our eyes. In experiments with macaques that were trained to make predictable saccades, brain cells that receive signals from one particular spot in the retina switched from responding to things currently in view there to things that would show up only after the saccade. And that happened before the monkeys moved their eyes. In this way, Wurtz thinks, the current image is gradually replaced by the future one. Confusing signals
So how do these brain cells know in advance that a saccade is on the way? Scientists theorized for many years that this would require them to receive an additional signal, from the brain area that gives the command for eye movement. And they have shown that such signals do occur, arriving at areas of the brain involved in coordinating what we see and where we will look next. Wurtz and others believe that this kind of signal nudges brain cells to start responding to things that their part of the retina will see only after the saccade.
All of this is very likely to work almost exactly the same way in humans as it does in monkeys. But if you ask people what they see right before a saccade, as Morrone and Binda have done, they don’t report a gradual replacement of one image by another before their eyes move. Instead, anything they’re shown during a 100-millisecond period right before the saccade becomes visible only after the saccade ends. The result of this delay is that stimuli appearing at different times within that short period before the saccade may all be perceived at the same time — 50 milliseconds after it ends.
And if these stimuli are sufficiently similar, they might be perceived as fused together into one thing, even when they were shown at slightly different times or places before the eye movements. Binda and Morrone call this time window right before the saccade the confusion period. The things we see may literally be con-fused — fused together — by our vision, and then more conventionally confused — mistaken for each other — in our minds.
In real life, this fusion of similar elements across space and time during saccades might actually help to prevent confusion, because the continuity helps us to grasp that things we saw before and after a saccade are the same, even if they have moved or if the light has shifted. So though the mechanism may seem very sloppy, Binda and Morrone believe this sloppiness usually works to our advantage.
A similar sort of desirable imprecision might be what allows us to enjoy Seurat’s painting in the first place. Instead of a perhaps more accurate perception of colorful collections of distinct dots, a beautiful Sunday afternoon emerges. Hats off to that — or, as the French would say: “Chapeau!”
10.1146/knowable-061219-1
Tim Vernimmen is a freelance science writer based near Antwerp, Belgium. While writing this article, he moved his eyes at least 200,000 times, though he admits he lost count at some point. This article originally appeared in Knowable Magazine, an independent journalistic endeavor from Annual Reviews. Sign up for the newsletter.
