Imagine someone chopping vegetables on a cutting board. No, really. Before reading on, take a moment to truly visualize the scene as vividly as possible.
Try as you might, odds are your mental image will omit some basic features. Was the picture thorough enough to specify the size of the knife or the material of the cutting board? The kind of vegetables? The person’s gender? The color of their hair and clothes?
A recent study published in Cognition found that most people respond “yes” to some of these questions and “no” to others. Of course, there’s nothing unusual about failing to fill in every detail of a mental scene; even in reality, we often overlook the subtle nuances of our surroundings.
What is strange, however, is that in imagination we seem to leave out fundamental properties — things we couldn’t fail to notice if they appeared before us in real life.
The Mental Imagery Debate
We tend to think of our memories and daydreams as akin to visual perception. We see these images in “the mind’s eye,” after all. But there’s actually a decades-old dispute over the nature of mental imagery — or, more precisely, over how the brain stores and represents visual information.
Historically, lines were drawn between the pictorial camp, which held that mental images really are like pictures, and the propositional camp, which considered them more like sentences that encode visual data in semantic form.
By now, the debate has more or less eased into a middle ground.
In a 2015 article, Harvard neuroscientist Stephen Kosslyn and Joel Pearson, a neuroscientist at the University of New South Wales, argued that humans “represent information in multiple ways, and that such representations can be used flexibly.”
In this view, we toggle between picture and sentence — and perhaps even other formats.
The paper’s title, “Ending the imagery debate,” announced the authors’ intent. And a resolution was made possible, they explained, by an influx of evidence from neuroimaging and computer modeling.
“People have been trying to find out what a mental image is, but there’s no one mental image,” says Nadine Dijkstra, a senior research fellow at University College London’s Wellcome Centre for Human Neuroimaging who was not involved in the Cognition study.
“Sometimes imagery is like language, sometimes it's like pictures,” she says.
Read More: Can AI Read Your Mind?
A Cognition Study of Unfinished Paintings
That dual nature shines through in the Cognition study. The researchers asked hundreds of participants to imagine scenes, each one following a formula: subject, primary object, secondary object.
To take one example, a person (subject) knocks a ball (primary object) off a table (secondary object).
After visualizing, researchers asked participants whether they imagined specific properties for each of those three elements — say, the color of the ball, the shape of the table or the height of the person.
The vast majority of people didn’t commit to at least some descriptors that would be readily apparent in a true visual image, showing that mental images are seldom fully formed.
About 4 percent of people have a condition known as aphantasia, meaning they can’t form mental images at all. But non-commitment, as evidenced here, is different. It seems to be a universal feature of the human brain, affecting even those with otherwise vivid imaginations.
Read More: Scientists Aren't Sure How the Inner Voice Works
Filling in the Gaps of Our Imaginations
If we didn’t have psychologists to point out our imaginative shortcomings, most of us would probably never notice them. Clothing, for example, is an oft-neglected aspect of mental images.
When asked in “yes/no” format whether they imagined it initially, most participants said they didn’t. Yet when asked open-ended questions, nearly all gave rich descriptions of pink blouses, long trench coats and even “black boots with a zipper on the sides and decorative corset-type string pattern on the fronts.”
Convincing, right? Harvard psychologist Tomer Ullman, one of the study’s authors, interprets this as on-demand embellishment.
“People are making it up on the spot,” he says. “‘Oh, you want the color of the bicycle? Sure thing, boss!’” This aligns with previous research on memory, which has shown how seamlessly we weave true and false information together in descriptions of the past.
Dijkstra notes that during confabulation (the psychological term for this self-deceiving process), it’s likely the participants really do see what they’re describing in the mind’s eye — even if they didn’t the first time around.
“You can get the brain to treat imagery as a picture, but you have to put effort into it,” she says. “If we don’t have to imagine those details, we’re just not going to bother.”
Read more: How Our Brains Build Models of the World While We Sleep, Daydream and Hallucinate
Staples of Imagination
But if clothing is relatively unimportant in mental imagery, other components seem almost indispensable.
In the person-knocking-a-ball-off-a-table scene, nearly everyone envisioned the size of the ball, for example. And a strong majority also pictured its color and trajectory, the person’s gender, and the size and shape of the table.
These patterns seem to reveal a hierarchy that, above all, favors spatial properties: where an object is, where it’s going, how much space it’s taking up.
It’s as if we start with “a finite amount of commitment juice,” Ullman says, and it drains away as we construct an image. By the time we’ve finished the important parts, we have little left for frivolous decoration.
The important parts may vary based on context, however, perhaps tilting toward non-spatial properties. If you picture a bride tossing a bouquet or a clown blowing balloons, clothing will obviously be more salient.
Whatever we portray on our internal canvas, we have a lazy, time-strapped brain to contend with. Ullman and his colleagues put it memorably in the conclusion of their paper: “While the imagination may indeed be a good artist, it’s on a deadline, and stingy about paint.”
Read More: A Wandering Mind Is Part of Our Evolution
