Why Quantum Mechanics Still Stumps Physicists

Here’s a thought experiment: Imagine astronomers didn’t really believe that Earth orbits the sun or that our world turns daily on its axis. What if they viewed the heliocentric model of the solar system merely as an abstract mathematical tool to track planets and stars with great precision, not as a literal description of the way things are? What if they claimed we can’t truly know whether the sun orbits Earth or vice versa and, moreover, that such questions were not even worth asking?

It would be preposterous. No respectable scientist would ever entertain such notions — except when it comes to the most powerful theory in the history of physics: quantum mechanics. More than a century after its birth, quantum mechanics, the physics of atoms, photons and other particles, remains as baffling as ever. Experiments have repeatedly confirmed the theory’s weird predictions with phenomenal accuracy — to a dozen or more decimal places in some cases. Technologies derived from it drive the world’s economy: The electronics industry as we know it wouldn’t exist without quantum mechanics. It explains why the sky is blue and how stars generate their light.

And yet, despite the theory’s unquestioned dominance and practical significance, physicists still don’t agree on what it means or what it says about the nature of reality. Some physicists deny that quantum mechanics describes any sort of objective reality.

At least a dozen interpretations of quantum mechanics vie for physicists’ hearts and minds, each with a radically different take on reality. Adán Cabello, a physicist at the University of Seville in Spain, recently summed up the confusing, incompatible gaggle of viewpoints as “a map of madness.”

There’s the Many Worlds model, which posits the existence of innumerable parallel realities. If that seems a tad extravagant, you might prefer QBism (pronounced “cubism”), where the quantum world and the scientists who observe it are inextricably bound together in an unpredictable, interactive universe. The central issue is that physicists don’t know what the most basic equation of quantum theory — a mathematical formulation called the wave function — actually represents. Does it describe a fundamental feature of the physical world? Or is it instead just a handy way to predict experimental results?

“There is no standard interpretation,” says Antony Valentini, a theoretical physicist at Clemson University. “It’s extraordinary. I don’t know of any comparable episode in the history of science.”

Where does that lack of consensus leave physicists? After all, quantum mechanics isn’t just a branch of physics; it is modern physics. “Most of the things that people are doing on almost every floor of every physics department in the world are quantum in one way or another,” says Matt Leifer, a physicist at Chapman University in California.

If physicists can’t agree on — or don’t know — what their reigning theory is all about, does it mean they’ve hit a wall in terms of understanding the world? Recent efforts to rule out some interpretations haven’t brought us any closer to an answer. If there’s one thing certain about the quantum world, it’s that nothing’s ever settled.

The confusion dates to the early days of quantum mechanics, in the 1920s, when Niels Bohr clashed with Albert Einstein. Bohr, an almost oracular figure in 20th-century physics, argued that when studying the atomic world, physicists must give up the notion of a reality that exists independently of their own measurements. The message of quantum mechanics is inescapable, he said, and exceedingly strange: Atoms and all other particles do not possess definite positions, energies or any properties until they are measured in an experiment. To be clear, it’s not just that physicists don’t know what the properties are; the properties literally only come into being at the time of the measurement.

Einstein categorically rejected Bohr’s view. While strolling the grounds of the Institute for Advanced Study in Princeton University one moonlit night, Einstein famously asked a colleague, “Do you really believe the moon is not there when you are not looking at it?” Einstein remained convinced until his death that quantum mechanics was only a steppingstone toward a deeper, more comprehensive theory that would make sense of the uncanny phenomena of the quantum world.

What makes quantum mechanics so confounding? Consider the following iconic, oft-repeated experiment: A beam of light shines through two parallel slits cut into a barrier and falls on a strip of photographic film beyond the barrier. Since light itself consists of a stream of particles — photons — it seems reasonable to assume that the photons pass through one slit or the other en route to the film. And if physicists set up the experiment with a photon detector at each slit, that is indeed what they see: Photons hurtle randomly through either the first slit or the second, which results in two separate clumps of dots forming on the film.

A slight adjustment, however, profoundly alters the results. If physicists remove the photon detectors, the pattern created on the film changes completely. Instead of two clusters of dots, alternating light and dark bands appear across the film, what physicists call an interference pattern. That pattern could form only if each individual photon somehow spread out like a wave and went through both slits simultaneously. Bright bands develop on the film where two wave crests coincide; overlapping crests and troughs create the dark bands. In other words, photons behave like particles with detectors present and like waves without detectors.

For Bohr, this showed that the objects we consider particles don’t have a definite existence until they are observed. On the very smallest scales, reality is blurry, not sharply defined — at least when no one is looking.

Since everything ultimately consists of those blurry particle-waves, why don’t we see quantum effects in our everyday lives? Why aren’t people, trees and everything else as wavy and indistinct as the atoms they’re made of? The short answer is no one really knows, hence the crazy cornucopia of quantum interpretations. In one way or another, the manifold versions all seek to answer a single question: Are these “quantum waves” as real as the ground beneath your feet, or are they purely mathematical constructs without any physical existence?

Some of the attempts to answer that question have, if anything, only added an extra dose of weirdness to the quantum brew. Perhaps the strangest of all the interpretations is the one first proposed in 1957 by Princeton physicist Hugh Everett. In his doctoral thesis, Everett argued that the equations of quantum mechanics should be taken at face value: Quantum waves are real, with each possible wave in effect representing a separate, independent reality. According to the Many Worlds theory, as Everett’s idea is now known, every possible physical event actually takes place — in its own parallel universe. The implications are staggering. At this moment, for example, an uncountable number of yous are reading this, possibly scratching their heads.

For all its universe-begetting outlandishness, the Many Worlds view has many advocates. “In a certain sense, it’s very conservative,” says David Wallace, a philosopher of physics at the University of Southern California. “It leaves the physics unchanged, and it holds onto the idea that scientific theories are supposed to give us a description of what is going on, even if what’s going on is much weirder than we thought.”

But, of course, there’s no consensus. Many physicists prefer the idea that quantum waves — or more precisely, their mathematical representations, wave functions — don’t correspond to actual physical entities; the wave function simply reflects the probability that a particular experimental outcome will occur. This eliminates the paradoxes of quantum mechanics without the necessity of conjuring innumerable universes. Case in point: Erwin Schrödinger’s hapless cat.

Schrödinger, a contemporary of Bohr and Einstein, and one of the founders of quantum mechanics, devised his famous thought experiment to highlight what he saw as the absurdity of Bohr’s ideas. His Rube Goldbergian experiment has six components: a steel box, a cat, a radioactive element, a Geiger counter, a hammer and a vial of cyanide. The cat is put in the steel box; the lid is closed. No one can see what’s happening inside. During any given interval of time, the radioactive element may or may not emit a high-energy particle. If it does, the Geiger counter detects it and triggers the hammer to smash the vial, releasing poisonous fumes that kill the cat. If it doesn’t, the cat survives.

According to the rules of quantum mechanics, the radioactive particle exists as a wave function in all its possible states — both emitted and not emitted. A single, definite state crystallizes only upon measurement. What does that mean for the cat? Is it both alive and dead until someone opens the box for a look? Schrödinger ridiculed the notion of a cat — or anything — existing in two different conditions at once.

To some physicists, Schrödinger’s thought experiment shows that the wave function can’t be real, that it represents nothing more than the probabilities of different events. The cat is alive or dead, not alive and dead. The cat’s condition is determined before anyone opens the box. The only thing that changes when the box opens is our knowledge of the cat’s fate.

In our everyday world, it seems, the laws of quantum theory lead to absurd results. But what about that two-slit experiment? If the wave function isn’t actually real, what creates those light and dark bands?

Four years ago, Matthew Pusey of the Perimeter Institute in Waterloo, Ontario, Jonathan Barrett, then at the University of London, and Terry Rudolph at Imperial College London published a paper in Nature Physics where they argued convincingly that quantum waves must be real. In an interview with Nature, Clemson physicist Valentini said, “I don’t like to sound hyperbolic, but I think the word ‘seismic’ is likely to apply to this paper.”

Pusey, Barrett and Rudolph’s theorem, known as PBR, uses a sophisticated mathematical argument to show that any interpretation of quantum mechanics that doesn’t treat the wave function as a real object invariably leads to results that contradict quantum theory itself. If they’re right and the wave function is real, interpretations like Everett’s Many Worlds, which take the reality of the wave function as a given, could start to seem more plausible. In that case, Schrödinger’s cat would be alive in one universe, dead in another. Alternatively, fans of Bohr’s view could claim that the cat exists as a fuzzy quantum wave inside the closed box; the frazzled feline would indeed be in a combined alive-dead state until someone takes a look.

To get the gist of the PBR argument, consider a simple card game between you and a dealer involving two decks of cards. One deck holds only red cards, the other deck only aces. The dealer gives you a card and asks which deck it came from. In most cases the answer will be easy. But for two cards — the two red aces — there’s no way to tell. The aces could come from either deck. That’s fine with a deck of cards, but the quantum version doesn’t play so nicely.

If the wave function is not a real physical object and instead only measures experimental probabilities, then more than one wave function could describe a single physical state, say the position of a photon (just like that red ace could come from either deck). The notion that a slew of different wave functions could describe the same underlying reality falls apart in quantum mechanics, says Pusey. Reality can’t come from two decks. He and his colleagues showed that the probabilistic interpretation becomes a problematic one.

“It leads to so many possibilities that you can prove that quantum mechanics wouldn’t allow it,” says Pusey. “It wouldn’t make sense for one physical state to be compatible with so many different wave functions. The predictions those wave functions make are so different.” The PBR theorem shows that quantum states must therefore correspond uniquely with something that’s real — that is, it proves the wave function actually exists and is not just an abstract measure of probability.

Despite some rave reviews, the PBR result hasn’t changed many minds. “I was a bit disappointed that the people who liked it were the people who already believed the conclusion,” says Pusey. The naysayers instead deny one of PBR’s main assumptions: that there exists an objective reality we can measure in the first place.

The notion of a completely objective reality is the bedrock principle of science, which is the main reason Einstein was so uncomfortable with Bohr’s “nothing exists without observation” take on quantum theory. Yet Christopher Fuchs, a physicist now at the University of Massachusetts, and Ruediger Schack of Royal Holloway University of London disagree. They contend that Bohr was on to something: Our notion of an objective reality needs modification. The physical world cannot be separated from our own efforts to probe it. How could it be otherwise, since we ourselves are embedded in the very world we’re seeking to understand?

They call their way of looking at quantum mechanics QBism, a modified version of a theory they developed with University of New Mexico physicist Carlton Caves called Quantum Bayesianism. QBism combines quantum mechanics with Bayesian probability, a variation on standard probability in which the odds of any given event are revised as one gains more knowledge of the many possible conditions tied to the event. For example, if a patient complains of headaches to a doctor, the initial odds of a diagnosis of brain cancer might be low. As the doctor examines the patient, the odds of a cancer diagnosis may go up or down.

QBism applies similar reasoning to physics experiments: Whenever physicists perform an experiment, they are updating their own subjective knowledge. There is no fixed underlying reality that different observers can independently experience. Just as a doctor must assess each patient individually, so too must a physicist approach the fresh, ever-changing phenomena presented by the quantum world. In QBism, the experimentalist cannot be separated from the experiment — both are immersed in the same living, unpredictable moment.

“If QBism says one radical and important thing about the nature of reality, then observer participancy is it,” says Schack. “Subjects matter. And reality, if QBism is right, cannot be conceived without always including the subject. That’s certainly a bold statement about the real world, about reality. It’s just a feature of reality that is very fundamental.”

Quantum theory, Schack says, offers profound observations about the real world, but the theory itself is not a description of the world. He posits that the right way to think of quantum mechanics is as a set of rules about how to correctly conduct experiments.

“Whether you see a wave or particle depends on what question you ask,” says Schack. “What do physicists do? They choose experiments. You could describe any experiment as a gamble on the outcome. Quantum mechanics is a useful guide to action: It tells you how to put together your experimental apparatus so that it works in the end.”

Schack says he and Fuchs like to use a term they’ve borrowed from the American philosopher William James, who saw reality as being “malleable.” QBism, says Schack, makes the same point. What sort of universe do we inhabit? Is it like a giant machine, with the future evolving from the past according to immutable laws? Or is it inherently interactive? “Why would you want a clockwork universe?” he asks. “QBism gives a much richer universe. It’s a reality in which we matter far more than we ever could in a clockwork universe.”

If QBism is right, if the wave function isn’t real and quantum theory doesn’t give us a direct description of reality, it leaves unanswered the most basic of all questions: What then is the quantum world actually like? What is it made of? Particles? Waves? Something beyond our ability to imagine? For theoretical physicist Valentini, the answer has been there from the earliest days of quantum theory.

In 1927, the French physicist Louis de Broglie, who first proposed that particles could behave like waves, developed an interpretation of quantum mechanics called pilot wave theory, where waves and particles are both equally real. Each particle rides its own wave. The pilot wave is a bizarre thing — it exists in multiple dimensions — but it is a real physical object.

Pilot wave theory explains the strange two-slit experiment: A particle always goes through one slit or the other; at the same time its pilot wave travels through both slits. But there’s no wave-particle paradox because the experimental apparatus and the wave-surfing particle all form one interdependent system described by a pilot wave. Adding or removing a detector from the experiment changes the system’s pilot wave and the pattern on the screen.

Bohr and other physics luminaries rejected de Broglie’s idea, though, in part because it didn’t provide any way to predict the exact paths of particles. In the 1950s, David Bohm, a leading American physicist, did some additional work with de Broglie’s idea, but for the most part pilot wave theory languished until the early 1990s when it hooked Valentini as a grad student.

Valentini has devoted his career to almost single-handedly resurrecting the pilot wave idea. Now his years of work actually have a chance — a small one, he admits — of being vindicated. Of the many interpretations of quantum theory, pilot wave theory is unique in that Valentini has found a way in which it might be experimentally tested. No other interpretation of quantum mechanics can make that claim. Many Worlds, Bohr’s interpretation and others are all experimentally indistinguishable — they reproduce the results of standard quantum theory. But if Valentini is right, certain effects predicted in pilot wave theory may have left an imprint on the cosmic microwave background, the primordial radiation left over from the Big Bang that still pervades all of space.

The temperature of that radiation is almost a perfectly uniform 2.725 degrees Celsius above absolute zero. Detailed observations, however, have found slight variations in the radiation. Standard quantum theory can explain nearly all of these variations, but in 2015, new data released by the European Space Agency’s Planck spacecraft revealed evidence of small anomalies in the background radiation. And that is just the kind of thing Valentini has been looking for. While conventional quantum theory predicts that random quantum fluctuations in the early universe have left celestial imprints, pilot wave theory predicts fluctuations that are less random, leaving slightly different wrinkles in the cosmic microwave background radiation.

“It’s tantalizing,” Valentini says. “We’re carrying out the analysis partly to understand things better and partly to see what the data can tell us about the predictions that we have.” Another two years of data and analysis should settle the question.

Valentini also feels encouraged by the PBR theorem because it lends support to a central tenet of pilot wave theory: The wave function is real. Nevertheless, he realizes the odds of his life’s work being confirmed are slim. “Who knows what will happen?” he says. “It may be 20 years of work down the drain. We don’t know. You have different camps pushing hard for their own interpretation. But really, if we’re going to be honest, as scientists, if a member of the public asks us what is the meaning of our most basic theory of physics, I think we all have to say we don’t know.”

Tim Folger is a contributing editor to Discover and series editor of The Best American Science and Nature Writing, an annual anthology. He lives in New Mexico.

[This article originally appeared in print as "The War Over Reality."]