Custom-made Jaron Lanier bobblehead by HeadBobble.com
In the 1990s, I had the good fortune to run a project out of the Internet2 engineering office called the National Tele-Immersion Initiative. Several colleagues, including Henry Fuchs of the University of North Carolina at Chapel Hill, Ruzena Bajcsy, now at the University of California at Berkeley, Andries Van Dam of Brown University, and I built and tested the very best tele-immersion rigs possible at that time. We were able to display people to one another holographically in full 3-D. We couldn't address all the known subtleties of interpersonal communication, but we probably got closer than anyone had before—close enough that I was able to develop a theory about the kind of device that might convey people to one another well enough to make the century-old predictions of presence-at-a-distance come true.
But before I tell you about that, let's backtrack to the first question. Why do you have a neck?
Evolution has produced a startling variety of configurations of eyes, ears, and other sensory organs. The most startling collection of eyes is found on the starfishlike brittle star, Ophiocoma wendtii. It has a bejeweled body almost completely covered by a compound eye, a multitude of crystalline lenses.
Animals closer to the top of the food chain have only two eyes, however. This is true not just for humans and other vertebrates but for squids and other cephalopods as well. It is easy to see why two eyes are better than one (enhanced depth perception and redundancy, for example). But why didn't evolution produce any large animal with more than two eyes? The answer is that two eyes that are highly maneuverable can see better than any number of eyes that are stuck in place—so well, in fact, that adding extra eyes would not add much performance.
It is natural to think of the brain as a computer and an eye as a simple camera connected to it. Here is a more accurate metaphor: The head is a spy submarine sent on a mission to perform a multitude of little experiments to learn more about its environment. These microexperiments are often carried out by constant, subtle changes of the position of the head.
By continually moving our head around in order to scan the scene, we simulate the effect of having far better eyes than we actually do, and in a far wider variety of placements. The motion of the head increases the quality of the image available to us: The brain integrates images seen from different positions to see more detail than is projected on the retina in a single moment. This motion is fundamental to human sight. If you immobilize your head in a vise, you will see far less well. If you also stop the motion of your eyeballs, you will soon cease to see at all. The world seems to vanish into gray.
There's a related problem concerning the screen. Fixed-position videoconference screens create a virtual tunnel through which people see each other. Faces fall out of sight if the people on the other end move their heads much, and as I've argued, heads naturally want to be in motion. I suspect this simple problem is one of the reasons videoconferencing has never caught on.
Now here's the interesting part that ties the two questions together. Everything changes if the whole telepresence device is put on top of a neck. And that's exactly what we are trying next.
The new experiment is called Cocodex, for "compact cooperative desktop explorer." I started work on it while I was the visiting scientist at Silicon Graphics. Oliver Staadt at the University of California at Davis and others are helping me develop prototypes of a compound gadget—including cameras, microphones, and a holographic display—that sits on top of a robotic arm. As you move your head, it follows you, giving your interlocutor a high-resolution view of all the subliminal details that matter. (When you look away, on the other hand, the Cocodex screen with its cameras doesn't stay in front of your face, but the other person will still see you from the side, able to judge the precise manner in which you are not maintaining eye contact.)
As it happens, putting the whole rig in motion solves a plethora of other problems as well. For instance, it becomes easy to assemble a lot of people from different locations into a single virtual space for lectures or meetings, something that is hard to do with fixed displays and cameras because they don't let people look around. It also frees you from needing a videoconference room with built-in screens and cameras.
It might sound strange at first to design a communication device that bobs around just as our heads do when we talk and listen. But when nature settles on the same solution to a difficult problem more than once, as it did with vertebrates and cephalopods, it is worth paying attention.
Although the initial prototypes of Cocodex will rest on metal robotic arms, we hope eventually to use soft organic designs, perhaps based on an elephant's trunk. In the late 19th century, technologies of the future were imagined to be cold and rigid, like steamships. Maybe the road to successful videoconferencing will require devices that not only move like graceful animals but look like them too.
Previous Jaron's World columns: