Perhaps the most intractable problem with the DataGlove was arm fatigue. Try holding your arm out without any support for a few minutes. You’ll soon start to notice little tremors in your arm muscles and before long you’ll wonder where all your strength went. We’re used to resting our hands at least a bit on the objects we’re manipulating, and that is one service a virtual object of the era just couldn’t offer.
Our difficulties actually led to some happy discoveries, including a new approach to physical therapy. You could toss virtual balls in VR using a first-generation DataGlove, but only very slowly; it helped if the balls moved in slow motion as well. This was an advantage for klutzy jugglers—make the balls slow enough and anyone can juggle. We realized that people could learn to juggle real balls by gradually speeding up virtual balls, a great training system. This concept is now commonplace in advanced rehab. For instance, there are therapy systems that use slowed-down VR to help people with injured limbs get up to speed.
Despite the flaws of the DataGlove, Mattel licensed the technology from VPL and sold it as the Power Glove, which worked with an early generation of the Nintendo game machine. Millions of these gloves were made, although they too suffered from woozy sensors and induced arm fatigue.
Beware the dystopic haptics in Minority Report
Many years after the glove era, I had a fun gig helping to brainstorm the movie Minority Report with Steven Spielberg, whom I met way back at that session with Lew Wasserman. The glove interface ended up with a starring role, and the filmmakers expended tremendous effort making that style of virtual interaction look viable when it actually isn’t. I thought it was a fitting memorial: a real design that didn’t quite work used to symbolize a fictitious future world that wasn’t quite working.
Fortunately, sensors and microprocessors have gotten much better and cheaper since then, so today the Wii controller can keep up with fast arm motions. The DataGlove used magnetism to track position and orientation; the Power Glove used ultrasound (pitches too high for humans to hear). The Wii controller uses a combination of technologies: It has a camera that watches invisible light beacons as well as accelerometers that measure motion in three directions, and both of these sensors are suitably fast. As a result, Wii players can throw and swing, things the human arm is great at doing. Unfortunately, the Wii controller isn’t any better at activities that involve continuity or construction than the old gloves were—and those are the most interesting things to do in virtual reality.
This all begs the question: What is the right way to manipulate virtual objects? When we can build the haptic interface of our dreams, what will it be like?
I’m betting on a strategy called the “octopus butler robot.” Some of my colleagues, including Henry Fuchs of the University of North Carolina at Chapel Hill and Ken Salisbury of Stanford, have worked on initial approaches to it, although we’ll need several generations of advances in computing and materials science before we can build it.
Here’s how it will work: Imagine you’re in a virtual world and you want to slam your hand down on a virtual countertop. Now suppose there’s an attentive robot placed nearby. (You don’t see the robot, of course, because you are looking only at the computer-generated virtual world.) The robot has an arm that is holding a tray, like a butler. As you start to slam your hand down, the robot calculates that you ought to hit a virtual countertop. The robot swoops in just in time, bringing the physical tray into alignment with the virtual countertop and creating the illusion for you that the countertop was there all along.
This setup would need to work for any virtual object regardless of shape, so instead of a tray, the end of the robot’s arm would be a morphing machine, inspired perhaps by the physiology of an octopus (see this earlier Jaron’s World). The “octopus” would take on whatever form your hand is trying to touch or grab, and it would be attached to the floor so you could rest on it and not suffer from arm fatigue.
Can it be done? Based on the faithful companionship of Moore’s law, I predict the octopus butler robot will appear in a lab by 2015 and in the home around 2025.
