Technology / Computers

Information is not knowledge. Those four words contain one of the most vexing paradoxes of the 21st century: We are accumulating data at exponential rates, but our understanding of what is being stored is creeping along. Yet somewhere in the stupefying volume of unanalyzed facts may lurk the next epochal discovery. In fact, one emerging frontier of discovery is called data exploration, for which researchers are developing techniques combining mathematics, information science, and software design to probe and extract patterns from vast collections of information.

One of the strangest and most promising methods seeks to uncover concealed patterns in large chunks of data by using topology, the mathematical study of shapes. Although researchers have long plotted their data onto graphs to identify significant connections between different variables, and 3-D bar charts are a dismally familiar feature of PowerPoint presentations everywhere, some patterns do not reveal themselves in two or three dimensions and become apparent only in higher dimensions—which ordinary mortals find impossible to visualize. (Just try to imagine the tesseract, or four-dimensional hypercube, which connects 16 corners instead of an ordinary cube’s eight.)

Topology provides mathematical ways of describing shapes in multiple dimensions. At Stanford University, a number of researchers, including Gunnar Carlsson, Vin de Silva, and Tigran Ishkanov, are investigating ways to use high-dimensional shape detection to determine whether a data set contains hidden patterns.

“Algebraic topology has long existed on its own as a part of pure mathematics,” says Carlsson, a mathematician. “But about 15 years ago, I thought to myself: What is it about topology that’s attractive to people and might be useful?” He was particularly excited by a classic analysis of diabetes research in the 1970s. The data spanned 145 patients with diabetes and included five measurements for each patient—four metabolic and one weight related. Another way to look at it, Carlsson says, “is that each patient was represented in five dimensions.”

The researchers used a traditional statistical technique called projection pursuit to plot the data and depict it in three dimensions. When they did so, it became clear that there were two very distinct types of the disease, as suspected. Each cluster of patient data formed a separate shape in the data plot, and the two did not overlap. In 1979 the National Diabetes Data Group formally recognized the distinction between type 1 and type 2 diabetes.

“What we want to do now is automate that kind of recognition and make it precise—that is, computational,” Carlsson says. “And we want to expand it to high dimensions. This could be important for treating other kinds of diseases or revealing other sorts of important connections in large data sets.”

To test their system, Carlsson and his fellow mathematicians collaborate with neuroscientists who implant electrodes in the visual cortex of macaque monkeys and then monitor how the monkeys respond to different visual stimuli. Those data record which neurons fire at which times. But they don’t answer large questions: How is the brain processing the imagery? Does it process different patterns in different ways? And could you design an artificial system to simulate optical processing in the nervous system?

That’s where Carlsson’s team comes in, applying his multidimensional analysis to patterns in the data array. “We’re beginning to see how to represent that topologically in, say, 100 dimensions. And it turns out that, depending on the collection of visual stimuli the monkey sees, the data form characteristic shapes, such as a torus, which is like a doughnut or a loop.” Eventually, Carlsson says, this sort of analysis may help scientists understand how the brain works. “Neuroscience by itself does not know how complex families of images are encoded. Topological analysis could provide those critical insights.”

Such insights might allow scientists to approach problems that are maddeningly difficult, such as designing computerized systems to make real-world visual sense out of the input from a couple of video cameras. “Suppose you need to build a robot whose job is to clean up in a bar,” Carlsson says. “The robot needs to distinguish between glasses, which go in the dishwasher, and bottles, which go into a recycling bin. But you don’t want to have to use a database of all possible glass and bottle shapes seen from all possible angles. You’d like qualitative cues that tell you the difference between a bottle and a glass.”

Topological analysis may do the job. “We don’t know yet exactly how sensitive the technique is. But the thing it has going for it is that it’s a single methodology that you can apply in a lot of situations.”