Harold Cohen was already an acclaimed artist when he represented the United Kingdom at the Venice Biennale back in 1966, and his work subsequently appeared in top-ranked galleries and museums around the world. So in 1969, when he began dabbling in computers, his intent was simply for the machines to help him create his drawings and paintings. I thought of designing a program as a kind of assistant, he recalls. I was to think up the heavenly paradigm and it was to do the earthly instantiation. But as Cohen found himself devoting less and less time and energy to his own paintings, his computerized alter ego, dubbed Aaron, began to take on a career of its own.
In 1983, Aaron took up a pencil in its robotic hand and tirelessly produced drawing after drawing for an audience of captivated visitors to the Tate Gallery in London. It didn’t matter to them that Cohen had to add color to the drawings with his own hand; many an onlooker walked out with one of the new drawings tucked under his arm. By last year, when the
Computer Museum in Boston devoted an entire exhibit to Cohen’s stepchild, Aaron had mastered paintbrush and palette and, once Cohen set up the apparatus, produced whole paintings, many of them quite pleasant to look at.
Cohen’s success with his computer program raises the question: Who is the creator of these paintings? The answer is by no means clear. Perhaps the creative intelligence is Cohen’s because, after all, Aaron merely does what he programs it to do. On the other hand, Cohen has no way of predicting what Aaron is going to do, and the paintings are produced by the machine’s hand.
Since the early 1980s, dozens of people have tried to plumb the potential of computers to make supplemental contributions to their paintings, writings, and musical compositions, or even to make wholly original works of art. Partly owing to the growth in speed and power of computers, which can be programmed to behave in ever more complex ways, these artists have largely succeeded in endowing machines with what seems to be the gift of creativity--computer programs can now compose original music in the style of Bach, play jazz saxophone like Charlie Parker, and even produce works that arguably bear their own style (or at least one that cannot be directly traced to the programmer-artist). But can a bucket of bolts and silicon chips ever truly be creative?
Philosopher John Searle at the University of California at Berkeley is a leading skeptic on the question of whether computers can ever think like us, and he puts his argument succinctly: Programs are all syntax and no semantics. In other words, a computer cannot ever have a mind, because it merely follows rules that tell it how to shift symbols, without understanding the meaning of those symbols. No matter whether you think of the symbols as ones and zeros, Fortran, or the English language-- they have meaning only to the human mind that programs the computer or uses it, not to the computer itself. Without meaning, creativity is shallow and unimportant. Searle admits that in principle a computer might be made to write like Shakespeare or compose like Beethoven, but no matter how profound the machine’s output, it would still be the result of mere symbol shifting. The brain, he insists, somehow transcends such manipulations.
Assume for a moment that Searle is correct--that computers could conceivably model human creativity, even though, lacking true understanding, they could never produce art that is meaningful in a human sense. Many interesting questions still remain. Just how creative can a computer appear to be? Is a Beethoven program possible in practice as well as in principle? And what about the apparently creative programs that already exist? How good are their artworks? Even if computers are doomed to produce output rather than creative works, perhaps they could nevertheless teach us to develop new ways of drawing or composing. Perhaps they could be made to come up with new ideas, draw surprising analogies, distinguish insight from triviality. And who knows, perhaps we might learn something about human creativity in the process.
In a studio at the University of California at Santa Cruz, a chamber orchestra is rehearsing a new piece. The music is familiar, yet unfamiliar. Some of it sounds as though it might be a previously unknown composition by Palestrina; at other times, it sounds like some kind of collaboration between Mozart and Scott Joplin. The musicians are human enough, but the new composer is a computer program called emi, for Experiments in Musical Intelligence.
David Cope, an accomplished composer and professor of music at the university, turned to computers in the early 1980s because he thought a cleverly written program could help him when he was stalled in the act of composing. I thought I could come up with something that might generate a few patterns that would help me through those periods when I had trouble thinking of what to write next, he says. The project grew from something that Cope could use as a compositional aid to a music-composing program in its own right. At first Cope gave the computer some general rules of musical style and specific musical patterns drawn from various composers, but the results were unsatisfactory--too boring and mechanical. Cope redesigned the program so that it would take a piece of music, figure out for itself what some of the rules of composition are, and compose a new piece using those same rules. The results are often persuasive, and sometimes uncanny. Starting with patterns drawn from several Bach chorales, emi sometimes comes up with what it thinks are new patterns but which are in fact very similar to phrases used by Bach in other chorales.
EMI doesn’t match Cope’s own originality. It can produce snatches of music that are interesting as well as occasionally breathtaking--like the ghostly echoes in its Bach-like chorales. Its compositions are intriguing, especially on first hearing. But the second time around, they begin to sound uninspired. Some of them sound like pastiches of past composers written by a competent music student.
Paul Hodgson’s program makes a more lasting impression. Hodgson is a jazz saxophone player from Brighton, England, who wrote a program called Improvisor. Like emi, Improvisor can mimic different styles, such as those of Charlie Parker and Louis Armstrong, and it can even capture much of the flavor of Bach’s keyboard music. Unlike emi, however, it composes in real time, often quickly. Since it randomly chooses and mixes melodic and rhythmic patterns, each performance is unique. How good is it? If I was new in town and heard someone playing like Improvisor, I’d be happy to join in, says Hodgson.
Of course, if Hodgson were new in town, he wouldn’t expect to find a creative giant of jazz, someone who could transform the genre as did Armstrong and Parker. Such people are rare. Improvisor is certainly not going to discover new genres. What it can and does do, however, is explore the possibilities of a given, well-defined musical style. Hodgson has managed to come up with a set of rules the program can follow to characterize the patterns of notes that appear in the music of a given style, and then imitate it. Improvisor actually analyzes the music of Charlie Parker and then re-creates the kind of fluid bebop melodic lines on alto sax that he might have played. Improvisor is, at bottom, derivative.
This doesn’t mean we should dismiss it as uncreative. After all, most human creativity is somewhat derivative, too. Plenty of professional jazz alto saxophone players have absorbed Charlie Parker’s style by listening to and studying his solos, and they’ve made it their own. They may not play with the inventiveness of the master, but they continue to explore the style he pioneered. Follow the rules--stay within a defined style--and creativity is still possible. Creativity, in this sense, is just novelty that is produced by experimentation within stylistic limits specified by rules. This holds for sculptors, painters, and choreographers as well as jazz musicians.
To define creativity in this way, though, may seem unsatisfying. Where does individuality enter the picture? Even mediocre art, after all, requires more than absorbing somebody else’s style. Can the language of computer science capture the artistic signature that each artist seems to leave in his or her work? The argument that it can proceeds as follows: The process of creating a work of art involves a series of decisions. The artist must decide whether to use a paintbrush or a pencil, to make a portrait or a landscape, to use green or blue, and so on. The rules at play dictate what types of choices these will be, but the method of choosing one from the others is a matter of preference. For instance, the form of the English sonnet prescribes a rhyming pattern, abab, but it doesn’t say which words to rhyme--wish with fish, bad with sad, or book with nook. The poet chooses. If the rules did not allow this choice, we’d have a deterministic system in which creativity was impossible.
A creative computer program also faces choices. The difference between the program and the poet, however, is in how they decide which course of action to take. Whereas the program generally chooses randomly-- by the toss of a coin, so to speak--the poet chooses according to idiosyncratic preferences and judgments rooted in personal experience and artistic vision. This is not to say that computers cannot in principle show individuality, too. Several copies of Aaron could run side by side, each programmed to exhibit a different tendency at choice points. They would all draw in the same genre, but each individual program would draw in a recognizably distinct manner. Human artists provide more subtle signatures, more wide-ranging clues, because human minds are incomparably richer than computer programs.
Even if we accept that EMI, Improvisor, and Aaron are exercising a form of creativity that in some primitive way models that of the artists who’ve written them, we’re still a long way from a complete model of human creativity. Radical creativity, of the type displayed by those creative geniuses who make it into the history books, occurs when the creator doesn’t merely explore a manner of making art but transforms it. In the art community, says Cohen, people talk about ‘style’ in terms of a set of ‘problematics,’ or what is worth attending to. The really powerful artists change these problematics. Then other people follow.
So far, he adds, no computer can question the problematics that are put in a program.
Cohen first became interested in the idea of art as a rule-based system back in the mid-1960s. He had noted that people often see surfaces, landscapes, and objects even in the most abstract paintings and was intrigued by the question of what allowed them to do so. To better understand what constitutes representation in painting, he began to experiment with open rather than closed curves, varieties of symmetry and shading, and so on.
Aaron arose from Cohen’s desire to continue this experiment more systematically. He gave his earliest programs rules that allowed them to make intelligible drawings--the computer was told how it could divide a space, for example, or was told that it could not cross a line that had previously been drawn. Aspects of randomness made each drawing unique, while the rules ensured that the result would remain visually coherent. Cohen later imbued Aaron with the ability to depict things people could readily recognize, and still later with knowledge about the objects it draws--it knows, for instance, that a ball is round and that humans have two arms and two legs. Over the years, Cohen has extended Aaron’s abilities so that it can depict scenes and objects in three dimensions, including, in the last several years, human portraits.
But consider the kind of rules that constitute Aaron. Cohen has told Aaron what an acrobat is and how the acrobat manages to stand on tiptoe on a medicine ball without falling over. He has told Aaron how arms and legs change shape in different body attitudes. And he’s told it that a body part shouldn’t appear in the drawing if, from the spectator’s point of view, there is something in front of it. Consequently, Aaron sometimes draws acrobats with only one arm visible. But it never draws a one-armed acrobat. And, left to itself, it never will. It can’t even consider doing this, because it has no way of varying its fundamental body schemas. For Aaron, all human beings have two arms. It can’t decide in a particular instance that art would be better served by depicting a human being with one arm--or, like an Indian goddess, six.
Aaron is thus a relatively unimaginative program. It could generate one-armed, or six-armed, people if the information that humans have two arms was provided in a way that made the numeral clearly separable (as in Number of arms: 2) and if it also had the rule Every so often, if a schema contains a numeral, substitute another numeral instead. Even this, of course, would cause problems, given Aaron’s realistic style. Aaron knows something about bodily balance for two-armed people. A one-armed acrobat standing on tiptoe might have to hold his or her arm or body differently to avoid falling over. So Cohen would have to alter Aaron’s balance schema, as well as its anatomy schema. (What if there were six arms, all on one side of the body?) For Aaron to be able to come up with the idea to draw a one- or six-armed acrobat, it would have to possess some critical faculty. It would have to evaluate the results of its own experiment and make modifications according to its own aesthetic judgment. What we call creativity is a relative term, says Cohen. Aaron is more creative than any other painting program, but I won’t think of it as being creative in an absolute sense until it can modify its own behavior.
Some computer programs attempt to go around Aaron’s limitation by making variation their prime directive. One of these is Mutator, the brainchild of sculptor William Latham and programmer Stephen Todd, at the Hursley Research Laboratory of ibm in Winchester, England. In the early 1980s, as a student at the Royal College of Art in London, Latham began experimenting with evolutionary art. He was inspired by biology, in which the repetition of very simple steps (cell division) leads to complex and interesting forms (animals and plants). He’d start with a single picture and produce from it half a dozen or so daughter pictures, each incorporating some small change to the original. The changes were not arbitrary but the result of applying very simple rules. For example, if Latham started with a cube, he then might produce one daughter by making a bulge in one side of the cube, another by cutting off a corner, and a third by cutting the cube crosswise. Then, for each daughter, he would apply the same set of rules to produce another generation. Sometimes he used rules that called for combining two parent pictures into one--a cube might be combined with a cone to produce a cone with a cubical collar around it. By repeating such simple operations over many generations, he constructed elaborate family trees whose complicated members at the bottom differed radically from the simple original at the top and came to resemble robots, spiders, tanks, and many unnameable figures. Often they bring to mind slugs and snails and other invertebrate creatures.
In the beginning, Latham produced his drawings by hand, using pencil and paper, putting a generation’s siblings across the width of a large sheet of paper and their descendants down its length, continuing from one sheet onto another. Some works were 30 feet long. Eleven years ago, however, he decided to use a computer to relieve him of the drudgery. He hit on a method whereby he would write a short program to generate a parent image and then let his computer introduce random changes to the program. Using his artist’s eye, Latham chose the best daughter image with which to breed the next generation. So surprising were the results that Latham believes using the computer has allowed him to create images that he otherwise wouldn’t have imagined. He sees the computer as a liberation, freeing his imagination from some of its human limitations. When you first start mutating forms and shapes, things are sort of familiar, he says. Then the forms start doing very strange things, like turning inside out, or suddenly sprouting a thousand little tentacles where you’d least expect them. Soon you’re producing forms with beautiful complexity that the human imagination could not conceive of in one jump. It’s a bit like finding strange insects under a stone when you’re a child--it’s got that fascination.
Karl Sims, a computer scientist at Genetic Arts in Cambridge, Massachusetts, produces evolving images in a way that is very similar to Latham’s in principle but in practice involves a more elaborate method of mutation. In effect, his method mimics the operation of dna. In Sims’s program, each image is generated by its own miniprogram. To produce a daughter image, the miniprogram mates with one or more other miniprograms. Mating, in this sense, involves splicing a few lines from one miniprogram together with a few lines from another. To keep things interesting, Sims also introduces occasional random mutations in the daughter program as well. An entire miniprogram can be added onto another, or nested inside it. After many generations, a miniprogram only a single line long may evolve into a ten-line program made up of several smaller pieces from many different descendants.
Sims’s program produces far more surprises than Latham’s. Its images are abstract patterns--sometimes black-and-white, sometimes with only one or two colors (perhaps with many subtly different shades), sometimes multicolored. The lines may be sparse or plentiful, straight or curved, clear or blurred, and the overall structure symmetrical or asymmetrical. Daughters of even the second generation can differ radically from their parents, and great-granddaughters usually retain no family resemblance whatever.
At first blush, then, although Latham is the one who’s trained in fine art, it seems that Sims has managed to create the more creative artist. This impression, however, is only superficial. Sims’s program, you might say, is all transformation and no judgment. It is dedicated to radical change, not subtle improvement. It does not know how to recognize a promising alteration and develop it to bring out its aesthetic or communicative potential. Art involves disciplined exploration as well as unstructured play. If an artist does play around for a while, he or she will stop doing so when something interesting arises and will explore it more systematically. This is clear from artists’ retrospective exhibitions. In early pieces, we discern the seeds of trends more fully developed later. Only rarely do we see a sudden, fundamental transformation of style, and only very rarely (with Picasso, for instance) more than one.
Latham’s images have a strong family resemblance because he chooses approaches he finds interesting and makes his program respect them. He allows it to make only minor mutations, at superficial levels of the code. A feature (a horn, say, or a coil) may be repeated or skewed, but the fundamental image structure can’t be varied. This isn’t a failing but a deliberate strategy on Latham’s part. Sims, by contrast, isn’t so much concerned with the aesthetics of his pictures as with the whole process of interaction between the artist and his program. His program produces radical transformations because he allows it to change the heart of the image-producing code.
Neither Latham’s nor Sims’s program is able to evaluate what it does in aesthetic terms. In both programs, the selection is done by humans at each generation. The same is true for Aaron, Improvisor, and most other so-called creative computer programs. That is not to say that computer programs cannot make judgments of any sort. Sims has written one that evaluates novelty. The program evolves animated creatures and selects the best from each generation on the basis of how well its anatomy is adapted for fighting. Other programs evolve their criteria of fitness as they go. It wouldn’t be difficult to imagine a program that recognizes patterns in music, pictures, or text that are generally considered attractive by humans, though in practice it might be too ambitious to do so in a satisfying way. Cohen, for one, has been trying for years to come up with a way of endowing Aaron with the ability to evaluate its own paintings, without success. The program is complicated enough to start with, he says, and when you start evaluating, the complexity scales up exponentially. Any kind of experiment is likely to crash the program.
The inability to make these kinds of judgments is a serious limitation because one of the most important steps in producing creative work is to choose between what is good and what is bad, what works and what doesn’t. There’s no accepted definition of creativity, but I think everybody would agree that it’s a combination of novelty and significance, says Douglas Hofstadter, professor of cognitive science and computer science at Indiana University at Bloomington. Novelty is a dime a dozen; anybody can be novel. What matters is to be novel and have some depth.
The fact is, the mental processes involved in producing (and appreciating) art are way beyond any current computer program. In practice, they may be beyond any future program too. While no one has yet articulated precisely what mental processes are involved when we respond with human richness of thought to a humanly rich aesthetic artifact, we can agree that they involve subtle associations and judgments, many of them unconscious. One problem facing psychologists here is that human minds contain a huge number of concepts and experiences, with many billions of potential associations among them. For now, we can’t hope to match this variegated richness in a computer model. However, some scientists have succeeded in breaking off bits of these associative processes and modeling them with computer programs. One program in particular, called Copycat, helps us see at a simple level just what association in the creative process might actually mean and what mechanisms might be able to bring it about.
Copycat was conceived by Hofstadter, author of the book Gödel, Escher, Bach, and his colleague Melanie Mitchell, director of the Adaptive Computation Program at the Santa Fe Institute in New Mexico. Hofstadter himself is an exceptionally creative person, fizzing with intellectual energy. He chose to concentrate on modeling analogical thinking because it is one of the most common wellsprings of creativity, both in the arts and the sciences. Many scientific insights came in the form of powerful analogies--the notion that the heart is basically a pump, for instance-- that we take for granted today. To the artist, analogy can provide a vehicle for breaking the current rules of art while staying within the limits of comprehension. Analogy might allow a painter, for example, to see a still life as a landscape. A poet might use it to link words and ideas (The moon was a ghostly galleon tossed upon cloudy seas) that would not normally have any connection.
Copycat, of course, is not nearly so ambitious. It looks for analogies between alphabetic strings of letters. If Hofstadter tells his program that abc goes to abd, and he then asks, What does pqr go to?, it will answer pqd, or, better yet, pqs--as most people probably would, too. To do this, Copycat must be able to describe the structure of each letter- string in many different ways, and it must somehow decide which of these descriptions is the right or the best one for generating the analogy. For instance, suppose you’re given the letter-string ffmmtt. You’d probably describe this as three pairs of letters. If you’re given klmmnotuv, you’d probably see it as three letter triplets. In each case, the letters mm are perceived differently: as one chunk or as parts of two different chunks. If you were given mm alone, you’d have no reason for seeing it as either. It’s the context that inclines you to describe an input in a certain way and perhaps to abandon an initial description for another.
Letter-strings aren’t the stuff of human delight, or even nightmare. But still, Copycat is a highly complex program. Imagine how much more complicated it would have to be to model more interesting concepts, like suicide or betrayal. Hofstadter’s aim isn’t to mimic the poet’s imagination but to illustrate the fluidity of thinking required for creativity, of which the sensitivity of analogy to context is one part.
Copycat can take a new situation and explore a range of possible analogies--and then it can judge which of these analogies is better than the others. Consider this puzzle: If abc goes to abd, what does xyz go to? Copycat suggests xyd along with several other answers, but it recognizes that wyz is the best. Why? Because the descriptions that generate it are less obvious, and the reasoning deeper. To produce wyz, you must realize that abc and xyz are the beginning and end of the alphabet, that D is the successor of C but that Z has no successors, and that successor and predecessor are opposites. You must also decide on three reversals: to traverse the alphabet backward and not forward, to substitute predecessor for successor, and to flip the sequence back again. By contrast, to get the answer xyd, you simply recognize that Z has no successor and copy the D across.
Copycat gives us, therefore, an example--albeit crude--of how a computer can exercise aesthetic judgment. Hofstadter’s work is closer to human thinking than most programs in artificial intelligence, which are structured too rigidly to recognize any analogies but those its programmers could think of in advance. Evaluating letter-strings, of course, is a long way from judging a painting or a melody, and it is by no means obvious how to get there from here. When all’s said and done, our knowledge of human thought processes is still very sketchy. Creativity is largely a mystery.
And what about the artistic vision that gives coherence and purpose to the great works of art? Could a computer ever be programmed to have one? This may be the biggest mystery of all. For one thing, nobody really knows what role such vision plays in the act of creation. One artist may set out with a clear idea of where he or she is going, with a particular effect in mind, while another allows a vision to emerge only gradually in the act of creating. When I draw, I sit down and I have no idea what I am going to do, no vision whatsoever, says Hofstadter. Then I start drawing, and it becomes: okay, here we go; I don’t know where we’re going with this; I’m going to let it develop as I go along. Perhaps vision can be made to emerge in a sufficiently rich computer model.
Hofstadter believes that capturing the processes that make up creative thinking in a computer program is possible, given that computers could be made big enough and fast enough to rival the vast complexity of the human brain. People model the evolution of galaxies on computers now, and they get startling results they never could have predicted, says Hofstadter. You wouldn’t say that an ant is intelligent, but collectively ants build wonderful bridges. We can’t even imagine how many cells we’re composed of. Likewise, I think the human mind is vastly complicated, but there is nothing fundamental about it that we couldn’t capture in a computer program. It’s a matter of complexity.
If Hofstadter is correct, then eventually we may have to decide whether to accept the aesthetic judgment of computer programs, and whether to recognize them as significant or useful. One might argue that the richness of the program is irrelevant: computer art isn’t art at all, just an empty pastiche. No matter what the medium, the work of a computer could not express suffering, courage, empathy, or wisdom. Perhaps this is so. But even if a computer’s notion of art is irrelevant to us humans, that doesn’t preclude a computer’s broadening our aesthetic horizons.
Aaron is not a human being, says Cohen, and it’s not intended to be. My goal is to allow Aaron to develop its own personality, to see what this particular kind of intelligence can do. What if a computer possessed the vision to develop, say, a new type of music or painting? To reject the innovation solely because it was produced by a computer would be to dismiss a host of intriguing and beautiful artifacts. We’d be the losers.
