Virtual Reality (VR) has moved from the California labs to find itself in the public eye. Various exhibits and 'Cyberthons', numerous articles, television spots, (WGBH's NOVA and National Geographic) have brought this infant technology to the mainstream . VR proves to be a powerful medium of the future. Unlike present media, VR relies and functions best as a creative tool, not passive mind control. -ESLF The Eastern Seaboard Liberation Front (ESLF) presents... -------------------------------------------- "In the future I see virtual reality as a medium where people improvise worlds instead of words, making up dreams to share an objective form of the Jungian dream. You might even call it the collective conscious" Interview with JARON LANIER from Omni, January 1991 \--------------------------+-------------------------/ On the living room wall of Jaron Lanier's disheveled bungalow in Palo Alto, California, hangs a poster of the four-armed Hindu goddess Kali. Her 16 fingers and 4 thumbs dexterously play a sitar. Most Westerners would find the image an exotic one, but in the context of virtual reality, the emerging field of which Lanier is the unquestioned guru, Kali looks as normal as Betty Crocker. Virtual (artificial) reality is the hot new computer technology that lets you do the impossible - from swimming through the heart's aorta to walking the dog on Saturn's rings. Technically virtual reality is a full-color, full-motion 3-D environment manu factured by computer and displayed inside a pair of goggles worn by the virtual traveler. Psychologically it's poised to become an open-ended, no-holds-barred experience that enables people to create their own dreams in Technicolor and then let their frie nds jump in. VPL Research, Inc., of which Lanier is the founder, CEO, and spiritual leader, is the first company to bring virtual reality technology to the market. VPL's customers include NASA, Apple Computer, Pacific Bell, and an assortment of universities and rese arch labs. On a visit to VPL's offices overlooking swank Redwood City sailboat marina, interviewer Doug Stewart explored a sample world that Lanier had spent barely an hour mousing together on a Macintosh II computer. To enter this world, Stewart pulled a pair of VPL's cumbersome, opaque EyePhones and a wired-up Data-Glove made of black Lycra. Sensors on the glove and goggles steadily sent a silent flood of signals to a powerful Silicon Graphics computer sitting on the floor. Stewart instantly found himse lf in a room filled with semifamiliar objects: a red apple on a table, a bunch of purple grapes, a banana spinning lazily end over end in midair, a yellow rubber duck bobbing in a rippling water of a hot tub. Periodically a small pterodactyl swooped down a spiraling chimney and out of the room. "Reach for the grapes," Lanier suggested to Stewart, who groped with his gloved hand and watched as a virtual representation of his bending fingers closed on the bunch. He moved his fist, and then the grapes moved with it. Leaning back in satisfaction, Stewart saw the room abruptly enveloped in red. He's accidentally backed his head into the apple. In minutes Stewart had mastered the "fly!" gesture (pointing a gloved forefinger while curling the thumb under), rocketed up the chimney, and was soaring thr ough the gray billows of distant computer-generated cloud. A self-taught thirty-year-old with neither a college nor high-school diploma, Lanier is, not surprisingly, a onetime video game designer. Virtual reality (VR), however, promises to be much more than the ultimate wraparound adventure game. VR systems wou ld be the perfect command post for sending remote-controlled robots where humans prefer not to go (a melted-down nuclear reactor, the asteroid belt). Medical students could practice surgery on virtual cadavers that spurt virtual blood after a misplaced in cision. Such uses are speculative so far, but few people doubt the technology's potential. No one offers more enthusiastic speculation than Lanier. A champion of virtual reality as a key to unlocking humankind's imagination, Lanier is probably the first man with full-length dreadlocks to be profiled on page one of The Wall Street Journal. Bea rlike, with heavy-lidded blue eyes and a soft, sometimes dreamy voice, Lanier dominates a room. Stewart interviewed him over five hectic days and nights punctuated by midnight drives, unexpected visits by delegations of Japanese industrialists, sudden aft ernoon naps, dinners with computer moguls eager to pick Lanier's brain, and the occasional 5:OO A.M. staff meeting. Lanier, the hacker-turned-capitalist, struck Stewart as more hacker than capitalist. With breathtaking nonchalance, Lanier disregarded scheduled appointments and housekeeping details (he even had his unlisted home phone inexplicably disconnected midweek ) or changing clothes over the course of five days. Most of the interviews took place in the living room of his rented bungalow. Lanier is an accomplished improvisational musician, and the room was crowded with more than 1OO instruments, from bagpipes to xylophones to unidentified horns and gourds. During breaks Lanier repaired to his grand piano and sent fluent, atonal chords crashing through the room. The Pied Piper of a growing technological cult, Lanier has many of the trappings of a young rock star: the nocturnal activity, attention-getting hair, incessant demands on his time. He is casual, giggly, his heedlessness verging at times on arrogance. Y et during the interviews he seemed oddly unspontaneous. When he found something amusing, which was often, he paused a beat before issuing a staccato burst of giggles. Hearing good news on the phone, he screamed in delight but hesitated before whipping an object across the room in celebration. (Hmmm. Should I throw this pen or shouldn't I?) Lanier behaved, in fact, as if he were observing himself from a distance. Omni: What is virtual reality? Lanier: It's an alternate reality filling the same niche otherwise filled by physical reality. It's created when people wear a kind of computerized clothing over the sense organs. If you generate enough stimuli outside one's sense organs to indicate the e xistence of a particular alternate world, then that person's nervous system will kick into gear and treat that stimulated world as real. You might be in a Moorish temple, or a heart that's pumping. You might be watching a representation of hydrogen bonds forming. In each case the world is entirely computer generated. Now, imagine that you had the power to change that would quickly - without limitations. If you suddenly wanted to make the planet three times larger, put a crystal cave in the middle with a g iant goat bladder pulsing inside of that and tiny cities populating the goat bladder's surface, and running between each of the cities were solid gold railways carrying tiny gerbils playing accordions - you could build that world instead of talking about it! Omni: Okay.... How does the computerized clothing work? Lanier: The goggles put a small TV in front of each eye so you see moving images in three dimensions. That's only the beginning. There is one key trick that makes VR work: The goggles have a sensor allowing a computer to tell where your head is facing. Wh at you see is created completely by the computer, which generates a new image every twentieth of a second. When you move you head to the left, the computer uses that information to shift the scene that you see to the right to compensate. This creates the illusion that your head is moving freely in a stationary external space. If you put on a glove and hold your hand in front of your face, you see a computer-generated hand in the virtual world. If you wiggle your fingers, you see its fingers wiggle. The gl ove allows you to reach out and pick up an artificial object, say a ball, and throw it. Your ears are covered with earphones. The computer can process sounds, either synthesized or natural, so that they seem to come from a particular direction. If you see a virtual fly buzzing around, that fly will actually sound as though it's coming from the right direction. We also make a full body suit, a DataSuit, but you can just have a flying head, which isn't really so bad. The hands and head are the business ends of the body - they interact most with the outside world. If you wear just goggles and gloves, you can do most of the stuff you want in the virtual world. Omni: What about touch? Lanier: VPL is working on developing touch sensors. We've done experiments with tactile feedback by putting vibration simulators inside the fingertips. When you fingertips feel vibrations that match what you see in virtual reality, you associate them with the surface of the virtual object. It's surprising how many sensations you can create with vibrations alone. Another way to simulate touch would be with a grid of tiny elements that move back and forth like little pistons so that the overall grid can tak e on shapes. That's tough to build because it would have to be very thin to fit onto the surface of a glove. Touch is a very complex activity. Tac-tile sensation is an action; it's not passive. You're constantly nudging things with your fingers, rubbing, squeezing things, feeling their weight and textures, judging the position of your arm and fingers, performing hundreds of subtle little tests. To synthesize the full sensation of picking up an object in VR, you'd have to do a number of things, all difficu lt, some perhaps impossible. Omni: Who are your customers? Lanier: Most are companies and institutions with their own technical know-how. Some use VR to test designs before building them. Some are trying to understand scientific or engineering data better. Some are people who want to have fun. Omni: Millionaires who want to play three-D games? Lanier: There's only been one example of that so far, which I don't encourage. But there's nothing wrong with a technology that unites work and play. VR allows you to do work that you couldn't do otherwise by making it playful. People in the business worl d are sick of being told that things that aren't fun are fun, like using a spreadsheet. Virtual reality actually is fun. You might think of it as a general-purpose simulator, or as a fantasy machine. But what makes it so special is that you and others wearing VR clothing can be networked together to share the same alternate reality. The content is completely variable - you could be on top of Mount Everest or the bottom of the sea - but the environme nt is the same for everyone in it. You and your VR partners can shake hands, dance together, play ball. You can construct buildings together. Virtual reality is an epistemological milestone, a new reality that's shared as the physical world is. Yet it is open and unhindered like dreams. Omni: What are some applications? Lanier: Each application by itself is a whole-amazing world, so in a way, anytime you talk about a particular application you're somehow losing sight of the overall picture. Omni: Still, don't your customers view it as an efficient tool rather than a mindboggling experience? Lanier: Absolutely. It's extremely efficient. An architect can make a building real before it exists and bring people through it. In a demo with Pacific Bell recently, two architects got together over the phone and explored a proposed day-care center in V R. One showed proposed features to the other, they could see each other moving around in the room and could make design changes. By holding the glove a certain way, they could change their bodies to take on characteristics of children's bodies. So they we re able to run around and test features like a water fountain from a child's perspective. Another example is city planning. Tom Furness is heading a lab at the University of Washington that's studying VR. We're helping them put a version of Seattle in to a VR that you can walk around in. You can add skyscrapers to the skyline to see what they feel like aesthetically, whose views are blocked, and so on. Omni: How can virtual reality advance medical technology? Lanier: We take information about the human body from scanning machines and turn it into objects in virtual reality. This means doctors can put their patients through a scanner, then walk in to virtual reality and pick up the patient's bones and internal organs. Suppose the patient has a serious deformity or injury. A surgeon could get a feeling for the three-D structure of that person's body to help plan surgery. This is still in the earliest testing phases, but we've done one project with the San Diego Supercomputer Center where we had people crawling around inside patients and looking at the structure of their brains. You can have two physicians inside the brain at the same time, and they can talk about what they see. One can point to the structure and say, "There's an abscess here. Omni: What's the smallest world anyone's made a virtual visit to? Lanier: Fred Brook and Henry Fuchs at the University of North Carolina have done some marvelous work letting chemists pick up molecules whose atoms are about first size. You can figure out certain chemical problems quicker by holding on to a sort of robot arm that comes out of the ceiling and pushes back at you to simulate a molecule's forces. So you can literally feel where chemical bonds could occur. A complicated organic molecule is something like a handful of little magnets in a cluster. Their forces combine to form a complicated, irregular field. As you move a new magnet over the big cluster, sometimes it's attracted, sometimes repelled. In a sim ilar way, a molecule has a landscape of atomic forces around it. There might be little patches exactly complementary, so that two molecules will bond at one point. That's easy to study in simple molecules, but it's much harder with large organic molecules like an enzyme. The systems that the North Carolina lab and others are developing are tremendous new tools for seeing and feeling how these molecules behave. Some mathematicians and physicists are using it to make intangible worlds real. We're doing some work with actuaries. They can fly over an abstract forest that represents various insurance statistics. It helps them notice patters in the data more easily than they could on even a very large computer screen. Computer programmers could look at a whole program at once. A large program might look like a giant Christmas tree, and you could be a hummingbird flying around it. Landing on any one branch, you coul d see in great detail the structure of that part of the program. From a distance, you could learn to plan a very large program spatially. Omni: Do people send you suggestions about uses you never dreamed of? Lanier: Tons. Some of them are truly crazy. We've had inquiries about putting animals in virtual reality from people who design animal clothing. Ministers call up to ask if we could use VR as a kind of methadone treatment. And virtual sex - you should see how stupid my mail is on this subject! A lot of the inquiries don't make any sense, but it's important to be open-minded. Omni: The National Enquirer reported that VPL was working on a spy glove. Lanier: Yes! [Laughter] They said we were working with the CIA to make a robot resembling a severed hand that could be remote-controlled by a DataGlove. It would crawl into enemy territory, climb over fences, steal enemy papers, and crawl back. It was sil ly. Omni: Could virtual squash someday replace the real thing? Lanier: Absolutely. Visually a simple squash game in low resolution might be doable right now on an inexpensive system. As for force feedback, you could design a robot that pushes back at your feet in a particular way. Perhaps there'd be a robot arm with a racket handle that comes out of the wall. You'd grab the handle, it would jerk back when the ball was hit. When the simulation is specific like this, you can go all-out and make it good. What's hard is to build a general force-feedback machine. Here are some Rube Goldberg examples: Imagine having tiny rockets over your body with little thrusters that are pushing back and forth at you so that any possible force can be applied to any part of you. Or imagine that there are all these little robots all around you, and like tiny bustlers, whenever you slam your hand down on a virtual table, they run up to receive your blow just before your hand gets there. You can take any form you want. You might pull your nose to make it longer or choose from a drawer of extra snouts or horns. You might point to another person or animal wandering around in the environment and turn up a knob that says BLEND and gradually turn into them or something halfway b etween. At VPL we've often played with becoming different creatures - lobsters, gazelles, winged angels. Taking on a different body in virtual reality is more profound than merely putting on a costume, because you're actually changing your body's dynamics. What surprised us is that people adapt almost instantly to manipulating radically different body images. They pick up virtual objects just as easily with a human one. You'd think your brain is hardwired to know your arm, and that if suddenly it grew thr ee feet, your brain wouldn't be able to control it, but that doesn't appear to be true. I become curious about how far I could push this, and added fingers to my hand and limbs to my body. But how do you control this extra limb? Wiggle your nose? Let's say you want a third, virtual arm in the middle of your chest. The most obvious way to c ontrol it is to make its position an average of your two physical arms, so its thumb is always halfway between your physical thumbs and so forth. Now, that's moderately interesting, but basically the new arm is just something that gets in your way. Imagin e a more complicated way of controlling it: a bodysuit that's constantly making dozens of measurements of different parts of your body - a little bit of ankle, writs, neck - all convoluted by the computer in a funny algorythm to control how far the elbow in a new virtual arm is bent at any moment. You've essentially snuck in control of a new limb while letting each individual part of your physical body move freely. It's like a hidden resource. Omni: You'd consciously learn to control the new limb? Lanier: It's too complex to do consciously. You'd learn to control it intuitively, by getting feedback. This suggests that you might help people who are paralyzed have the experience of walking in virtual reality. Sensors placed on uninjured parts of thei r bodies could let them control a complete body in virtual reality, allowing paralyzed kids to play sports with other kids. Would this activity keep parts of the brain awake that might otherwise atrophy through lack of use? This is completely unknown righ t now. I haven't studied it as a scientist; I've only hacked it as a technologist. The field is crying out for more study of phenomena like this, which VPL is not set up to do. We were thinking of selling a booklet, "1OO Dissertations for 5O Cents." Omni: What about vacations, say, in a virtual Maui? Lanier: The existence of a virtual Maui will just make the physical Maui that much more precious and desirable. I don't think virtual reality will ever serve as a substitute for the physical world. It's not as good. A virtual Maui could never be a full si mulation. By putting it into a computer, you remove its mystery; it's blander and clunkier. You turn it into a finite model. Omni: Still, you talk about the awesome illusions possible. Lanier: The emotional character of virtual reality is completely different from that of the physical world. VR is a craft you create. People say, "I want to try virtual reality because I want the thrill of having these experiences wash over me," but in fa ct the experience is the opposite of that. It's very intentional. A better name for it, actually, might be intentional reality. The physical world is thrilling because it's infinitely subtle: There's always more to perceive. It surrounds us with a sea of mystery. Those of us in science and technology tend to live under the delusion that we mostly understand the world, that there are just little patches that are mysterious. But in fact, we've just constructed around us a small set of things that we underst and. Also, particular environments in VR will never be terribly exciting because they're so readily available. That ornate silver drum over there is an unusual object, which gives it a certain preciousness. If we were in virtual reality and you saw one of those, it wouldn't mean a damned thing, because you could make a hundred of them as easily as one. So particular forms become mundane. What's exciting are the frontiers of imagination, the waves of creativity as people make up new things. Omni: How is it possible to build a virtual world? Lanier: There's no one answer; anything's possible. We're working on technology that will grab a part of the physical world - an architect's rendering or brain scan - and translated that into the virtual world. Ultimately, though, the most efficient way w ill be to use virtual tools you find on the inside. For instance, if you saw a big block of stone in a virtual reality, you might also see a chisel that you could pick up with your virtual hand and start carving with. The difference is that virtual tools will have super powers. You might make an eyedropper that could touch an object and squeeze it in, then squeeze it out somewhere else to make copies of the original. You might have another tool that stretched anything it came across and made it long. Some tools could be very expressive. This room is filled with musical instruments because I find them to be the most eloquent tools ever made. I want to make tools for VR that are like musical instruments. You could pick them up and gracefully "play" re ality. You might "blow" a distant mountain range with an imaginary saxophone. You'd be using gestures instead of building something stone by stone. When you can improvise while inside it, making it up as fast as you think and feel, you can reach other people. As a babies, each of us has an astonishing liquid infinity of imagination on the inside, that butts up against the stark reality of the physi cal world, which resists us. That the baby's imagination cannot be realized is a fundamental indignity that we only learn to live with when we decide to call ourselves adults. With virtual reality you have a world with many of the qualities of the physica l world, but it doesn't resist us. It releases us from the taboo against infinite possibilities. That's the reason virtual reality electrifies people so much. In the future I see it as a medium of communications where people improvise worlds instead of words, making up dreams to share. An ideal VR conversation would have the continuity, spontaneity, expressiveness of a jazz jam but the literal content that's missing from music. Things being made would be objects - houses, chemical processes, or whatever the conversation is about. It would be a reality conversation, an objective form of the Jungian dream, the collective unconscious. You might call it the colle ctive conscious. Omni: What about virtual sex? Is it a possibility or not? Lanier: Oh, God [glumly].... I suppose virtual reality can contain any kind of imagery, so why not sexual? But the whole subject of virtual sex forces the question. What is sexy? What is intimacy? There are some interesting ways to have intimacy in virtua l reality. Consider trading eyes. You'd hook up your virtual eyes to look out of another's head and vice versa, so that you control each other's point of view. It's hard at first, and you really have to learn to dance together at a very intimate level to make it work. If ind that more interesting than the idea of virtual sex, which seems a little funny to me. Omni: When it reaches the mass market, might people settle for touring mass-market worlds? Lanier: Once you get a taste for making up your own reality, you don't go in for passive realities anymore. I'm not saying that everyone will make up stuff all the time; there will be catalogs of old stuff. But I'll bet you a pizza in thirty years that pe ople turn out to be creative. We're living in one of the strangest periods that has been or will be. In the twentieth century our society has been completely warped by technology, but the technology is still astonishingly primitive. Considering that kids grow up with TV, a one-way medium, there's a tendency toward noninteractivity. This is the first century when technology has been the primary mode of people reaching each other. After this, it will be interactive technology. Omni: Who first made VR work? Lanier: A lot of people. Ivan Sutherland [a computer graphics pioneer] built head-mounted display with interactive graphics back in 1968. In the Seventies, Tom Furness, who was working for the U. S. Air Force, made enormous strides in the technology. My r ole has focused on turning this into a shared medium. That had never been done before. Until then, there'd be one person inside a simulation just looking around in it. I also figured for the first time. Tom Zimmerman, who was VPL's first hardware engineer in the early Eighties, built the first glove, and I integrated that into a way of picking up imaginary objects in space. Tom's original idea was to use the glove to play air guitar. You could play music with a guitar that doesn't exist. I've made a few e laborate versions of this glove including one last year where I played Jimi Hendrix solos. Omni: A Wall Street Journal headline said virtual reality was "electronic LSD." Lanier: That's stupid. The idea of spacing out in virtual reality is absurd. It would be like getting a model train in order to fall asleep over it. VR is a medium; it affects the world outside your sense organs and that's all. It has nothing to do with brain chemistry or your state of being. If one becomes euphoric in virtual reality, it would be because you were reacting to the outside world that way. The first moment of freedom is always ecstatic, but after that you're on your own. Actually I'm unqual ified to talk about the subject because I've never taken LSD. I don't take drugs and I don't drink alcohol. Omni: Why do you wear dreadlocks? Lanier: I think of myself as a student who experiments with different things at different times. I had much more conventional hair two years ago, and I'm sure I will again in two years. I had no intention of becoming a well-known person this year. One nic e thing about my hair is that if I ever want to get out of the hassles of being well-known, all I have to do is cut it off. I can always save it as a wig and put it back on when I give talks. Omni: You're a high-school dropout? Lanier: Escapee is more like it. I left it ar fifteen and then kind of snuck into college - New Mexico State and other places. I was never much on the rituals of By rituals I mean turning in papers and finishing degrees. Computer science is a splendid fi eld and most of the founders are still alive. I learned by apprenticing myself to some of them. Marvin Minsky was extremely important to me, and I used to just hang out with him. He is rethinking the whole world from the bottom up all the time. That's an inspiring quality. Omni: Did you invent make-believe worlds as a child? Lanier: What I remember most about my childhood isn't so much inventing make-believe worlds as being overwhelmed by the experience of different physical places. My sensitivity to the mood of a particular room was sometimes so intense I could hardly talk. I didn't know how to communicate that feeling to others. I love words - I love to read, write, talk - but I think words leave out almost everything. That frustration more than anything else - feeling that what we can share with other people is so much mor e limited than what we actually experience - is what has driven me into this technology. Sometimes I think we've uncovered a new planet, but one that we're inventing instead of discovering. We're just starting to sight the shore of one of its continents. Virtual reality is an adventure worth centuries. \--------------------------+-------------------------/ O1/14/91 3:32 pm WRD: 4845 CHAR: 23816 thanks to K.I.M. -Shock LSD for the ESLF