91-07/Bob.global.initiative From: cyberoid@milton.u.washington.edu (Robert Jacobson) Subject: TOWARD A GLOBAL INITIATIVE: Presentation at Nikkei Symposium. (LONG) Date: Mon, 15 Jul 1991 18:02:37 GMT Organization: Human Interface Technology Lab, Univ. of Wash., Seattle ) Human Interface Technology Laboratory 1991 BRINGING VIRTUAL WORLDS TO THE REAL WORLD: TOWARD A GLOBAL INITIATIVE Dr. Robert Jacobson Human Interface Technology Laboratory Washington Technology Center, FJ-15 c/o University of Washington Seattle, WA 98195 USA (206) 543-5075 (206) 543-5380 fax cyberoid@milton.u.washington.edu Introduction and Overview Virtual worlds technology promises to greatly expand both the numbers of persons who use computers and the ways in which they use them. Already, the demand for applications of this technology far exceeds the capability of the technology to satisfy these needs. Independent development of virtual worlds technology has been the norm for at least three decades, with researchers and developers working privately to build unique virtual-worlds systems. The result has been redundancy and a slow pace of improvement in the basic technology and its applications. This paper proposes a "global initiative" to coordinate and to some extent unify R&D activities around the world, the quicker to satisfy an eager market (that may not, however, stay eager for long) and meet genuine human needs. About Virtual Worlds The virtuality paradigm The virtuality paradigm redefines the human-computer inter- face. In brief, it states that what we perceive, for us, _is_ the world; and how we act upon it _is_ how we live. There can be a closer relationship between the images of the world we build in our minds and bodies QJthe subjective models we build to represent an "objective" world outside -- and the worlds we create within the computer. Enhancing a person's perceptions, understandings, and actions should be the reason for which all computer systems are built. Putting the person at the center of the computer system is not as easy a task as it may sound. People are difficult to "design with," so it is much more appealing to engineers to design for them. What we get is systems that are the engineers' interpretation of what is correct for people. Particularly in the fields of machine interfaces and software applications, the engineers need to be joined by people with other orientations, like quality industrial designers, environ- mental psychologists and planners, and even artists. If one accepts the virtuality paradigm, then this collaboration is essential. In a virtual worlds system, the participants interact in a natural way with digital objects within the computer-generated environment. Currently, to generate such an environment, or virtual world, a model is created. Objects and their relationships are incorporated in this model, which runs on a single powerful computer or on a group of networked computers, with each contributing its slice of the virtual pie. Rendering engines display the models as visual and aural information that mimics physical objects or designed abstrac- tions, in three dimensions. These objects and/or abstractions are presented to the participant via an LED- or LCD-based headset mounted stereoscopic visual device and headphones. A position- sensing device on the headset instructs the computer where the participant is and where he or she is looking. The participant also wears a special glove glove or uses a Spaceball or similar tool to maneuver through the world and manipulate objects within it. The computer or computers maintaining the world respond to the signals from the sensors and tools and adjust the world accordingly. The net effect is a circumambience of information that is readily accessible and susceptible to modification. The virtual world can assume the appearance of the physical world, just as the objects within it can take on the characteristics of objects in the physical world The virtual world can be as simple as the line imagery pioneered by Warren Robinett at the NASA Ames Research Center in the 1980s or as complex as the molecular- modeling space now being refined at the University of North Carolina. A property common to all virtual worlds, however, is their _spatiality._ Information within these worlds is presented in three (or more) dimensions. The participant, able to work in three or more dimensions, experiences a wondrous synergy among mind and senses. Nearly everyone who enters a virtual world, while lamenting the low resolution of today's visual presentations, experiences an "Ah-hah!" when his or her spatial sense cuts in. When three-dimen- sional sound (already well-developed) is added, the virtual world attains a verisimilitude that, if not equal to the physical world, is much more familiar than the usual computer interface. We can see and hear things In the future, when the tactile interface is com- plete, we may touch them, too. In only a few years (so we like to think) the componentry of virtual interfaces will present credible images that are of high resolution for all the senses. A critical history of virtual worlds invention The job of recounting the specific histories of various virtual worlds developments has already been done, in Japan by Katsura Hattori's _What's Virtual Reality?_ and in the U.S. by Howard Rheingold's _Virtual Reality,_ Myron Krueger's _Artificial Reality II,_ and other books to be published this year. I want to emphasize the individualistic character of the invention that has taken place so far, which may explain why our technology is less than we would have it be. Clearly, many inventors were inspired by science fiction stories, in which people traveled through space and time, either physically "teleporting" their bodies or sending their thoughts around via telepathy. Vannevar Bush was perhaps the first modern computer scientist to conceive of knowledge as a medium through which one might travel by machine. His "Memex" was a fantasy computer that would put all knowledge at the disposal of its user in multimedia form, Now Vannevar Bush's dream of universal access to knowledge has become international. In the U.S., in the 1960s, Ivan Sutherland started experiment- ing with stereoscopic images created by a computer, to build a "data field." Around the same time, Myron Krueger, another American, was using video cameras and other techniques to reverse the flow, put- ting the user "into" the computer terminal and merging him or her with the images on the screen. And, in the next decade, Thomas Furness began directing the "Super Cockpit" project for the U.S. Air Force, a completely pilot-attuned ensemble (complete with virtual world projectors) that the pilot would wear. While these eminences knew of each other, their work did not coalesce but continued in dis- tinctly different directions: Sutherland, toward flat-screen simula- tors; Krueger, deeper into art and media environments; and Furness, toward continuing to refine data-presentation and manipulation. The popularization of virtual worlds occurred with the simpli- fication of a stereoscopic, head-mounted data display with position- sensors by NASA; and the subsequent commercialization of a similar display complete with world-design software, the "Eyephones" and "Body Electric/Swivel 3D" by VPL Research, in Redwood City, Cali- fornia. With the appearance of these systems, and later the 3D sound unit, the Convolvatron, marketed by Crystal River Engineering, virtual worlds took off in the press and popular imagination. Auto- desk, of Sausalito, California, announced its work on Cyberspace, a 3D CAD program. Cyberspace begat Sense8, also of Sausalito, a small firm working on a Sun workstation platform rather than the traditional, more powerful Silicon Graphics computers that had been the tradition until then. For awhile it looked as if the Mattel Power- glove, a derivative of VPL's Dataglove developed by A.G.E., in New York, might take virtual worlds into the exploding Nintendo game market, but this did not occur. However, although work in virtual worlds was going on in many places by the 1990s, almost all of this activity was completely independent and uncoordinated. Developments and inventions would usually become known within the virtual worlds community only after their introduction at one of the computer professional conven- tions or trade shows, and even so inventors liked to hold onto secret code or hardware tricks to keep their pint-sized corporate empires intact. This tradition of individuality and secrecy is only slowly being eroded by professional and social communications. Unfortu- nately, it is no longer the legacy of small firms; today, many larger firms practice the same self-serving tactics, to the disadvantage of our field. The emerging virtual worlds industry Today, virtual worlds research is taking place around the world. Here, in Japan, research is taking place in many university and MITI laboratories, as well as in private firms. Fujitsu, Matsu- shita Electronics, and ATR are have interesting projects underway, though none is yet what we might call commercial. I am less an expert in Japanese science than you are, but I understand that the University of Tokyo, the Technical University of Tokyo, and Tsukuba University are main centers of academic research in the field. Japanese researchers are making deliberate gains, especially in the fields of telerobotics and tactile worlds. Nevertheless, while the level of local activity is broad, it is not always as well funded as it might be. More than anywhere else, too, institutional boundaries perhaps serve to isolate researchers from each other. This sympo- sium may signal a welcome end to these boundaries. The Japanese potential for cooperative, creative work has been demonstrated in other fields and may be here, too. In North America, larger firms are joining in the research effort. Twelve companies, including Digital Equipment, Boeing, Sun, Microsoft, Alias Research (the leading Canadian firm), and U S West (a local telephone company) have joined in the Virtual Worlds Consortium, which supports the virtual-worlds industrial R&D conducted by Seattle's HIT Lab (the Human Interface Technology Laboratory), where I work. Also, firms are creating their own research laboratories: Boeing, Digital Equipment, Sun, Alias Research, and Cray are among them. Also, many more universities, including the University of Central Florida, the University of Alberta, Syracuse University, the University of Virginia, and MIT have joined the University of North Carolina and the HIT Lab, at the University of Washington, as North American centers of research activity. The tiny firms of TiNi Alloy, in Oakland, California, and EXOS, in Cambridge, Massachusetts, have even begun to explore the tactile environment -- how things "feel" in a virtual world. The pattern of individual, private enterprise holds, however, for large firms and small. With the possible exception of the Virtual Worlds Consortium, one might justifiably say that the North American industry is highly fragmented. Europe expresses an interesting paradox. On the one hand, nowhere is press attention to virtual worlds more extreme. Artists gather to ponder the value of this technology to their work, and intellectuals pontificate on the philosophical meaning of cyberspace. This in itself is not unusual; it occurs in Japan and North America, too. What is surprising is that, where this popular attention is greatest, the work on virtual worlds is least advanced. It is in the European "hinterlands" where the exciting work is being done. In Britain, certainly off the Continental intellectual circuit, W Indus- tries is successfully pioneering virtual-worlds entertainment. And the Advanced Robotics Laboratory is building unusual tactile devices for future worlds. In Sweden, likewise off the beaten path, SICS, the Swedish Institute for Computer Science; the Royal Institute of Technology, in Stockholm; and Linkping University are collaborating in the MultiG, or "Multi-Gigabit" project, to make televirtuality a reality. Perhaps the only really substantial Continental research is taking place in three slightly peripheral sites: 3D acoustics at the Ruhr University, in Bochum, Germany; general experimentation at the CyberLab, at the University of Milan, Italy; and the excellent percep- tual research taking place at the Technical Universities of Delft, Eindhoven, and Utrecht, in the Netherlands. Once again, as in Japan and North America, the European work is scattered and not well- related. Toward A "Global Initiative" The problem: ad hoc, uncoordinated development. Lack of coordination and communication in our field results in several deleterious effects. Among these are: % Duplicative projects make unnecessary demands on scarce labor and produce few advances (or, as we say in English, there is too much "reinventing of the wheel"). % An overemphasis on highly visible research prob- lems (like a better visual display) leads to the neglect of less popular problems (like producing usable earprints for 3D sound). % Interdisciplinary isolation (between fields, even within the same organizations) denies the best-informed solutions. % Rumors, suspicion, and premature competition sap the emotional energy of inventors and developers and hinder the development of the field. % Commercial users' needs and general human needs go unmet, and potentially rich markets lie fallow. When we have time, we can talk about these observations at greater length. Although some of my colleagues may disagree about the degree to which these conditions prevail, no one will deny that, for now, they characterize many aspects of our infant virtual worlds industry. The solution: a global initiative. Perhaps the crisis is not yet felt acutely by all researchers and developers, but at some point it will become clear to most that premature competition and lack of cooperation is severely damaging to our future interests. In the few short years since 1989's first Virtual Reality Day, it is not uncommon these days to come across people who are despondent about virtual worlds technology. They relegate it to the same status as AI, or artificial intelligence, the favorite scapegoat of the short-sighted. These individuals have had their expectations raised by sales pitches and the press -- and very often, by their own imaginations -- only to discover that the technology cannot do what they hoped it would. If this is not going to become a universal experience, we need to take steps to bring our technology up to snuff. We have perhaps three to five years of public and, more importantly, commercial tolerance of our need to experiment with trial and error. After that, if we have not produced virtual-worlds systems with applications for the real world, our credibility will be seriously impaired. We must come to a common understanding on this point: we need to intensify and accelerate our research efforts. We can build upon this understanding to create an international, interdisciplinary effort -- what I call the "Global Initiative." The Global Initiative's primary goal would be to hasten technological development in our field by (1) opening better channels of communication; (2) ensuring that rewards are equitably disbursed to researchers and developers who have earned them; and (3) developing to the fullest the many markets for virtual worlds technology, thus ensuring more than enough work for everyone in the industry. By the estimate of one telecommunications company's analyst, the annual market for virtual worlds-based technology and services, in the U.S. alone, is over $1 trillion. It doesn't matter if this figure is off by a factor of one hundred or one thousand, it is still enormous; and the same is true of every other advanced industrial market. We must recognize that there are more applications for this technology than there are products to satisfy these demands. And we must meet them or watch our potential customers go elsewhere, to other technologies like HDTV or computer-automated operation that seem to offer an alterative to virtual worlds. We know that these other technologies cannot do the things our systems can do, but others may not, and it is their confidence we must get and keep. I propose the following steps to produce cross-disciplinary, interlaboratory, and international communications and cooperation, which will lead us to our ultimate and common goal: 1. Establish for ourselves an identity as a distinct community of researchers whose work is uniquely our own yet of benefit to many communities outside our own. The publication of the new journal, _Presence,_ by the MIT Press sometime this year will herald our academic credentials; but we still need a way of unifying the work of the academy and the commercial laboratories. 2. Initiate conferences bearing on our field. These days we are the honored guests of the giant internation- al computer conferences, SIGCHI (human-computer interaction) and SIGGRAPH (computer graphics), and their national equivalents. Tomorrow we may be out on the street. In any case, participation in these conferences ends up diluting our attention and energy. We do not have enough time to talk among ourselves about what is impor- tant, and we do not hear from others outside the computer field. At our own conferences, we could share vital information efficiently and also hear from others, outside the computer industry, whose work is important to our own: psychologists, architects, linguists, designers, artists, and various end users. It is time, I believe, to seriously consider consolidating conferences like this week's, sponsored by Nikkei; the HIT Lab's Industry Symposium on Virtual Worlds Technology; the recent meeting at SRI International; and others into one common annual meeting or a regular series of moderately sized get-togethers. We need to stop being exotic guests and become homeowners and hosts. 3. Expand the reach of the USENET and other forums for electronic information sharing. I mentioned earlier the emergence of _Presence,_ the MIT Journal. Even before the first issue of _Presence_ hits the stands, however, there will have been over a year of professional dialogue about our field, all conducted online, by computer network. This is the _sci.virtual-worlds_ newsgroup on the USENET, the global public-service computer conferencing system. Nearly every university and many companies in North America, Western Europe, Australia, and parts of Asia (mostly Japan and Tokyo) are tied to the USENET. Consequently, when the HIT Lab began hosting _sci.virtual- worlds_ in early 1990, even with our peers being so few and scattered, we immediately signed up 500 users. Now, with the addition of Japan and Eastern Europe to the USENET, our numbers have tripled and quadrupled. The 1,500-2,000 people now using _sci.virtual-worlds_ have generated over three megabytes of stored text, which we archive on the University of Washington computer. The quality of this newsgroup and the exchanges of information it promotes suggests that we can make greater use of electronic networks to our common benefit. 4. Start collaborative projects bilaterally and under the aegis of national and international organizations. I do not know how this can be done, only that it should be done. There exist mechanisms that provide favorable conditions for colla- borative research, both nationally and internationally, and they are well used in other fields. Why not our own? At this early stage in our science and industry, we stand to gain the most from building good relationships and sharing knowledge. 5. Announce applications/products with common standards and production characteristics. I saved this for last because it is the most controversial of my proposals. Standards are commonly taken to mean an end to com- petition, as it is often the case that established industries use standards as an obstacle to new competitors with better ideas. This may well be the case in some fields, but in ours, it is the _lack_ of standards is what is proving a hindrance. It is the _lack_ of consis- tency, even down to the way componentry is wired or written about in manuals, that is proving to be our constant nemesis. Perhaps in our field we can do things differently and begin to talk about pre- liminary standards now, before any one entity has a vested interest in them. When, over the next few years, as the systems we are toying with become sufficiently stable, we should consider agreeing on certain standards or specifications to carry us into the stage of commercialization. Then, when the technology is more advanced and its applications more certain, we can revisit the issue of standards and decide to keep what we have or go off in new directions. I will not get deeply into the organizational means for accom- plishing these ends; they are easily enough thought of. Perhaps the most important thing is to agree to start working on setting up the channels of communication necessary to building our common future. Conclusion I am optimistic about the future of our field. I believe that we will make both technological and organizational progress in the next five years that will astonish our critics and please our advocates. Clearly, the move toward an international community, which I advo- cate in this paper, is already underway (although not yet firmly established). Let me make a final plea, however, and that is to our intellec- tual patrons and financial supporters: _The common aspirations among the researchers and developers in our field, which can lead to much good for humanity, must be protected from those without vision._ The temptation to use virtual worlds technology for pur- poses that are trivial, base, or dangerous may prove overwhelming for those who seek only material gain. We who labor in the field are generally without great resources, either within our organizations or without, and we are susceptible to the influence money wields. For too many in our field, daily life is simply a matter of trying to survive. That is not a healthy environment for a progressive indus- try. If you want us to deliver on the promises we have made -- to enhance education, improve health care, make design and manufac- turing more successful, and convert deadening entertainment into an edifying experience -- then provide us with the support we need to stay focused on what is important rather than what is expedient. As we say in America, help us to "do well by doing good." It is a positive irony that the virtual worlds we create may become a medium for greater international understanding and pro- gress in the real world. That has been my experience so far. I am proud to be a member of our community. Please permit me to close with a _koan_ for our times: A Master and student were walking down a dusty, virtual road, in search of Enlightenment. Eventually they came to a fork in the road, for which there were no signposts. The student turned to the teacher and asked, "Master, which way shall we proceed, to find Enlightenment?" To which the Master, in reply, merely smiled, reached down, and pulled the plug. --