up
previous
down
search
General Information Table of Contents Springer Verlag News

J.Mol.Model. 1997, 3, 386 - 402

Review

Virtual Reality in Chemistry

Wolf-D. Ihlenfeldt

Computer-Chemie-Centrum, University of Erlangen-Nürnberg, Nägelsbachstraße 25, D-91052 Erlangen, Germany
wdi@torvs.ccc.uni-erlangen.de

Received: 26 August 1997 / Accepted: 28 August 1997 / Published: 2 September 1997


Abstract

With the advent of ever more powerful computer graphics hardware and visualization packages, new graphical methods of scientific visualization and data exploration are beginning to be explored. This includes fully immersive environments where the chemist is surrounded by data objects in 3D space. New models of animation and interactive manipulation of graphical entities are developed to help the chemist in gaining insight from or navigating in large amounts of data. This review discusses some representative approaches and systems which demonstrate where chemistry-related visualization and data management is headed.

Keywords: Virtual Reality, Visualization, Computer Graphics, Molecular Modeling, Databases, Data Mining, Document Navigation


Introduction

There is one expression in scientific and commercial visualization that is heard more often than anything else: Virtual Reality. The sound of threse words evokes notions of cyberspace and visual landscapes unseen by man, promising the solution of all visualization problems by the fully immersive manipulation of information objects giving insights impossible to achieve by any other means. However, probably no other words from this field have also been so heavily abused. If one follows advertisements and also quite a number published papers, about every display program which employs some 3D rendering is now adorned with the magic words Virtual Reality (VR). This includes applications which conceptionally do not go beyond standard 3D graphics, which have been routinely used in molecular modeling and similar fields for more than two decades. So if virtual reality is supposed to be more than simple molecular graphics, what is it?

Strictly speaking, it is computer graphics which replaces the natural environment seen by the user (immersive virtual reality) or at least adds graphical information to the perceived natural environment which is updated according to the movements and position of the user (augmented reality). The possibility to interactively manipulate the graphical objects in 3D space is an important component for many systems of both types. One current definition is: Virtual reality may best be defined as the wide-field presentation of computer-generated, multi-sensory information which tracks a user in real time". Another says: Virtual Reality is the use of computers and human-computer interfaces to create the effect of a three-dimensional world containing interactive objects with a strong sense of three-dimensional presence". [1] A plain 3D screen display with some rotating molecule fits neither category, even if viewed through stereo glasses.

Applications of true virtual reality character have been implemented and practically tested with chemical problems. However, these applications are leading edge research and relatively scarce. A review of these few systems would not fill many pages. So in order to broaden the perspective of this review, we decided to relax the rigid definition of Virtual Reality, and include a wider range of innovative visualization approaches which employ ingenious 3D metaphors and innovative ways to generate and manipulate graphical objects. These applications earn their special character and appeal because they help to generate insight and knowledge by the manipulation of graphical objects, even if a normal color monitor is used as display device instead of a full VR setup. With this broadened definition of Virtual Reality, an interesting and inspiring set of visualization methods for a wide range of application fields is covered. We hope the following overview and selection of examples will give a good impression where VR in chemistry is headed at the dawn of the next millennium.

VRML is not Virtual Reality

Most of the time the term Virtual Reality pops up, it is in the context of VRML, the Virtual Reality Modeling Language. However, while VRML can be a tool for VR, there is no direct relationship between VR and VRML, and certainly not every VRML visualization qualifies for the VR attribute, even in loose definitions. Nevertheless, VRML is such an important topic that a few comments about this technology and its chemical applications are in place.

VRML [2] is a portable, platform-independent and flexible file format for the transport of 3D graphical information, not more. There are two main variants of VRML: The old VRML 1.0, which could only be used to transport static scenes, and VRML 2.0 [3], which adds a whole world of animation and multimedia to VRML. Strictly speaking, VRML 1.0 and VRML 2.0 are not compatible, but all VRML 1.0 features can be expressed in VRML 2.0 syntax, and most VRML 2.0 browsers still support VRML 1.0. VRML is now developed by an international consortium [4] and many of the major computer manufacturers and software houses have become members.

VRML scenes are built in a tree-like fashion from basic building blocks, which specify lighting, camera (viewer) positions, geometrical primitives such as spheres, lines and surface patches, materials and textures mapped on the surface of geometric objects, and geometric transformation nodes. The scene graph is traversed from the root. VRML scenes are viewed either by stand-alone programs, which can be used as helper applications for Web browsers, or increasingly by plugins [5], which make it possible to render VRML scenes in windows which are embedded in HTML pages. VRML viewers and plugins are available for all major computer platforms. This can be attributed to the fact that VRML is not specific to chemical or scientific visualization but is also recognized as a valuable medium for on-line commerce and games. Browsers support different scene navigation modes which are suitable for different application areas. For chemical visualization, simple external rotation and zooming of the full scene is normally more useful than walk-through modes with gravity and collision detection, which are advantageous for architectural models etc.

Except for its platform- and implementation-independent nature and the associated ease of transport of 3D information, VRML 1.0 does not go significantly beyond other graphics exchange formats. Actually, it was derived from the ASCII file format of the OpenInventor [6] graphics class library maintained by SGI and Template Graphics. Nevertheless, VRML 1.0 was soon discovered as a suitable tool for Internet-based chemical visualization. These first attempts at VRML visualization were limited by the capabilities of VRML 1.0. The first scenes on the WWW [7][8] [9] were static and pre-computed. It is possible to use clickable 3D buttons to download another file with changed display attributes to introduce a minimum of dynamics, but this is an awkward solution. VRML 1.0 has been employed as 3D output format for serveral Internet-accessible databases [10], and there is even an CGI-driven on-line 3D coordinate generator which directly produces VRML representations from 2D connectivity input. [11]

The biggest progress in VRML 2.0 for scientific applications, compared to the previous version, is scriptability. In VRML 1.0, the relative geometric positions of node objects were fixed. VRML 2.0 introduces a whole set of new techniques for animation. [12] Among these are interpolator nodes, an event routing mechanism to make objects react when certain actions are performed by the user, collision detection among objects and, most important, script nodes. Script nodes can contain nearly arbitrary instructions in VRMLscript [13], a JavaScript variant ([14], soon to change the name again to ECMAscript [15]). A Java interface (EAI, External Authoring Interface, [16]) to control the status and behavior of VRML nodes has also been defined. However, on the Java side, the Java3D [17] API (Application Programming Interface), which has no relationship to VRML, is being touted as a better integrated and potentially faster alternative to VRML. A serious struggle for dominance between these two standards for animated, interactive Internet-based graphics should be expected as soon as full Java3D implementations become available.

The first chemical VRML applications which use the extended 2.0 feature set have already made their appearance. One of the first and most convincing applications [18] contains a clickable image of a simulated IR spectrum, which is coupled to a Java-animated molecular VRML scene. The VRML part visualizes the normal vibration belonging to the selected peak. While the atoms move according to the current vibrational mode, the molecule can be rotated and inspected by the user.

Pure chemical Java applets which do not rely on a native Java 3D library or a VRML API have also been created for demonstration purposes. A Web page with a Java applet which performs a molecular dynamics (MD) simulation on the client`s computer and continuously displays the calculated vibrations in several display styles can be visited at [19].

For many application areas it is desirable to have script-animated 3D scenes which can be viewed from freely chosen viewpoints, or provide guided tours highlighting points of special interest in complex structures such as proteins. Video sequences, which have been used with some success to visualize chemical processes, do not provide the freedom to select the viewing position. Also, video does not offer any choice to remove parts of the scene for a clearer view or to change display attributes of scene objects, which are both useful features to clarify the image of chemical structures.

Simple trajectories following a pre-computed path can be replayed by many 3D molecular graphics programs, but only a few provide the flexibility to use scripted, annotated animation sequences which display molecular animation and/or annotated scene sequences. The user should be allowed to interrupt program operation and continue with his/her own explorations. KineMage [20] has become a standard in this area, especially in the field of protein structure feature exploration. The VMD [21] program has the necessary scripting and trajectory processing abilities to be a comprehensive solution, but is significantly more complex to operate. The Chime plug-in [22] possesses currently rather limited scripting abilities. They will probably be extended in the future in order to establish Chime as a de-facto standard for Web-based structure visualization. The advantage of a Web browser plug-in is that it can be used directly within a Webpage and thus does dot demand switching between a full-text viewer and the display application. KineMage can display textual paragraphs, but with very limited formatting and linking capabilities compared to HTML pages.

VRML and other media described in this paragraph have proven their usefulness as transport media for chemical 3D information [23]. Embedding this information into a Web environment and transporting it to recipients who do not need to have specific hardware available or software installed should be considered as an achievement per se. However, none of the current chemical VRML applications can even be remotely called an VR application. In the next paragraphs, we will discuss chemical VR applications which adhere closer to the original definition of VR.

Virtual Reality Environments

What we find in today`s Web-accessible chemical media is decidedly not VR. So what kind of technology is used for more immersive VR?

VR comes in a lot of incarnations, and most require special hardware for input and output. The full effect of virtual reality is attained through three basic components: A high-performance computer graphics system, a head-tracked display that presents the virtual world from the user`s current head position, and three-dimensional input devices that allow the user to provide input directly in three dimensions.

There are several ingenious technological approaches to facilitate the creation of three-dimensional impressions, reaching from simple stereo images to fully immersive environments. The ultimate aim is to invoke the feeling of being an integral part of an alternate reality.

Some of the commonly employed devices to produce a visual 3D impression are well known to chemists. In the old technique of stereo images, two pictures showing the same scene but rendered from a viewpoint approximately three degrees different, are displayed side by side. [24] The person viewing them can be aided by stereo goggles with mirrors inside. These mirrors make it easy to for the images to be seen exclusively by one eye each. Talented viewers can simply relax or squint their eyes until each eye sees the correct image. This method allows color and animation in the images. Other systems such as anaglyphs (red/blue overprints, viewed through colored glasses [25]) or random dot stereograms (SIRDS [26]), which were a fad a while ago, support color only in a very restricted fashion and are therefore not suited for more sophisticated applications. This does of course not imply that these techniques have not been used successfully for limited chemical visualization or even videos.[27]

The next level of sophistication uses shutter glasses. These are transparent LCD panes which can be blackened separately for each eye by applying a voltage. The voltage is synchronized, typically by an IR signal, with the video display, which switches in rapid sequence between two different views of the scene, one for each eye. In contrast to parallel stereo images, the display can use the full area, and the head can be moved. Of course the change of the images must be fast enough to avoid flicker, and the display bright enough to produce a well-lit impression if viewed only half of the time by each eye.

Head-mounted displays (HMD) and binocular omni-oriented monitor (BOOM) systems do not use a single screen any longer. They consist of two CRT or LCD screens, one for each eye. A HMD is worn like a helmet with a face cover. The user movements are tracked by auxiliary devices so that the display generator is always informed about the head position and view direction. A BOOM is connected via a system of rods to a fixed pedestal. At the end of the extension, a view box with the screens is located. When the user pushes, pulls, tilts or otherwise moves the view box, not unlike a periscope, his/her current location and orientation is reported by sensors measuring the angle of the joints connecting the rods.

One of the most spectacular, but also most expensive setups for virtual environments is the CAVE (Cave Automatic Virtual Environment, a recursive acronym [28]). The first CAVE was installed in 1992. A CAVE is a box, measuring about three meters on all sides, installed inside a larger, dark room. The walls, floor and potentially also the ceiling consist of rear-projection screens. Images are projected from the outside by video projectors. The user steps inside the cave and wears shutter glasses for an additional enhancement of the scenic impression. More than one user can be inside at a time, but only one user determines the viewpoint, the others are taken along as in a roller-coaster ride. CAVEs usually also include 3D-audio processing equipment, so directional sonic stimuli can be used to support the 3D viewing experience.

For many applications, a fully immersive simulation is not necessary. There are a number of interesting devices which produce only a small virtual dispay region. The Responsive Workbench [29] and the Immersadesk [30] are prototypical examples. These are tables with projection facilities hidden inside and a projection screen as surface. People can group around it, wearing shutter glasses, and three-dimensional objects appear above the table. Images are generated by the usual stereo image display switch mechanism. The workbench paradigm is well suited to model a number of work-related environments, such as laboratory workplaces.

A number of direct volume displays have been built which project a true 3D image into an enclosed volume. [31] The three-dimensional image can be viewed from the outside without any mechanical aids because the projection space is itself three-dimensional. One of these displays uses a rotating plexiglass spiral inside a cylinder. Laser light is absorbed and scattered by the plexiglass, but remains invisible when crossing the empty volume. The laser projection is synchronized with the spiral rotation. The spiral occupies every volume element of the cylinder for a short time. The resolution of these displays is currently rather low, and the axis in the middle is a problem if a rotating surface is involved, but it is a notable step towards true volume display.

Computer-generated holograms (CGH,[32]) are another 3D imaging technology which may become more widely used in the future, because no headgear is necessary for viewing. Computing the interference patterns which form a monochromatic or white-light hologram is not trivial but has been done even in real-time on massively parallel computers. The main problem lies in selecting suitable output devices. The resolution of the patterns on film or other media must approach the wave length of light the make the interference pattern work. Common output equipment such as printers, monitor screens or slide exposers provide resolutions which are magnitudes too low. Creating holographic prints or films currently involves using tools such as modified scanning electron microscopes to write the patterns with a tight electron beam onto film [33], or acousto-optic modulators, where ultrasonic waves are transmitted into a crystal whose refractive index is locally changed by compression. A wide laser laser beam shining into the crystal experiences interference. [34] Holography is the only imaging technique which gives all visual depth cues. Stereoscopic images to not provide ocular accommodation, and volume displays cannot provide occlusion. Holograms of chemical structures exist, but have not yet received recognition beyond showcase pieces.

Keyboards and mice do not adapt well to the immersiveness of VR, so advances in voice, gesture, and touch recognition will be essential to facilitate user input. Input elements which have been proven to work well in 3D environments are for example Space Mice and Wands. A Wand is basically a joystick plus a set of buttons, but in addition it transmits is position and orientation is 3D space to a receiver via radio signals. User and wand positions are registered by sensing disturbances of the electromagnetic field caused by pulsed magnetic coils in the device, or by receiving ultrasonic or radio impulses from the devices via multiple sensors or antennae. Tracking the user position is important because in projection-based VR, the positions and view direction of the head must be known to adapt the projected scenes.

Trackers and Wands must be worn and are an encumbrance. In the VR field there has always been an interest in methods to determine the user`s eye positions without additional gear and to research control mechanisms which do not depend on external devices. Cornea reflex tracking with infrared beams has now become facile and is an efficient method to determine the direction of vision. Program control mechanisms based on gesture recognition [35][36] have also left the laboratory stage. Gesture recognition is an improvement to the older data glove. A data glove [37] is a glove with sensors which measure the crease angle of the fingers in the glove, and may be coupled with a tracker to register the position of the center of the hand in 3D space. Another technology which is useful in the VR context [38] are speech recognition systems [39], which have been a research topic for many years outside this particular application field.

Reality, virtual or natural, is of course more than a visual image of a scene, although it has been estimated that more than 90% of the volume of the data streaming into the cortex is coming from the retinae. If an attempt is made to model reality, other senses should not be excluded and could provide valuable auxiliary data channels for the representation of complex data relationships requiring a maximum bandwidth input. This is really an area of active research, but its results seem not to have yet applied in the chemical domain, except sonification.

Sonification [40] is the dynamic creation of spatially located, distinctive sounds by mapping property values or system states to frequency, timbre or amplitude of sound. It has been applied to chemical modeling, although no details of the process and property mappings seem to have been published. Tactile stimuli by coating the surface of hardware molecular models according to property values have been used on computer-generated hardware molecular models (see later section) and in the virtual domain in the educational Pauling World VR environment.

We are not aware of any chemical application using other feedback channels [41], such as temperature or moisture stimuli, force, tactile stimulation by vibration or pressure feedback [42], dispersion of olfactory chemicals or direct stimulation of brain waves. These stimuli have all been brought to use in other VR-related experiments and may find a creative use for chemical information processing in some future projects.


Immersive VR Molecular Modeling

Immersive VR for scientific visualization has evoked a lot of interest, [1]. A review book which contains some information about this subject has been published. [43] In order to attain the virtual reality effect, the system must deliver very high graphics performance. Specifically, requirements for a usable VR environment include a user feedback response time of less than 0.1 s for fast, accurate manipulation of the environment, a frame rate of at least 10 frames per second to avoid annoying flicker and image jumps, and finally the application environment must contain objects if a sufficient level of fidelity to allow the performance of meaningful tasks. For practical applications this means that for datasets of scientifically interesting complexity for the visualization part alone nothing short of multi-processor SGI Reality Engine as renderer alone will suffice to produce the desired level of detail and response time. Additionally, extra processor power must be assigned to model the system response.

The motive behind the step into the third dimension is that non-immersive interactive visualization systems implemented {} for the desktop and mouse are effective for moderately complex problems only. Virtual reality displays aid in the unambiguous display of these structures by providing a rich set of spatial and depth cues. Virtual reality interface concepts allow the rapid and intuitive exploration of volume containing the data, enabling the phenomena at various places in the volume to be explored."[1] Chemistry certainly deals with complex 3D data, especially in the protein modeling field, and so VR was used for chemical applications very soon after these systems became available.

The most spectacular application of VR in chemistry is immersive protein modeling in a CAVE. Numerous systems and toolkits for modeling in this environment are under construction, and public demonstrations have been given.

The hardware price tag of the equipment used in these studies is usually impressive. In one representative study, two SGI Onyx (20 and 8 processors, 2 and 3 Reality Engines) for the graphics in two CAVEs, an IBM SP2 massively parallel computer for the docking computations, and an SGI Indy for voice processing. This setup was barely able to generate about 10 frames per second, although it must be mentioned that as an additional complication the data (geometry, tracking, video audio) was exchanged over a high-speed network between two sites in order to demonstrate the feasibility of remote collaboration in such environments.

In this study [44], which was carried out at the Cornell Theory Center, the docking of chlorpromazine and mepacrine with the trypanothiophane reductase of Trypanosoma cruzi, a parasite causing Chagas` disease, was studied. The primary focus of this project however appears to have been a test for the WorkSpace toolkit used to implement the 3D user interface with its various manipulatable objects such as molecular models, directories, video panes showing the collaborators at the other site and models of the parasite and its vector, a small bug Triatoma infestans. Figure 1 shows a snapshot from the modeling session.

The combination of visualization and molecular dynamics is very popular and found in most of the state-of-the-art chemical immersive VR setups. The rationale is that these systems enable molecular scientists to have a visual and auditory experience of a chemical system while manipulating its physical properties by steering, in real time, a simulation experiment on a supercomputer. As a result, scientists are immersed in a realistic representation of a chemical system. Most traditional molecular modeling environments are limited to qualitative information that is obtained from static 3D models. However, complete understanding of molecular interactions requires knowledge of both the dynamic and static features of molecular systems. This is achieved by merging real-time molecular dynamics computer simulations, which can be very computationally expensive, with an immersive display of the chemical system.

One of the more simple applications of CAVE modeling without the need for massive computational backup for simulations has been the visualization of protein surfaces for the comparison of several geometric representations, including the simple space-filling sphere model, the solvent-accessible surface, molecular surface and the novel alpha complex algorithm.[45]

The C2 Interactive Molecular Modeling project at the Iowa Center for Emerging Manufacturing Technology aims at supporting molecular biologists in guiding drug molecules into the active site of a protein while receiving real-time feedback from a molecular dynamics simulation.[46][47][48][49] The system in operation is shown in Figure 2. It supports the typical molecular mechanics/VR combination of features.

The Virtual Biomolecular Environment (VIBE) on the Information Superhighway is another high-profile collaborative effort from the Argonne National Laboratory, Iowa State University and the University of Illinois, once more with support for the molecular dynamics/VR imaging combo. [50] The VIBE system uses a custom parallelized version of the CHARMM program which runs on a 64 node SP2 parallel computer. The performance is about 15 computation/rendering frames per second with 16 SP2 processors for HIV protease plus a drug-sized ligand molecule, and about 48 frames for pure display without MD. The interactive tasks of the chemist using this system include the placing of the structures on starting points for the docking process and the modification of torsional angles. The potential energy of the current configuration is expressed as audible sound effects. The combination of sound, visual impression and immediate response is reported to have been well received by scientist test persons.

A third system called STALK is a little different in its setup because its computational core is not a molecular dynamics program. STALK is also used for the docking of ligand molecules to proteins. The computational side of STALK is a genetic algorithm optimizer which changes the molecular conformations in an attempt to minimize the molecular interaction energy. The scientist in the CAVE interacts with the genetic algorithm module by manually changing the position of molecules to guide the search and setting various parameters of the optimizer while watching the docking progress.[51]

The VMD molecular visualization program [21] from the Theoretical Biophysics Group at the University of Illinois is a hybrid. It works both in VR environments and with traditional screen displays. VMD is the visualization component of MDScope, an interactive environment for molecular modeling and dynamics. Part of the project are experiments in new modes of human-computer interaction. In conjunction with the parallel NAMD molecular dynamics program, VMD once more supports interactive ligand-receptor docking. Another application area is intelligent structure analysis, where part of the molecule are pointed at and natural-language questions about the selected features such as helices are answered. In the VR arena, VMD is used a a test-bed for gesture analysis and speech recognition in molecular biology. [52]

While most of the pioneering applications of immersive VR for chemistry originate from the US, visualization centers in other countries are catching up and beginning to develop their own ideas about chemical VR. The High Immersive Molecular Modeling project at the VIS-lab of the Frauenhofer society in collaboration with the biology department of the University of Stuttgart is a recent example.[53] While the primary focus of HIMM is once more docking and property visualization, the next project step will include for the first time interfaces to chemical information retrieval systems and thus open a connection to database applications which play a very important role in chemical research.

Not all chemical immersive VR has been performed in the CAVE environment. The Responsive Workbench of the GMD (Gesellschaft für Mathematik und Datenverarbeitung) visualization research institute has also been used as an augmented reality modeling tool, as demonstrated in Figure 3.

Finally, VR in chemistry is no longer confined to research. Pauling World [54] is the first chemical VR environment which was exclusively designed to teach challenging concepts in science to students. Pauling World runs on a 4-processor Onyx Reality Engine with a HMD, a magnetic positional sensor, stereo sound and a custom vest which delivers haptic sensations. Pauling world is only one virtual world from a larger collection of educational worlds which form the ScienceSpace. The rendering capabilities of Pauling World are currently limited to various rigid standard molecular display forms (ball& stick, wire frame etc.) which can be opened and closed to icons by pointing with the index finger. At the time of writing, there appears to be no other method of interaction or animation. The didactic background and the reason for the use of definitely expensive equipment to deliver conceptionally simple visualizations which do not go beyond those of standard 3D molecule viewers which run on every PC, is not evident to us, in spite of having access to about a dozen full-text publications from the Web site. The other worlds of this project offer significantly more interaction and VR experience, justifying the VR effort, so there may be interesting developments in the future.

Few researchers are so lucky to have access to a CAVE or similar equipment to experiment with immersive modeling. This type of environment is very demanding both from the visualization side and from the force field responsible for the structure optimization, and thus expensive. However, smaller systems which use comparable algorithms and run on standard computer hardware exist. They rely on a normal computer monitor for output, use ingenious markers and other visual hints to compensate for the less perfect three-dimensional impression and employ a radically simplified force field to provide interactive response times for molecules up to a few hundred or thousand atoms. This class of programs is probably the first application in chemical modeling where the primary information gain is from the direct 3D interaction with molecular structures. In these programs, the visualization is not simply a display of some final computational results, but the graphical representation and the user interacting with it is the continuous source of new information.

The best known among these programs is Sculpt [55][56], now available in a commercial version from Interactive Simulations. [57] In a Sculpt modeling session, the user pulls atoms with the mouse, for example while attempting to dock into a receptor pocket, and the program continually minimizes the potential energy of the structure. Visual cues are provided by highlighting good and bad van der Waals or electrostatic interaction as cup-shaped symbols, so the user sees immediately why the energy goes up or drops during a specific operation. In contrast to many classical fitting programs, no isolated atom positions or bond angles are changed. The whole molecule is continually reshaped according to the forces originating from the pull point and atomic interactions. Figure 4 is a snapshot of Sculpt modeling.


Document Navigation

Information is at the heart of all chemical research. Results are generally not produced from a void, but by expanding existing knowledge. No project can be started without obtaining precise and complete information about related work which has been performed (and possibly patented) elsewhere. At the core, the whole process is concerned about finding comparatively few documents from an enormous body of literature, patents and other information sources, typically still in printed form. Information publishers and information consumers can both profit from improved access paths. If people fail to understand the full range and nature of information available, this can be costly and damaging for both sides, especially in an industrial environment. The established big abstract databases and full-text sources fail to deliver user-friendly advanced searching and navigation. Their typical access and query methods are cumbersome and no longer state of the art.

Document navigation is of course a topic which is of central interest for digital publishing and consequently for this journal. In their daily work, chemists rely heavily on access to documents, and the search for relevant literature can consume a significant part of the time allotted to a project. Digital Internet-based publishing makes the access to the full-text information easier, but does not necessarily provide a solution to the problems of finding the right papers in the first place. While the hyperlink model works well enough for linking digital documents to hypermedia attachments, and to build a hierarchy of access paths leading from the homepage of a publisher via its individual journal pages, indices of numbered issues to the single paper, this access path is very much geared to the subscriber who peruses every issue of a journal or the researcher who has obtained a citation and knows precisely which paper he or she wants to retrieve. Navigation within concept space is completely different. While it is certainly possible to locate interesting papers by author, keywords, or full-text search on the abstract or the complete textual corpus, there is always the danger of missing interesting papers due to nomenclature differences, missed keywords, unknown authors or boundaries between different sciences or fields within a science. Algorithms exist which cluster documents by statistically derived word/content relationships [58] or by analyzing Web linkages [59], and will be able to find relevant documents even when nomenclature varies, provided that documents are stored in the database which builds a bridge between different concept spaces and nomenclature systems. While it is possible to provide a textual query interface to this type of system [60], navigating a multidimensional document proximity space benefits from the 3D metaphor, as it has been shown in the Xerox Parc Z-GUI project. [61]

Since the mid-80s, it has been questioned more and more whether the ubiquitous WIMP (windows, icons, menus, pointers) user interface approach [62] would be able to cope with the increased amount of information which needs to be handled in the workplace of scientists, engineers and clerks. It has been complained that mind-numbing mechanical activity would get in the way of users spending time and energy on what is really relevant and what the users want to accomplish.[61] The current desktop model and its inherent navigation conventions are 2D - but is has been proposed that 2D is not the space we are living in and that from biological considerations alone 3D should be a much more natural space to navigate in. There has been a considerable interest in the development of intuitive 3D-widgets. One focus has been the navigation of document collections, which tend to become too large to be handled efficiently by the traditional interfaces.

At Xerox Parc, the use of 3D and dynamic 2D graphics for navigating large information repositories has been studied for more than ten years. The result of this research include innovative 3D widgets such as the Hyperbolic Tree, which allows to navigate hierarchies hundred or thousand times larger than a conventional view can deal with. By simple click and drag, the display warps to gain a new perspective on different portions of the hierarchy. The Table Lens object helps to make sense of large table hundreds of times larger than a normal spreadsheet view will allow. The Table Lens fuses symbolic and graphical representations into a single understandable view. The Cone Tree organizes vast numbers of documents into a tree structure. The Perspective Wall organizes documents according to multiple criteria to reveal patterns and relationships. If objects are clicked at, the wall moves into focus to home in on the specific clusters of documents or data of interest.

Apart from navigating contents, 3D models even for organizing the access to standard unordered Web information sources have been proposed. One well known example is the WebBook/Web Forager project. [63] The authors of this project claim that their 3D approach yields significant improvements in integrating WWW information foraging with other information processing activities. Another example with comparable goals is the VRMosaic project. [64] In VRMosaic, the Mosaic Web browser functionality is transferred into a multi-user fully immersive VR environment. One of the aims of this project has been to prove that classical 2D applications can be embedded into a VR environment and become important supporting features for documentation and referencing purposes when operating on 3D data such as engineering models or complex scientific visualizations.

Document navigation has of course many links to database interfaces, since larger document collections with a common structure tend to be stored in full-text databases. 3D database interface models will be discussed in the next section.


3D Database Interfaces

Databases play a very big role in chemistry. Compared to other sciences, a much larger part of the relevant literature is extracted, and selected information stored and made available on-line. Chemistry has also developed a number of unique database types, such as structure and reaction databases, where molecular graphs and graph transformations are queried, or spectral databases, which employ spectral similarity operators. So chemistry has both a large number, and very diverse types of databases. Currently, access to almost all databases is limited to a one-dimensional command-line model or 2D graphical interface models, with response forms and 2D schemes. Hits are typically displayed in a sequential order, sometimes clustered to reduce the tedium of paging through very many similar hits.

The traditional 2D display model has become a limitation in many respects. 3D database interfaces are currently a major focus of research in the database community. This trend is especially observed with respect to object-oriented databases, when there is no longer a database structure with can be easily represented with tables and navigated by requesting query values in columns. Object-oriented databases are increasingly used, especially in bioinformatics, to cope with the rapidly growing amounts of very diverse data from genome analysis which is difficult to map onto classical relational table structures. The concept of 3D interfaces to databases is unrelated to the storage and retrieval of 3D atomic positions in structure databases or the provision for 3D search operators on the data, which is of course a well-known feature of numerous chemical databases. There is a rich literature on the problems of 3D structure searching, and this topic will not be part of this review.

3D database interfaces [65] let the user interact with the database and its contents in 3D displays. Chemical and chemistry-related databases have always been a favorite target for database interface experimentation, beginning with one of the very first 3D database interface ever implemented, which used the periodic table as a minimal chemical database. [66] Two basic application fields can be distinguished: Query building and result visualization. In recent systems, there is a trend to integrate both aspects into a common environment.

Visual query languages (VQLs) simplify formulating queries and understanding the relation between the query and its results. [67] They use visual representations of the database structure and contents, where direct manipulation of input devices controls both selection of information and query formulation. Many systems implementing VQLs adopt visual representations and interaction strategies with concepts such as diagrams displaying the relationship between objects or icons which supply immediately graspable representation of object classes.

The AMAZE project [68] is probably the first instance of a 3D interface to a complex chemical database. The developers of this system share the conviction that the WIMP interface paradigm with its desktop metaphor has outlived its usefulness, because it does not provide the expressive power software engineers require when faced by the currently observed shift in priority from functionality to usability. Since most chemists still relegate more complex database queries to experts, the chemical community which has been surrounded by powerful, but complex database systems can probably relate to that statement. The main object-oriented database of the AMAZE system contains protein structure data with object classes such as the basic atomic structure, helices, loops, strands and other descriptors known from the chemical literature. These data objects typically are part of one-to-many or many-to-many relationships and are of a type which is awkward to encode and search efficiently in a classical database. Specifying meaningful and syntactically correct queries on the right objects from a complex database structure (about 20 object classes) with a textual query language has proven to be very difficult. It was however found that biochemists, who are routinely using 3D graphics in their modeling work, became proficient very rapidly when a 3D interface was offered. In this interface, the conceptual schemes and returned sets are depicted as blocks, forming a result maze. Biochemists can navigate to blocks which are a result set and attach more query operators to this set until the set is reduced to members which exhibit a complex pattern of characteristics in the various descriptive database classes. Query operators and operator combinations are represented by various shapes. Editing the parameters of an operator box is performed by selecting a block and opening input panels.

The WINONA database interface [69] also operates in three dimensions, but employs a completely different set of metaphors for database navigation. Its central elements are hierarchical layers of object relationships visualized in 3D space not unlike multi-layered floor plans and a polygonal wall model. Every class occupies a sector of the polygon, with meaning attached to the size and other attributes of a sector wall segment. Selecting objects from the class containers by manual picking or query on the inside of the polygon creates various types of lines across the interior of the polygon, highlighting object relationships.

A final system called LyberWorld [70] focuses primarily on interfacing full-text databases in a VR or classical screen environment. Chemists know that retrieval of papers from abstract databases is often a problem. Finding the significant entries with a free vocabulary is not easy, and the number of abstracts yielded as the result of broad-range generic queries often so large that a manual perusal is impossible. LyberWorld uses innovative and very detailed abstract graphical objects in 3D space, such as the Relevance Sphere and the Search Tree widgets, to display relevance rankings, content relationships and similar information. LyberWorld has been demonstrated as a 3D interface to the INQUERY probabilistic information retrieval system of the CORDIS database. CORDIS stores textual information about research projects, including chemistry projects, sponsored by the European Community. Currently LyberWorld is being integrated with the ReLiBase database in the Docking-D [71] project. ReLiBase [72] is implemented on top of an object-oriented database system (VODAK) and stores data from heterogeneous sources in an integrated receptor-ligand database. The main application of ReLiBase is posing associative queries for analyzing the available data, exploiting the object relationships derived from different databases and algorithms under the ReLiBase umbrella. The handling of large numbers of textual documents in LyberWorld is shown in Figure 5.

The systems described so far were designed for single users. The concept of Populated Information Terrains (PITS) [73] extends database technology by multi-user VR and Computer Supported Cooperative Work (CSCW). Various other models of telecooperation for chemistry are discussed in the Collaboration and Conferencing section of this review. The rationale for using PITS were twofold. First, traditional database interfaces provide ample support for sophisticated querying (if the users can master the grammar), but lack in support for browsing. Because of the larger information density in volumetric representation, navigation for browsing should be facilitated. The density of information comes from the fact that attributes of entries or tables can be mapped both to extrinsic dimensions (position) and intrinsic dimensions (color, shape, size, spin, texture) simultaneously and there is a gradual scaling of visibility, as opposed to paging through traditional 2D output, where a record and its attributes are either visible in detail or not at all. Second, many databases provide concurrent multi-user access, but do not offer any method to share the data as part of cooperative work. Using systems like Q-PIT or DIVE-Q, multiple users wearing HMDs (Head Mounted Devices) can cooperatively explore data relationships and manipulate their common workspace by attaching queries or performing other manipulations.

There exist a significant number of other approaches to visualize retrieved data which could have interesting applications in chemistry. Space does not permit a further elucidation. References [74] and [75] contain overviews and links.

Recently plans were announced to combine immersive modeling with 3D database interfaces in order to be able to retrieve and analyze properties of structures taken from databases with the instant option of moving objects from the database to a new modeling process or comparing modeling results with data from previous experiments stored in a database. Refer to the HIMM project described in the immersive modeling section of this review for a link to such a project.


Data Mining

Closely related to the database interfacing issues outlined in the previous section, which focus on navigation, query construction and browsing, are the recent developments in the field of data(base) mining. [76] The term database mining describes the process of sifting through large amounts of data with the aim to detect significant relationships between individual data items and to create rules with predictive power from these observations. A scientist can work effectively with a few thousand observations with a small number of measurements each. Effectively digesting millions of data points, each with tens or hundreds of measurements, is another matter. A variety of techniques are used for database mining, including hypothesis generation and verification, unsupervised global search for interesting patterns, supervised goal-directed mining, and graphical visualization of data relationships. [77] Since the variable space is often of a very high dimension, or the structure of the results, such as decision trees, has some complex tree-like shape - convincing visualization of the findings of the mining process is of considerable importance. Walkable 3D-scenes for trees, rules and evidence patterns have been found to be a useful model for this type of application. Trees which are larger than what can be printed in readable form on a sheet of paper are difficult to view and navigate with traditional 2D-type tree displays. Database mining is a field in chemistry which has just begun to become systematically explored, although there is a considerable potential. Many companies possess extensive datasets, for example from QSAR studies originating from drug development projects. Until recently, QSAR studies were typically limited to small datasets of similar compounds. With the progress in available computational power and mining algorithms, large-scale reevaluations of the data collected over years has prospect to unearth hitherto overlooked patterns which might lead to reactivated projects leading into areas not explored in the original work. Another promising application area is the analysis of DNA and RNA sequence collections. VR environments, as well as novel non-immersive display metaphors, have been found to be very effective tools for the supervision and guidance of mining sessions.

The SANDBOX (Scientists Accessing Necessary Data Based On eXperimentation) system [78] has been designed as a general-purpose interface to scientific databases which contain large tables of measured data. The SANDBOX is a virtual reality tool which allows an investigator to recreate` the original experiments, collecting data from the scientific database in much the same ways that the original data was collected. The investigator places virtual instruments into a virtual environment and collects data from the scientific database without ever typing in a query. The instruments give feedback, allowing the user to browse through the data. It has been demonstrated on data collections containing hundreds of tables, more than 100 gigabytes of data and visual material such as maps that the VR approach is accepted by scientist test users as very natural and effective.

Another system with a similar application area is the Virtual Data Visualizer. [79] The VDV is a highly interactive, immersive environment for the visualization and analysis of data which does not inherently focus on a specific type of data. Data is represented by glyph elements which the user can make, change, or remove on a workbench. This system has already been used for chemical data analysis, for example to visualize and control molecular dynamics runs on cluster/surface collision interactions. Figure 6 is a snapshot of a chemical VDV session.


Virtual Collaboration and Conferencing

There has been an increasing demand to share graphical tools or full virtual environments for scientific data analysis and access to instruments without having to meet in person. [80] General-purpose conferencing tools for the transmission of audio, video, textual information and whiteboard drawings are readily available and can form a basis for scientific collaboration applications. An extension of the basic tools into the 3D and virtual reality domain is natural and helpful for many fields. The point with this class of application is that graphical objects such as molecules can be manipulated and discussed by two or more parties simultaneously, with immediate display of the result of manipulations and annotations at all involved sites. Collaborative data exploration environments are used in a wide range of scientific disciplines, [81] but chemistry has its own requirements on the type of data exchanged.

A number of tools exist which were specifically designed for chemists to exchange and manipulate 3D data during a conference session. In one of the first implementations, interactive data exchange and manipulation has been achieved through a combination of WWW servers and custom chemistry modules for the IRIS Explorer visualization program. [82] [83] In another important implementation, data of the Molecular Inventor system of SGI can be exchanged in real time through the InPerson conferencing system. [84] The exchange of raw information is of course not sufficient. Methods to couple modeling and computational packages to the conference system are of course crucial in order not just to be able to manually highlight features and discuss them, but also to submit structures to computation and numerical analysis immediately in order to verify observations or examine the consequences of structure manipulations. Integration of conferencing and data analysis/production packages is making visible progress, for example in the MSI Cerius II / InPerson program couple. [85]

The access to remote laboratory equipment is another area where conferencing-type programs are used. Examples are the Spectro-Microscopy Collaboratory at the University of Wisconsin-Milwaukee and the Lawrence Berkeley National Laboratory [86] and the Core test-bed at the Pacific Northwest National Laboratory [87], where the remote operation and data sharing of NMR and MS spectrometers is one of the project goals. Both projects are part of the Distributed Collaboratory Experiment Environments (DCEE) program. [88]

Finally, even complete CAVE immersive VR environments have been coupled, for those who have access to the expensive hardware and the necessary bandwidth.[89] On a smaller scale, the representation of coworkers as avatars or other integral scene objects in distributed 3D environments is an area of active research.[90] In the database-related fields, first experiments are taking place with multi-user VR database access models. Refer to the database section of this paper for more information.


Real Virtuality

In spite of all the described approaches to enter new virtual dimensions of three-dimensional visualization by all kinds of sophisticated devices, many researchers think that 3D molecule models built in touchable form are still the most intuitive and easily manipulatable form of structure display. Computer technology has come to aid also in this case. A number of innovative apparati can be used to produce highly detailed molecular models from a variety of materials. These models can be touched and put into display cases - but of course they are static in nature. Docking experiments will not model structural changes in ligand or receptor. If the models are bent or stressed, the shape alterations will generally not mirror the computed or observed changes on the molecular level.

The direct drilling of models from plastic blocks on a lathe is rarely a viable alternative. Molecular models tend to possess cavities and non-convex parts, which are often the most interesting regions. These are inaccessible for drills and other instruments. For the same reasons, the molding of models by casting liquid polymers into forms is difficult because of the problems involved in manufacturing the form.

Two new technologies for solid rapid prototyping have been successfully applied to build hardware models of proteins and other chemical structures with complex surfaces. [91] The first method, stereolithography, uses a vessel filled with liquid monomers. One millimeter or less below the surface a pedestal which can be lowered by a motor is installed. A switched UV laser scans the surface, inducing polymerization at the irradiated surface points. After each surface scan, the pedestal is moved downwards by the thickness of one layer, moving the polymerized surface parts deeper into the monomer bath and protecting it from further laser irradiation. Slice after slice is produced this way. The polymer of a new layer will form a homogenous material when regions overlapping the layer below are irradiated. Layers, which are intersections of the model, can consist of unconnected polymer patches, so arbitrary shapes can be produced, as long as there are holes allowing the monomer to flow from the interior of cavities.

The second method called LOM (Laminated Object Manufacturing) uses stacks of paper. Sheets of specially treated paper are spread onto a growing stack. After a new sheet has been added, a laser cuts parts of the topmost paper sheet, and the new sheet is laminated by heat and pressure to the paper stack forming a solid block. When the stack has been completed, the outer parts of the stack can be torn off and the model core appears. With respect to cavities and concave regions, this procedure is not as flexible as the polymer bath method, but its capabilities are sufficient to produce instructive protein models. [92]

On a much smaller scale, direct manipulation of atoms on surfaces has become a reality with Scanning Tunnelling Microscopes (STM) as sensor and effector. Instead of creating macroscopic real models of compounds for virtual manipulation, microscopic real entities are manipulated from macroscopic virtual models. A VR environment, the Nanomanipulator [93], is under development which translates actions of the chemist into voltage pulses send to the microscope needle which can be used to move single atoms on surfaces. The scientist wears a HMD and operates with a mechanical arm, not unlike those used in radioactive isotope processing plants. The chemist can feel the surface structure by moving a pointer with the mechanical arm over the virtual sample surface. This cases the sensor tip of the microscope to be moved, and the mechanical arm produces a force feedback according to the voltage at the sensor tip. The Nanomanipulator has been used to study the mechanisms of surface modifications on gold and silicon substrates, monolayers of various carbonic acids, and in a number of other projects.


Creative Abuse of Chemical Visualization

Superficially, it appears that many visualization requirements of chemistry are peculiar, because chemistry has structure graphs in 2D and 3D and many other unique objects of interest which have no counterparts in other fields. This observation sounds obvious, but on closer inspection, it is wrong.

Graphs are not limited to chemistry. Chemistry certainly has a rich tradition of handling this type of information in 2D and 3D, but this is no unique interest. Graph layout is also of much interest to the mathematical community, although the more general graphs studied in this field are often much more complicated than the average structure graph with its limited range of nodes and edges per node. The handling competence of chemistry with respect to small-sized graphs and its materialization in a wide range of readily available visualization programs has recently been discovered by other sciences where graphs are used. One notable example is sociology, where graphs are for example exploited to show relationships between individuals and groups. It has been demonstrated that chemical structure display programs for 2D and 3D data are very useful tools. By creatively encoding people or groups as atoms and social relationships as bonds, social links and group structures can be visualized. [94] Embedding these images into a VR context is the logical next step and has been proposed. Figure 7 is not a molecule but the result of a sociological study.


Does it Deliver?

We have seen in this review that there are quite a number of ongoing projects to deliver enhanced informational content through a variety of methods in 3D space. There are even a few start-up companies which are convinced that 3D information access tools or new breeds of modeling programs are a viable market niche, and certainly there a unspoken aspirations to be at the forefront of a major new technology wave.

However, virtual reality environments, transportable reactive or scripted 3D scenes, direct response modeling and 3D information navigation are all very recent developments, as can be seen from the dates on the cited literature references. Some of these technologies still require computational resources and special-purpose hardware far beyond the budget of ordinary users. As far as the necessary computer power is concerned, this is probably nothing which should be worried about in the long term. The performance numbers of desktop equipment continues to double every year and a half, so computations requiring a dedicated parallel processor costing millions of dollars today may be affordable mainstream desktop equipment in five years. Similarly, the performance of graphics boards is being pushed at a rate even faster than that of raw processor power. So these performance criteria which render many of the more advanced applications described in this paper unusable for most of the potential audience should not be considered a serious problem preventing VR to become a mainstream tool.

Another aspect to be taken into account is that some of the technologies remain to be usable with standard computer screens, probably in combination with lightweight auxiliary equipment such as shutter glasses or lightweight 3D input devices. These interfacing and modeling paradigms certainly have the potential for a more widespread use. The increase in data analysis performance actually experienced by educated users of such a system does not need to reach orders of magnitude to gain acceptance if the cost is not excessive, since the user is still able to work at his/her office without relinquishing the traditional work environment.

In constrast, the advantage of installing and using a fully immersive VR environment such as a CAVE, or other special equipment such as a VR workbench, is much more debatable. These systems must demonstrate their usefulness in a much more dramatic way in order to justify the investment in a cost-conscious environment. There are a number of obstacles to be overcome by this approach: The costs are very high and not likely to fall with the same speed as those of standard computer equipment, the size of the installation exceeds the space of a normal office, and the user has to leave his or her normal office to work with the system, disrupting the workflow. To make these systems a success, killer applications are required. Operated by a skilled and trained user, a data analysis or modeling performance must be achieved which without doubt outclasses 2D and simple 3D (computer screen projections, possibly combined with shutter glasses) technology for the same task. According to our judgement, this kind of performance still waits to be demonstrated for chemical applications. We feel that this is not impossible, since convincing demonstrations for other scientific fields such as fluid dynamics, oceanography [95], meteorological data analysis or military intelligence work [96] have been given. In the field of modeling, the described results obtained with immersive VR fail to impress from the modeling standpoint. We do not yet see how the results of the described docking experiments could not have been achieved in a more limited environment, provided the computer power for the forcefield updates with immediate response were available - or even with a more restricted forcefield on standard hardware.

If we claim that immersive VR modeling is an area where more work is needed on cogent demonstrations proving the power of VR, what about the promised advantages of using 3D metaphors in the other areas described in this review? The potency of using non-VR 3D graphics of all kinds for molecular visualization is probably unchallenged, since chemical structures are so obviously three-dimensional. In the area of 3D navigation and database queries, quantitative studies exist which were executed in a controlled environment and corroborate that 3D metaphors lead to a measurable improvement in the time required for tasks such as finding a document of given content or assembling complex database queries. Both are in fact everyday tasks of chemists, performed more often than modeling or 3D visualization. These tasks related to information retrieval exhibit specific chemical peculiarities but are certainly not unique to chemistry, or a subfield of chemistry. Modeling and molecular visualization however are domains of specialists. The potential number of users, and the market, for database access systems is much larger in comparison. We feel that once momentum is gained in these areas, a widespread introduction of the described class of systems for document navigation, database retrieval, and database mining can be expected within the foreseeable future.

The field of VR as a whole is of very recent origin, and scientist have to get used to the underlying ideas to be able to make creative use of its capabilites. Scientists in Wonderland [97] is a phrase which still describes much of the ongoing work. Making a definitive prediction about the future directions of VR is certainly difficult. Molecules and molecular interactions are inherently 3D, and chemistry has the largest document, factual and structure-related databases of all sciences. We are convinced that VR in general has promise to become an important tool in managing chemical information in the long run. The conceivable application areas in chemistry are very broad. They reach from the examination, modeling and documentation of the behavior of a single small molecule to distributed database navigation with millions of structures, spectra, sequences and documents. Exciting developments will most likely take place in the next years which have not yet been thought about.


References:

1. Bryson. S. Comm. ACM 1996, 39, 5/63
2. Hartman, J.; Wernecke, J. The VRML 2.0 Handbook, Silicon Graphics, 1996
3. http://vag.vrml.org/VRML2.0/FINAL/
4. http://www.vrml.org
5. http://askhp.ask.uni-karlsruhe.de/~doc/vrml/fokus/pages/bro_page.html
6. http://www.sgi.com/Technology/Inventor.html
7. Casher, O.; Leach, C.; Page, C. S.; Rzepa, H. S. Theochem, 1996, 368, 49
see also
http://chemcomm.clic.ac.uk/VRML/
8. Vollhardt, H.; Brickmann, J. Proc. Pacific Symp. on Biocomputing 96, Hunter, L., Klein, T. E., Eds.; World Scientific: Singapore, 1996, 663
9. Vollhard, H.; Henn, C.; Techner, M.; Brickmann, J. J. Mol. Graphics 1995, 13, 368
see also:
http://ws05.pc.chemie.th-darmstadt.de/vrml/
10. The WWW Chemical Structures Database at
http://schiele.organik.uni-erlangen.de/services/webmol.html and the JST structure database http://fact.jst.go.jp/cr-vrml.html
11. http://schiele.organik.uni-erlangen.de/services/vrml.html
12. Roehl, B.; Couch, J.; Reed-Ballreich, C.; Rohaly, T.; Brown, G. Late Night VRML 2.0 with Java, Ziff-Davis, Emeryville, 1997
13. http://vrml.sgi.com/moving-worlds/spec/vrmlscript.html
14. http://home.netscape.com/eng/mozilla/3.0/handbook/javascript/
15. http://www.ecma.ch/ecma-262.html
16. http://vrml.sgi.com/spec/moving-worlds/spec/part1/java.html
17. http://java.sun.com/products/java-media/3D/
18. http://ws05.pc.chemie.th-darmstadt.de/vrml/vib/
19. http://www.pc.chemie.th-darmstadt.de/java/
20. Richardson, D. C.; Richardson, J. S. Protein Sci. 1992, 1, 3
see also
http://www.prosci.org/
21. Humphrey, W.; Dalke, A.; Schulten, K. J. Molec. Graphics 1996, 14, 33
see also:
http://www.ks.uiuc.edu/Research/vmd/
22. http://www.mdli.com/chemscape/chime/chime.html
23. Bowie, J. E.; Olson, J. E. Data Visualization in Molecular Science: Tools for Insight and Innovation, AW Computer & Engineering Publishing Group, 1995
24. Foley, J.; van Dam, A.; Hughes, J. Computer Graphics: Principles and Practice, Addison Wesley, 1995
25. Watkins, C. D.; Mallete, V. P. Stereogram Programming Techniques, Charles River Media, 1995
26. http://www.sirds.com/see http://schiele.organik.uni-erlangen.de/gallery.html for a chemical SIRD
27. http://mofet.macam98.ac.il/~anna/compgraf.html
28. http://www.ncsa.uiuc.edu/VR/cavernus/
29. http://viswiz.gmd.de
30. http://evlweb.eecs.uic.edu/pape/CAVE/idesk/
31. Clifton, T. L.; Wefer, F. L. IEEE Comput. Graphics and Applications, 1993, 13, 4/57
see also:
http://www.elec.canterbury.ac.nz/labs/graphics/av_vol.html
32. Dallas, W. J. Computer Generated Holograms, in: The Computer in Optical Research,Topics in applied Physics 41, Springer: New York, 1980, 291
33. Leseberg, D.; Bryngdahl, O. Applied Optics, 1984, 23, 2441
34. Hilaire, P. S.; Benton, S. A.; Lucente, M.; Jepsen, M. L.; Kollin, J.; Yoshikawa, H.; Underkoffer, J. Electronic Display System for Computational Holography, in: SPIE Proc. 1212, Practical Holography IV, Benton, S. A., Ed.; Soc. Photo-Opt. Instr. Engrs: Bellingham, 1990, 174
Full text at
http://www.media.mit.edu/groups/spi/pierre/spie90.ps.Z
35. Huang, T. S.; Pavlovics, V. I. Proc. Int. Workshop on Automatic Face- and Gesture Recognition, 1995, 73.
36. Maggioni, C.; Kämmerer, B. Proc. Workshop on Computer Vision in Man-Machine Interface, Cippola. R., Ed.; Cambridge University: 1996, 34
see also:
http://www.bath.ac.uk/Centres/MEDIA/SIEMENS/GestureComputerMailFrame.html
37. Mulder, A. Hand Centered Studies of Human Movement Project Technical Report 94-1, Simon Fraser University, 1994
Available in full text as
ftp://fas.sfu.ca/pub/cs/graphics/vmi/HMTT.pub.ps.Z
38. Clay, S. R. IEEE Comput. Graphics and Appl. 1996, 16, 2/31
39. Rabiner, L.; Juang, B. Fundamentals of Speech Recognition, Prentice Hall, New Jersey, 1993
40. Begault, D. R. 3-D Sound for Virtual Reality and Multimedia; Ap Professional, 1994
41. Burdea, G. C. Force and Touch Feedback for Virtual Reality, Wiley, 1996
42. http://www.cogsci.kun.nl/~miami/taxonomy/node30.html
43. Virtual Environments and Scientific Visualization; Göbel, M.; David, J.; Slavik, P.; van Wijk, J. J., Eds.; Springer: New York, 1996
44. Wood, F.; Brown, D.; Amidon, R. A.; Alferness, J.; Joseph, B.; Gillilan, R. E.; Faerman, C. IEEE Comput. Graphics and Appl. 1996, 16, 4/72
see also
http://www.tc.cornell.edu/Visualization/Staff/richard/NIH/Faerman/Chagas.html
45. Akkiraju, N.; Edelsbrunner, H.; Fu, P.; Qian, J. IEEE Comput. Graphics and Appl. 1996, 16, 4/58
46. Disz, T.; Papka, M.; Pellegrino, M.; Stevens, R.; Taylor, V. Proc. 1995 Simulation Multiconference Symposium, Society for Computer Simulation, Phoenix, 1995, 483
47. Lent, G. E.; Rowlan, J.; Bash, P.; Cruz-Neira, C. 223. SIGGRAPH Visual Proceedings, Computer Graphics Annual Conference Series, 1994, 223
48. Cruz-Neira, C.; Sandin, D. J.; DeFanti, T. A. Proc. SIGGRAPH 93, ACM Press, New York, 1993, 135
49. http://www.icemt.iastate.edu/Projects/C2/html/molecule.html
50. Cruz-Neira, C.; Bash, P. in Proc. Workshop on Simulation and Interaction in Virtual Environments, digital proceeedings only at http://www.cs.uiowa.edu/~cremer/sive95/.sive-sessions.html, 1995
Full text at http://www.cs.uiowa.edu/~cremer/,sive-abstracts/cruz-neira-bash.ps.Z
51. http://www/mcs.anl.gov/home/levine/DOCKING/index.html
52. Sharma, R.; Huang, T. S.; Pavlovic, V. I.; Schulten, K.; Dalke, A.; Phillips, J.; Zeller, M.; Humphrey, W.; Zhao, Y.; Lo, Z.; Chu, S. Proc, 13th ICPR 96, 1996, 964
53. Drees, R. C.; Pleiss, J.; Roller, D.; Schmid, R. D. in Computer Science and Biology, Proceedings of the German Conference on Bioinformatics (GCB`96), Hofestädt, R., Lengauer, T., Löffler, M., Schomburg, D., Eds.; Univ. Leipzig, 1996, 190
see also
http://vr.iao.fhg.de/vr/projects/Himm/OVERVIEW-en.html
54. http://www.virtual.gmu.edu/pauling.htm
55. Surles, M.; Richardson, J.; Richardson, D.; Brooks, F. Protein Science 1994, 3, 198
56. Surles, M. Network Science 1995, 1, 2/2
see also:
http://www.awod.com/netsci/Science/Compchem/feature5.html
57. http://www.intsim.com
58. Pirolli, P.; Schank, P.; Hearst, M.; Diehl, C. Proc. ACM SIGCHI Conf. on Human Factors in Computing Systems, ACM Press, New York, 1996.
Full-text electronic proceedings at
http://www.acm.org/sigchi/chi96/proceedings/
59. Pirolli, P.; Pitkow, J.; Rao, R. Proc. ACM SIGCHI Conf. on Human Factors in Computing Systems, ACM Press, New York, 1996
Full-text electronic proceedings at
http://www.acm.org/sigchi/chi96/proceedings/
60. see for example the Excite WWW search engine, http://www.excite.com
61. Rao, R. IEEE Comput. Graphics and Appl. 1997, 17,71
62. Card, S. K.; Robertson, G. G.; Mackinlay, J. D. Proc. ACM SIGCHI Conf. on Human Factors in Computing Systems, ACM Press, New York, 1991, 181
63. Card, S. K.; Robertson, G. G.; York, W. Proc. ACM SIGCHI Conf. on Human Factors in Computing Systems, ACM Press, New York, 1996
Full-text electronic proceedings at
http://www.acm.org/sigchi/chi96/proceedings/
64. Angus, I. G. IEEE Comput. Graphics and Appl. 1996, 16, 3/6
65. Rapley, M. H.; Kennedy, J. B. Interfaces to Database Systems - Lancaster 1994, Sawyer, P., Ed.; Springer: 1995, 143
66. Ahlberg, C.; Williamson, C.; Schneiderman, B. Proc. SIGCHI 92, ACM Press, New York, 1992, 619
67. Caterci, T. IEEE Comput. Graphics and Appl. 1996, 16, 2/63
68. Boyle, J.; Fothergill, J. E.; Gray, P. M. D. Design of a 3D User Interface to a Database in Interfaces to Database Systems - Lancaster 1994, Sawyer, P., Ed.; Springer: 1995, 127
69. Rapley, M. H.; Kennedy, J. B. Three Dimensional Interface for an Object Oriented Database in Interfaces to Database Systems - Lancaster 1994, Sawyer, P., Ed.; Springer: 1995, 143
70. Hemmje, M.; Kunkel, C.; Willet, A. Proc. ACM SIGIR 94, ACM Press, New York, 1994
Full text at
ftp://ftp.darmstadt.gmd.de/pub/VISIT/papers/hemmje/SIGIR94.pssee also: http://www-cui.darmstadt.gmd.de/visit/Activities/Lyberworld/
71. http://este.darmstadt.gmd.de:5000/dimsys/docking_d/home.html
72. Hemm, K.; Aberer, K.; Hendlich, M. Proc. Int. Conf. on Intell. Systems in Mol. Biology 95 (ISMB 95), Cambridge 1995
Full text at
ftp://ftp.darmstadt.gmd.de/pub/dimsys/reports/P-95-13.ps.Zsee also http://www-relibase.darmstadt.gmd.de/gmd/home.html
73. Benford, S. Virtual Environments for Data Sharing and Visualization - Populated Information Terrains in Interfaces to Database Systems - Lancaster 1994, Sawyer, P., Ed.; Springer: 1995, 168
74. Card, S. K. IEEE Comput. Graphics and Appl. 1996, 16, 2/63
75. Spoerri, A. Visual Tools for Information Retrieval, in: Proc. IEEE Symp. on Visual Languages, IEEE CS Press, Los Alamitos, 1993, 160
76. Fayyad, U.; Haussler, D.; Stolorz, P. Comm. ACM 1996, 39, 11/51
77. Becker, B. G. IEEE Comput. Graphics and Appl. 1997, 17, 75
78. Johnson, A.; Fotohui, F. interactions, 1995, 2, 34
see also:
http://evlweb.eecs.uiuc.edu/aej/AndySand.html
79. van Teylingen, R.; Ribarsky, W.; van der Mast, C. IEEE Transactions on Visualization and Comput. Graphics, 1997, 3, 1/65
see also:
http://www.cc.gatech.edu/scivis/projects/ve/ve.html
80. Kouzes, R.; Myers, J. D.; Wulf, W. A. IEEE Computer, 1996, 29, 8/40
81. Pang, A.; Wittenbrink, C. IEEE Comput. Graphics and Appl. 1997, 17, 2/32
82. Casher, O.; Rzepa, H. S. J. Mol.. Graphics 1995, 13, 268
Full text online at
http://www.ch.ic.ac.uk/talks/acs/anaheim.html
83. Casher, O.; Rzepa, H. S. Computer Graphics 1995, 29, 52
Full text online at
http://www.ch.ic.ac.uk/rzepa/CG/CG.html
84. http://www-europe.sgi.com/ChemBio/MolecularInventor/users.html
85. Krieger, J. H.; Chem. Eng. News, 1996, April 22
Full text at
http://pubs.acs.org/hotartcl/cenear/960422/genomic.html
86. http://www-itg.gov/~deba/ALS.DCEE/
87. http://www.emsl.pnl.gov:2080/docs/collab/CollabHome.html
88. http://www.itg.lbl.gov/DCEEpage/DCEE_Overview.html
89. Disz, T. L.; Papka, M. E.; Pellegrino, M.; Stevens, R. Proc. Int. Workshop on High-Performance Computing for Computer Graphics and Visualization, Swansea, 1995
90. Capin, T. K.; Noser, H.; Thalmann, D. IEEE Comput. Graphics and Appl. 1997, 17, 2/42
91. Technology overview at http://www.sdsc.edu/tmf/Sig96Notes/RpForSciVis.html
92. Olson, A. J.; Taylor, P.; Belew, R. K.; Ten Eyck, L.; Bailey, M. Gather/Scatter 1996, 12, 3
see also:
http://www.sdsc.edu/GatherScatter/GSummer96/olson.htm
93. Taylor, R. M.; Robinett, W.; Chi, V. L.; Brooks, F. P.; Wright, W. V.; Williams, R. S.; Snyder, E. J. Proc. SIGGRAPH 93, ACM Press, New York, 1993, 127
see also:
http://www.cs.unc.edu/Research/nano/nanopage.html
94. http://eclectic.ss.uci.edu/~lin/chem.html
95. Gaither, K.; Moorhead, R.; Nations, S.; Foax, D. IEEE Comput. Graphics and Appl. 1997, 17, 1/16
96. Risch, J. S.; May, R. A.; Dowson, S. T. IEEE Comput. Graphics and Appl. 1996, 16, 6/33
97. Cruz-Neira, C.; Leigh, J.; Papka, M.; Barnes, C.; Cohen, S.; Das, S.; Engelman, R.; Hudson, R.; Roy, T.; Siegel, L.; Vasilakis, C.; DeFanti, T. A.; Sandin, D. J. IEEE 1993 Symposium on Research Frontiers in Virtual Reality, 1993, 59
see also:
http://evlweb.eecs.uic.edu/EVL/SHOWCASE/cave/vrpapers/report.html

Figures

1. Protein modeling in a study of Chagas` desease
http://www.tc.cornell.edu/Visualization/Staff/richard/NIH/Faerman/vr14_crop_small.gif on page
http://www.tc.cornell.edu/Visualization/Staff/richard/NIH/Faerman/Chagas.html

2. Manual Docking to HIV protease in the C2 Environment
http://www.icemt.iastate.edu/Projects/C2/gifs/hiv_molecule.gif on page
http://www.icemt.iastate.edu/Projects/C2/html/molecule.html

3. Molecular modeling on the Responsive Workbench
http://viswiz.gmd.de/VMSD/PICTURES/momo3.gif

4. Sculpt modeling session
http://www.intsim.com/ring.gif on page
http://www.intsim.com/sculpt_details.html

5. The VisualSphere LyberWorld widget showing content releationships between 1000 documents
http://www-cui.darmstadt.gmd.de/visit/Activities/Lyberworld/Demos/sphere1000.gifon page
http://www-cui.darmstadt.gmd.de/visit/Activities/Lyberworld/demos.html

6. Scene in the VDV virtual data visualization environment showing a NaCl cluster smashing into a solid neon surface
http://www.cc.gatech.edu/scivis/projects/ve/ve3.gif on page
http://www.cc.gatech.edu/scivis/projects/ve/ve.html

7. Friendship among teens in suburban Dublin
http://eclectic.ss.uci.edu/~lin/kirke.jpg on page
http://eclectic.ss.uci.edu/~lin/chem.html


J.Mol.Model. (electronic publication) - ISSN 0948-5023


Online publication: September 2, 1997
em-helpdesk@springer.de
© Springer Berlin Heidelberg 1997