Haptics, derived from the Greek "haptikos" meaning "to touch", is divided into two major categories of sensory information: tactile and kinesthetic. Tactile sense information include temperature, skin curvature and stretch, velocity, vibration, slip, pressure and local force. It tells us about the initial contact with objects, surface textures and geometry. Kinesthetic information relates the physical forces applied to and by the body. This includes proprioception or sensing and awareness of body position. Kinesthetic rendering is often called force-feedback or force-reflection. Motion platforms, including walking simulators, are also a form of haptic rendering, acting on the vestibule system as well as other force sensing. They were covered in this column in April 2000 and will not be discussed this month.

Haptics has been a part of VR research for many years, but the costs of building devices and learning the control algorithms greatly limited its application. Most VR systems do not include haptics. Unfortunately, no matter how good the visual and auditory rendering may be, all pretenses to reality is shattered the moment you pass through an object without feeling it. Changing object colors or triggering audio tones (sensory substitution or ghosting) just is not the same. The ability to feel the environment greatly enhances the quality of the experience. However physical limitations and computational complexity still make it the hardest aspect of the elusive Holodeck to realize.

Quality commercial products have become available lately at lower cost, making research easier and commercial applications cost effective. This has helped fuel in an increase in haptic events. The 1^st International Haptics Symposium was held last summer in Glasgow. The two day event (Aug 1/Sept 2 2000) had quite a few papers, posters and demos. Many of these are available at the conference web site. The University of Southern California sponsored a Haptics workshop in February 2001, which I was fortunate to attend. They had several demos and 17 presentations covering networking, collaboration, human factors, physiology, devices, APIs and applications. Unfortunately, the presentations were not collected into a proceedings. Some of the presentations were precursors to full papers to be given at conferences this year. IEEE VR had three sessions dedicated to Haptics, along with several posters . The proceedings should be available in the IEEE digital library later this year. The Tenth Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems will held next year March 24 & 25, 2002 in conjunction with IEEE VR 2002 in Orlando Florida USA

Prof. Bob Stone, Scientific Director of Virtual Presence, (err Muse, no its part of AVS now) keynote talk at the Glasgow haptics symposium provides a good history of Haptics in VR, with a bit of a euro-bias. (Bob is a member of the Advisory Group for VRNews Community Forums) An excellent reference on the physiology and technology of haptics can be found in the 1996 "Review of Virtual Environment Interface Technology" from the Institute for Defense Analysis (IDA Paper P-3186). (It’s a great all round, easily accessible reference on most of the physiology for VR). Grigore Burdea, director of Rutgers Human-Machine Interface Lab, wrote the 1996 Haptics reference book, "Force and Touch Feedback for Virtual Reality" (ISBN 0-471-02141-5. These both provide good background material, however they are five years old and most of the commercial devices they mention are no longer available. The Haptics Community Web Page offers a wealth of on-line information and links to commercial and research sites. There is an extensive library of article links and an impressive gallery of haptic devices. Being an on-line site, it is a bit more up to date than the IDA paper, although it does have its share of link-rot. Haptics-L is an electronic mailing list for the international haptics community. The list is not very active but does carry good conference announcements, etc. Another online resource is Haptics-e "The Electronic Journal of Haptics Research" , however, the latest papers are from March 2000.

Tactile Rendering

The physiology of tactile sensation is complex. The IDA paper discusses "four kinds of sensory organs in the hairless skin of the hand that the sense of touch", each with different thresholds and modes of excitement. Temperature and spatial details are difficult to accurately reproduce.

Vibration is often substituted for other tactile senses and requires minimal processing overhead. It is fairly easy to create safe levels of vibration allowing many types of devices to be augmented. Virtual Technology’s CyberTouch glove provides vibration rendering for each finger tip. Vibrators can be triggered when the virtual hand contacts objects, providing some sense of physical presence. Antrotronix developed an arm sleeve containing 5 pager-type vibrators for use in a Situation Awareness application. The Logitech iFeel mouse uses vibrations generated by the oscillations and accelerations of an Inertial Harmonic Drive motor controlled by Immersion Corporation's TouchSense™. This brings tactile feedback to the PC desktop and web pages. You can feel when the mouse moves into a scroll bar, over icons, across window borders, etc. I'm not so sure you will be able to "visit an e-commerce web site where you could experience the texture of fabrics, finishes, and lotions." (http://www.immersion.com/desktop.html). Perception of such fine textures uses sensing of other tactile aspects like skin curvature, slip and stretch. While these vibration devices can be effective alternative communication modes, they are sensory substitutes and do not provide accurate renderings of tactile environment.

Sound is a form of vibration and several products take advantage of this to provide simple tactile rendering. Thunderseat markets a chair with large audio transducers to give vibrations as you fly your MS Flight Simulator. Clark Synthesis "Tactile Sound Transducers" are specialized devices for 15Hz-800Hz sound. They have been effectively used in a variety of military simulators and theme parks as well as home theatre installations.

One approach to rendering the spatial aspects of tactile sense is the use of pin arrays - fingertip sized array of small pins that can be moved individually. A number of such projects are discussed in the IDA paper. Y. Ikei, et al from the Tokyo Metropolitan Institute of Technology presented a recent updating of the pin array approach in a paper at IEEE VR2001. Another paper at that conference by T. Nara, et al at the University of Tokyo discusses the use of a Surface Acoustic Wave (SAW) Device for tactile display. H. Kajimoto, also from the Tachi Lab at U. Tokyo, presented a poster on the direct stimulation of the tactile nerves using electrical currents at the conference.

Kinesthetic Rendering

Newton’s basic laws of physics present a major hurdle for Kinesthetic rendering in virtual worlds. A haptic device must have a base to push against to exert a force on the viewer. The physics of force fields and tractor beams is still speculative fiction.

Fixed control devices such as steering wheels and joysticks can be fairly easily modified to support force feedback. There are a number of such devices available for high quality simulators. There are still a few lower cost consumer devices, although many have left the market several since our last Haptics column. The main survivors are force members of the Logitech Wingman and Microsoft Sidewinder families of game devices. The feedback on these devices is often limited to "effects" and may not be an accurate rendering of a physical simulation. The approximation can work well in many scenarios.

A device is for true simulation is harder to build, requireing a larger working volume. Forces may be rendered in 3D (translation only) or full 6 degrees of freedom (translation and rotation). One common approach uses a stylus mounted on the end of a sophisticated robotic arm to provides both tracking and force feedback. The PHANToM from Sensable Technologies is probably the most commercially successful device. Sensable makes several different models from the nicely designed 3D desktop version to a full 6DOF version for more demanding applications. ReachIn Technologies created an interesting 3d workstation by integrated the Sensable PHANToM Desktop under a mirror reflecting a stereoscopic monitor. The stylus appears in the virtual world roughly where the real one is under the mirror. This gives a good extra sense of realism. MPB Technology offers an alternative stylus based haptic device, the Freedom 6s, which provides a 6DOF rendering.

Virtual Technologies (now part of Immersion Inc.) has the CyberForce, a force-feedback armature which doubles as a position tracker. It can be combined with their exoskeleton CyberGrasp glove which provides resistance force to each finger. Exoskeleton devices used to be more prevelant, but this is the only commercial one left.

Robot arms are also used in encounter type devices to simulate touching multiple physical objects. The robot moves to represent different objects in response to the tracked movements of the users hands. Y Yokokohji of the Yoshikawa lab at Kyoto University presented a paper on path planning for such a device at VR2001.

Another method of force rendering uses strings connected to fingers or a stylus. The Virtual Technology Cybergrasp is an example that uses very short wires to deliver force on the fingers. Researchers at the Intelligent Cooperative Systems Laboratory at the University of Tokyo have developed HapticGear, a backpack that uses wire tension to provide haptic feedback in an Immersive Projection Display (IPD). A paper discussing its development was presented at VR 2001

All of these kinesthetic devices use motors and active force generation. Such devices can pose a hazard if they move uncontrollably. Indeed it is not difficult for errors to cause haptic simulations to become unstable. A paper at IEEE VR 2001 from M. Sakaguchi et al of Osaka University presented a passive force display. Their experimental device uses Electroheological (ER) fluid brakes. ER fluid changes its apparent viscosity when an electrical field is applied.

Programming Haptics

The rendering devices are but one aspect of haptic technology. Supporting haptics in the virtual world requires a fair amount of extra computation and design. The location of the user must be accurately measured and simulated in the virtual world. Surface textures may need to be modeled and geometry needs to be accurate. Detailed collision detection at all points for which rendering is provided must be calculated. Then collision reactions must be formulated. This may require some extensive force calculations. Finally the output device must be controlled and monitored to deliver accurate, stable forces and avoid injury. These are non-trivial problems and accurate rendering depends as much on the software as the rendering device. The complexity of haptic simulation often requires an additional processing system. Many haptic applications utilize dual processor systems or networks of computers.

There are several APIs available to help the VR developer implement haptic rendering. Vendor systems such as VRPN (Virtual Reality Peripheral Network) supports a wide assortment of VR interface devices, including the Sensable PHANToM. Novint's e-Touch is a new comer providing a complete VR API (graphics by OpenGL) with first class support for haptics. While the core of e-Touch is proprietary, large parts of it are exposed through the extensible Open Module system. Source code for Open Modules is freely available for research applications, although commercial users will require licenses. Contribtors of e-Touch extensions can earn royalties on commercial licenses of their contributions. Novint expects to release the product on April 15, 2001 and has announced a contest for the best Open Module contribution with the top prize being a Sensable Phantom.

Applications

Sensable's Freeform haptic modeling application is one of the few successful VR applications in use today outside of research. Talented sculptors and model makers with no computer skills have quickly learned the system. Product design, ceramic, animation and other companies have found the system quite useful.

Haptic simulation is finding a major market in medical applications. Simulators for minimally invasive surgery and similar procedures can greatly increase the skills of a physician. Since surgeons depend heavily on touch, haptic rendering is an essential part of these systems and the new generation of telerobotic surgery devices. The April 2001 issue of MIT Tech Review has an USC Interactive Art Museum at the Fisher Gallery has been exploring the use of haptics for their expositions. This can allow visitors to virtually touch objects that are normally off limits. The reference paper at the web site has an excellent bibliography of haptics information.

Column Info