Projects: A tactile glove for computer graphics for the blind

This project uses electrical stimulation of touch to display computer graphics to the fingertips of the user. The electrodes are attached to the hands and update their position based on the position of the hand on a computer monitor. See the final report for details.

National Science Foundation (NSF)
Grant number IIS-0083347
Project period 3/15/2001—6/30/08
Principal Investigator: Paul Bach-y-Rita

Final report for entire project period (.doc)

Project summary from NSF application:

Access to computer graphics and graphical user interfaces (GUI’s) by blind and visually impaired persons has always been a problem. As technology has advanced, and the use of web-based, graphically-rich information sources has skyrocketed, both new and previously very competent blind computer users have been excluded from equitable access to resources on the digital highway.

This project, which is based on our long-term studies of late brain plasticity and tactile sensory substitution, is intended to: demonstrate the feasibility of delivering patterned computer-graphic information for haptic scanning by using a fingertip-mounted optical-to-tactile conversion system; to explore perceptual and sensorimotor issues related to the acquisition of two- and three dimensional images by blind persons, and to explore the potential utility of such a system as both an educational tool and a practical device for everyday use by blind and visually impaired users.

The completed system will have a tethered glove, which the blind person puts on one hand, which can be freely scanned across a large flat computer LCD monitor (mounted facing up in front of him/her). The graphics displayed on the monitor will be perceived by spreading the fingers to cover a larger part of the image, or pulling them together for exploration of a more complex image. The blind user will be able to control the size of the image (using zoom/loom) and the characteristics of the display, such as edge-enhancement, black-white reversal, and other characteristics of the image. Characterization of these control features will be part of the software development effort.

We will develop an optical interface to a commercially available tactile display glove. Performance analysis will be performed to explore the specific operating characteristics of both hardware and software for a future version of such a device that would be affordable for any school serving the educational needs of visually impaired students. Perceptual and sensorimotor studies will be designed to obtain scientifically valid evidence of sensory information transport. We hypothesize that not only will these forms of graphical information be accessible with such a device, but that that the wealth of information potentially available from such images exceed that achievable by any other means presently available.

We envision a system in the future that will allow a visually impaired user to haptically explore almost any video-graphic information, from a pie chart shown on a desktop CRT display in a 3rd grade classroom, to a complex molecular structure displayed on students laptop LCD in a biochemistry lecture that is downlinked with the instructors’ PC. The program described herein is designed to yield results that we believe will lead to the development of a useful system for tactile access to visual graphics.