An Adaptive, Affordable, Humanlike Arm Hand System for Deaf and Deaf Blind Communication with the American Sign Language

To communicate, the ∼1.5 million Americans living with deafblindess use tactile American Sign Language (t-ASL). To provide DeafBlind (DB) individuals with a means of using their primary communication language without the use of an interpreter, we developed an assistive technology to promote their autonomy. The TATUM (Tactile ASL Translational User Mechanism) anthropomorphic arm hand system leverages previous developments of a fingerspelling hand to sign more complex ASL words and phrases. The TATUM hand-wrist system is integrated into a 4 DOF robot arm and a human motion recognition and human to robot gesture transfer framework is used for signing recognition and replication. In particular, signing trajectories based on vision-based motion capture data from a sign demonstrator were used to control the robot’s actuators. The performance of the system was evaluated through contact pressure applied by a blinded user and for its accuracy with novice, sighted users.