Teenage Inventor Brings Sign-Translating Glove to NIDCD

March 19, 2002

For George de Mestral, inventor of Velcro, the notion came to him as he removed large prickly burrs from his clothing after a walk in the woods. For Bette Graham, secretary extraordinaire who developed White-Out, a nagging wish to eliminate typing errors and a talent for painting helped spark her idea. And for high schooler Ryan Patterson, inspiration struck one hot August afternoon in the unlikely setting of a fast-food restaurant over an order of burgers and fries.

"I was trying to think of a science fair project to do, and I thought, 'What have I seen over the past year that I can try to improve? What needs to be done?,'" recalled Patterson, an 18-year-old student at Central High School in Grand Junction, Colo. "Then I remembered a time when I was at the same restaurant and saw some people who were deaf who needed an interpreter to help them place their order. I thought I could try to develop an electronic method that would make it easier for people to communicate."

Seven months later and--so what else is new for a busy high school senior?--one day before the Western Colorado Science Fair, Patterson completed a prototype glove that can translate the hand positions employed by people who use American Sign Language (ASL) to fingerspell words in English by converting them into large, easy-to-read letters displayed on a computer screen. To Patterson, a tightly closed fist with the thumb pressed flat and to the side, an outward-facing palm with the thumb neatly tucked in, and five fingers curved in the shape of an incomplete circle can be translated as easily as A-B-C.

Patterson's invention, the "Sign Language Translator," was a Grand Award winner in the 2001 Intel International Science and Engineering Fair, where it was named best of category in engineering, and the first place winner in the individual category at the 2001 Siemens Westinghouse Science & Technology Competition. On Mar. 11, Patterson received top honors and a $100,000 scholarship at Intel's Science Talent Search, a competition often referred to as "the junior Nobel prize."

Patterson's glove offers a new way in which individuals who sign might express themselves during brief, one-sided conversations with people who don't understand sign language. As part of its science education program, the National Institute on Deafness and Other Communication Disorders invited Patterson to Washington last month, where he spent the day demonstrating his invention; trading ideas with scientists, administrators and students participating in the Intramural Research Training Award program; and touring the intramural research facilities to learn about other areas of human communication research. He also met with leadership in extramural research, including the program administrator of NIDCD's Small Business Innovation Research and Small Business Technology Transfer programs, to discuss his future plans and to learn about funding.

"New assistive technologies hold tremendous promise in helping people with communication disorders interact more easily in everyday settings," said Dr. James Battey, director of NIDCD. "We at NIDCD are delighted that a talented and creative young person like Ryan is interested in contributing his skill to challenges of human communication. We hope to do all that we can to nurture the interests of young scientists."

Why Patterson was perhaps the best person to design a translating glove becomes obvious at first meeting. The young man has deep curiosity and enormous technical ability, and has been fascinated with electrical engineering since before he could walk. The stories he tells are mind-boggling, if not just a little bit scary: how at the age of 3 he carried around an electrical cord instead of a blanket; or how he asked Santa to bring him an extension cord for Christmas one year. Or the time when, at 6 years old, he helped his dad rewire the addition to their house; or the less favorable times in which he felt the surge of 110 volts of electricity whenever he accidentally stuck a screwdriver into an outlet.

"I lived," he said with a laugh--half in amazement, half in defense of his early attempts to tackle the formidable learning curve that awaited him.

By the third grade, when neither his teachers nor his parents could answer his questions--his father is a foreman at a metal fabricating plant; his mother is a kindergarten teacher's aide--he was introduced to John McConnell, a retired particle accelerator physicist from Los Alamos who had recently moved to the area. McConnell, whom Patterson respectfully refers to as "my mentor," worked with Patterson every Saturday from 9 a.m. to 5 p.m. for the next 7 years, teaching him how to build circuit boards, read schematics and design electronic circuitry. McConnell and his wife Audrey, who helped in the stitching of the prototype glove, have spent so much time with Patterson during his growing-up years, they consider themselves his "third set of grandparents."

"It's important for retirees to reach out to kids and teach them what we know," said McConnell, who directs a math and science center for K-12 students and teachers in Grand Junction. McConnell estimates that approximately 1,000 children take part in center activities each month. "We give them wings so they can fly on their own. Now, Ryan's doing things that I can't," he noted proudly. Patterson's past award-winning science fair projects have included an autonomous robot and Sleuthbot, a computer-controlled search robot.

Once Patterson settled upon the translating glove for his project, he began learning as much as he could about the deaf community in general, and sign language in particular. A visit to the NIDCD web site provided him with enough background information to determine that the invention might indeed be useful for deaf people as well as other individuals who have difficulty communicating such as people who have had strokes, throat cancer, speech-impairment and possibly individuals with cerebral palsy who use ASL.

Patterson set a number of goals for his translating glove, the first being accuracy. "I know how annoying it is when I'm using speech recognition software and the computer doesn't recognize what I'm saying," he said. His other goals were cost-effectiveness, comfort, energy-efficiency and ruggedness.

The Sign Language Translator consists of two separate components: a leather-thin golf glove, which tells the computer the precise position of the hand, and the computer, which is programmed to associate each hand position with a corresponding letter that, in turn, flashes up on its screen. Ten flexible sensors sewn into the glove monitor the position of the fingers by measuring the electrical resistance that is created by the fingers as they bend. A small microcontroller on the back of the hand converts the change in the electrical current into the 1's and 0's that the computer can recognize and ships the information to the laptop via a short rubberized antenna jutting from the microcontroller. The computer then reads the numerical values and, in a fraction of a second, converts them into the letters of the alphabet, which appear on the screen.

With so much numerical precision involved, one might wonder if individual differences, such as a larger hand size or a slightly different position of the fingers, might throw the computer off. But Patterson took care of that. Before a glove wearer signs her first word, she trains the device to recognize her own hand positions by forming each letter of the alphabet and setting the new "customized" values with the tap of a button.

"I find it interesting that the sensors involve deflection, because that's how our inner ear works," remarked Battey, referring to the hair cells found in the inner ear. Sound vibrations cause hairlike bundles at the top of the hair cells to bend, kicking off a chain of events that generates an electrical signal, which travels to the brain. "There are actually a lot of similarities between how the glove works and how the inner ear works," he said.

Although portability was not one of Patterson's top priorities, it became an obvious concern--after all, who wants to carry a laptop around all the time?--so he also developed a scaled-down version of the device. He replaced the laptop with a small wireless receiver and display unit roughly the size of a TV remote control. Two small microcontrollers perform the translation.

Even with the accolades, Patterson has many improvements in mind for his bilingual glove. Already, he has incorporated speech dictation software, so that a computerized voice says the word aloud after it has been spelled. To increase translation speed, he is attempting to fit all of the computing hardware onto the glove itself. He also plans to replace the first microcontroller with a powerful math processor so that a full range of hand motions--not just stationary hand positions--can be translated. And he'd like to redesign the glove so that all of the machinery is hidden from view. Currently, he has obtained a provisional patent on the glove and plans to apply for a full patent later this spring.

"The sign-translating glove is an interesting and effective way to use a computer to communicate," said Lori Ingram, personnel assistant at NIDCD, who attended one of Patterson's many "hands-on" demonstrations. Ingram, who is deaf, said that she appreciates the glove's ability to capture finger-spelling, and looks forward to a day when it can truly capture ASL.

Patterson concedes that, despite its name, the glove can read only one letter at a time and is not able to translate true sign language, which includes a full range of signs--face, hands and arms--such as those employed in ASL. The latter, he says, would require a lot more hardware and computational ability. Portability also would most likely be compromised. For the time being, his goal is to construct his translating glove so that it works as well as, if not better than, writing a message with a pen and paper.

"Of all the projects I've done so far, this has been the most interesting because of the interface between electronics and people," said Patterson. "If I can make an innovative device that could help people out--particularly people with disabilities--I'll feel as though I've made a difference."

Top