Thumbs up! Taiwan team develops sign language translator

Taiwan Yingmi Technology develops glove that translates sign language

Ben Chen, the Project Manager of Taiwan Yingmi Technology demonstrates sign language translator.

Ben Chen, the Project Manager of Taiwan Yingmi Technology demonstrates sign language translator. (By Taiwan News)

TAIPEI (Taiwan News)—Taiwan Yingmi Technology (台灣盈米科技股份有限公司), wowed the crowds at the Computex global trade show for information communication technologies that took place from May 29 to June 3 in Taipei City, with its new sign language translator that enabled the hearing impaired to communicate in a new way.

"It is very difficult for deaf people to find jobs in Taiwan because people cannot understand hand signs, and we wanted to help them by developing a product that could assist their communications to others,” said Ben Chen (陳國瑋), the Project Manager of Taiwan Yingmi Technology.

The sign language translator that took two years to develop is a black glove made from synthetic fabric that the deaf can wear to communicate to the general public.

Finger sensors, gyrators and accelerators built onto the wrist of the glove can transfer large amounts of hand gesture data onto a smartphone app using Bluetooth technology.

The app then enables the smartphone to read aloud the hand sign meaning, enabling those unfamiliar with sign language to understand what the deaf are conveying.

Most of the hand sign analyses are processed on a cloud server that the company developed in-house, which incorporates deep learning artificial intelligence (AI) to help fine tune the analysis, said Chen.

"There are 10 joints in each hand," explained Chen. "Relying on the smartphone computing capacity alone is insufficient, since 60 data points are being transferred from the device each second."

The company's hand sign translator glove was manufactured in partnership with National Taiwan University of Science (NTUS) and Technology and Taiwan Textile Research Institute (TTRI).

Developer kits of the hand sign translation gloves are being sold to research institutes in small volumes, and the company is charging a license fee for the usage of the glove's software development kit (SDK).

Asked about future plans, the company is aiming to bring the precise hand gesture detection technology it has developed to Virtual Reality (VR) game console controllers and later cater to the education market.

"The glove has a 90 percent accuracy rate for hand gesture detection," said Chen.

Current VR controllers are clunky and users cannot feel the texture of items or grab it in the same way as in real life, which is why the company hopes it can gain a competitive edge with its light weight glove developed by partner TTRI.

As for educational purposes, the company has documented the hand gesture movements of 90-year-old glove puppet master Chen Xihuang (陳錫煌), and digitized the traditional art form.

"We think digitizing Chen's hand gestures could help preserve our traditional culture since it can become training material for the younger generation of glove puppet apprentices," said Chen.