Sign Language Recognition - Vision

The other common approach to sign language recognition is Vision-based. Video footage of subjects signing are fed into the computer either in real-time or as a video file recorded earlier.


  • Mainstream hardware. Video cameras are easier to obtain, configure and control than the CyberGlove.
  • Closer to natural. Signers don't wear any special equipment that may impede in their signing.
  • Easier to productionize. No (extra) special equipment needed in applications.


  • Sensitive to lighting conditions, and background noise.
  • Occlusion is a problem. In certain positions, the hands and arms will obscure the face or the other hand/arm.
  • Not as accurate as Direct Measure and it depends on the camera resolution.

If it wasn't obvious already, I'll be going with the Vision-based approach.


Sign Language Recognition - Direct Measure

There are two general approaches to sign language recognition in terms of how the computer "sees". Direct Measure approaches rely on devices that sense the position of the fingers, thumb, arms, and so on. The CyberGlove is one such measuring device. The subject wears the glove and when the subject moves, the glove relays the information from sensors built-in the glove to a computer or other recording device.


  • Exact measurements of positions, angles, velocities of the fingers, thumb, arms and so on.
  • It is unaffected by lighting conditions, static/dynamic backgrounds, color and pattern of clothes worn by the subject, and subject skin tones.
  • No obstruction problems. It doesn't matter if one hand is in front of another.


  • Cumbersome in non-laboratory settings.
  • Artificial; most people don't wear gloves in their day-to-day activities (at least here in the tropics).
  • Gloves bring additional equipment costs if we are creating practical applications of sign language recognition.


video recording update

Update: changed photo URLs to point to Flickr

After 4 sessions, we now have 80 signs, with each sign performed by two signers. Many thanks to my models Rommel and Mary Jane.

FSL-01 Rommel waits patiently while we get the lights going.

A reflector was placed in front of the signers (supported by two chairs) to bounce the light up and soften the shadows in the face. We had two lights, one on the left and one on the right, facing the signer approximately 180 cm away.

Mary Jane and Rommel discuss the signs to be performed.
When Rommel is seated in front of the camera, Mary Jane is seated at the behind the camera to prompt Rommel which sign to perform next. Then they switch places after one set of signs have been performed. The signs were arranged into groups 10.

Next: Really cheap lights