We would like to get 150,000 signatures within this month to pass two laws:
Learn more about the House Bills and sign our petition list at http://housebills4deaf.webs.com/
To all hearing interpreters , sign language enthusiasts, anybody interested to be sign language interpreters; to all deaf interpreters and deaf individuals:
There will be a general assembly meeting for sign language interpreters
Date: 2011 May 21 (Saturday)
Time: 7:30 am - 3:00 pm
Venue:
CAP Development center
7th floor, CAP Building
126 Amorsolo corner VA Rufino
Legaspi Village, Makati
If you are from outside Manila and cannot come to Makati, the assembly will be webcast, e-mail philnasli@yahoo.com to get the instructions. Place in the subject "SLI webcast".
You should have a Yahoo account (http://messenger.yahoo.com/) or a Skype account (http://www.skype.com/intl/en/home) to be able to view the webcast.
I recently got interviewed by loQal about my research. You can read the article here. I guess I didn't explain some things clearly enough and I'd hate to give a false impression regarding the research. Here are some items I'd like to clarify or correct.
DSP and CVMIG are both of the College of Engineering, UP Diliman.
I think that covers most of it. If you have more questions, leave a comment. Thanks for reading!
What makes it hard to do Sign Recognition? I touched this topic briefly in an earlier post. Simply put, there are a lot of things going on in sign language. In spoken languages, you just have to listen to one thing; in sign language you pay attention to the face, the body, the hands and arms simultaneously.
Another part of the problem is the complexity of sign language itself. Facial expressions, and body posture are part of the language. Some form of facial recognition and expression detection is needed (although I ignore this in my research, a topic for another post). The signs themselves vary when used in a sentence, much like the sounds of words change slightly when spoken in different sentences, and in different contexts.
Variation is another source of problems. Each individual performs the sign differently, similar how different people sound different in spoken languages. Even from the same individual, the signs vary slightly when done at different times. And top it off with regional and local variations of the same sign. This is one reason why I restricted my research to signs used in Metro Manila; if I didn't I'll never finish.
The other source of difficulty is general difficulty of computer vision. How do you tell which is the background vs the foreground? How do you distinguish several people in one image/video? Humans have an incredible ability to figure out faces and postures even when viewing from the side, how do we duplicate this ability in computers? To reduce these issues, I recorded one person signing wearing a plain black shirt in front of a plain black background.
FSL video progress
Proposal/Thesis Paper writing
Challenges in Sign Language Recogniton
Multiple types of events
From an earlier post, I mentioned that sign language has several things going on. We need to look at the hand shapes, the arm motion with respect to the body, the body posture, and facial expressions; each one a different problem. For example, recognizing facial expressions require a different sort of processing than tracking arm motions.
Multiple channels
We can think of a channel as "one thing to keep an eye on". In sign language we have at least three channels: the left arm, the right arm and the face. Thankfully, we only have a limited set of combinations with types of events. For example, hands are found on the ends of the arms. If we can track the arms, we can easily find the hands. In general, we will never find a head on the end of an arm.
Multiple Interpretations
This is where things get messy. When you see a gesture, it could be one of three things: (a) a lexical(?) sign, (b) a (cultural) gesture or (c) visual action. (I am not sure if lexical is the correct term to use.)
For example, let's say the signer taps the side of their head with their index finger. Does that mean (a) the sign for THINKING; or does it mean (b) "he's crazy"; or is it (c) acting out "something long and thin poked my head"?
Sign Language linguistics
On top of all of that, we have sign language linguistics. How can we tell when one sign ends and another begins? Unfortunately, we don't have a clear model for Filipino Sign Language at the sentence and discourse levels, which means that automatic, real-time translation is not possible at the moment.
It is time to re-evaluate our progress. What have we accomplished? What have we learned?
Accomplishments
mencoder options for my data files. mencoder is part of the MPlayer project.
Non-Accomplishments
The other common approach to sign language recognition is Vision-based. Video footage of subjects signing are fed into the computer either in real-time or as a video file recorded earlier.
Advantages
Disadvantages
If it wasn't obvious already, I'll be going with the Vision-based approach.
There are two general approaches to sign language recognition in terms of how the computer "sees". Direct Measure approaches rely on devices that sense the position of the fingers, thumb, arms, and so on. The CyberGlove is one such measuring device. The subject wears the glove and when the subject moves, the glove relays the information from sensors built-in the glove to a computer or other recording device.
Advantages
Disadvantages
Update: changed photo URLs to point to Flickr
After 4 sessions, we now have 80 signs, with each sign performed by two signers. Many thanks to my models Rommel and Mary Jane.
A reflector was placed in front of the signers (supported by two chairs) to bounce the light up and soften the shadows in the face. We had two lights, one on the left and one on the right, facing the signer approximately 180 cm away.
Mary Jane and Rommel discuss the signs to be performed.
When Rommel is seated in front of the camera, Mary Jane is seated at the behind the camera to prompt Rommel which sign to perform next. Then they switch places after one set of signs have been performed. The signs were arranged into groups 10.
Next: Really cheap lights
In order to teach the machine FSL, video recordings of native signers are needed. For the first (pilot) stage, 50 1-handed (1H) signs and 57 2-handed (2H) signs will be recorded. Only traditional signs were selected. Each sign will be recorded in isolation; hands and arms start at a neutral position and will return to the neutral position after each sign. For comfort, the signer will be seated. To simplify processing, signers will wear black short-sleeved shirts and will be seated in front of a plain black background. Each video will be shot (and probably cropped) to include the whole signing space and little bit of space on the sides. The picture shows the neutral position.
In addition, 96 hand shapes will we recorded in isolation. Only the hand and a bit of the forearm will be included in the shot. Each hand shape will be recorded moving through six palm orientations.
The signs are grouped according to their basic hand shapes: Solid, Plane, Arc, Line and Other. Solid hand shapes are formed by forming a fist (or similar to it); Plane hand shapes have all (or most) fingers extended; Arc hand shapes have the fingers curved (round); Line hand shapes have one (or two) fingers selected and extended; Other is for everything else. Signs belong to one group; the groups are mutually exclusive. Note that this grouping is more for organizational purposes and nothing else; several hand shapes can be argued to belong to two (or more) groups .
This is not meant to be a comprehensive recording of FSL but rather a starting point for the research. A comprehensive collection of FSL recordings are another project.
Signing space is a three-dimensional space from about the mid-torso to just above the head, extending forward from the chest to about one-arm length away, and extending about half an arm's length on both sides. During most signs, the hands and arms do not go beyond this space.
Fingerspelling space is a small space just large enough to fit the hand; it is located midway near the chin and shoulder. Note that the fingerspelling space is still located within the signing space. Fingerspelling space is where the hands are used to spell out letters borrowed from a written language.
One or both hands may be used in signing, depending on the sign and the sign language. In the case where two-hands are used where only one hand is moving, the moving hand is called the dominant hand (DH) and the stationary hand is called the non-dominant hand (NDH) or the passive hand. Two-handed signs where both hands move in the same path and use the same handshapes are sometimes called symmetrical signs.
There are no left-handed or right-handed signs, one-handed signs may be performed with either left hand or right hand; and either hand may be used as the dominant hand in two-handed signs. In practice, right-handed people usually use their right hand for one-handed signs, fingerspelling, and as the DH in two-handed signs; and left-handed people usually use their left.
Update: Part 2 here
Sign Language is the natural language of the Deaf. It is a visual language and those who use sign language are called signers. Signers use their hands, shoulders, arms, torso, neck and face to communicate. In spoken languages, the basic unit of sound utterance is called a phonemes. Similarly, the basic unit of sign language are also called phonemes even though they are not based on sound.
The Liddel and Johnson model Sign language has five parameters that describe phonemes:
Initial inventory of Filipino Sign Language (FSL) observed over ninety handshapes, approximately twenty locations, and six orientations. Movement can be grouped generally into two categories: gross arm movement (tracking the path of the hand and arm) and internal movement (changes in hand shape).
Liddell and Johnson further grouped these into segments; a Movement segment (M) and a Hold segment (H). Movement segments are portions of the sign where the hands (and arms) are motion or the hand shape is in transition. Hold segments are portions of the sign where there is no motion or where hand shapes are in steady state. Signs are then composed of one or more segments. For example, HMH means there is a Hold segment followed by a Movement segment followed by a Hold segment.
Segments observed in FSL include H, M, MH, HMH, and MHMH.