Clarification and Correction

I recently got interviewed by loQal about my research. You can read the article here. I guess I didn't explain some things clearly enough and I'd hate to give a false impression regarding the research. Here are some items I'd like to clarify or correct.

• The Filipino Sign Language (FSL) Archive project is a collaboration between:
  1. Philippine Deaf Resource Center (PDRC) - an NGO
  2. Philippine Federation of the Deaf (PFD) - an NGO
  3. Digital Signal Processing (DSP) Lab of the Electrical and Electronics Engineering Institute
  4. Computer Vision and Machine Intelligence Group (CVMIG) of the Department of Computer Science

  DSP and CVMIG are both of the College of Engineering, UP Diliman.

• The FSL Archive Project is a separate project from the Filipino Speech Corpus (PSC) project. For one thing, Sign is not Speech.
• As far as I know, the linguistics research is being done by PDRC and PFD, not UP.
• I don't have an application or system yet that can convert FSL into text. That is a long way off. What I have are experimental programs. Nothing practical. Also; syntax and semantics of FSL is currently poorly understood. Until we get a better handle on that, FSL to text sentences is not possible.
• FSL vs ASL (vs SEE vs MCE). It cannot be denied that American Sign Language (ASL), Manually Coded English (MCE) and Signing Exact English (SEE) has a huge influence on FSL; however, many Filipino Deaf refer to their language as Filipino Sign Language. This is a social, cultural and political issue in addition to a technical issue. For example, the Deaf I met in Cebu called their sign language Cebu Sign Language. And yes, there is a lot of variation between regions, and provinces.
• FSL vs English (vs Tagalog). This one confuses a lot of people. Sign is not Speech. FSL is not English. FSL is not Tagalog. FSL is a separate, distinct language. It helps if you think of Written English as a separate language from Spoken English. There is no equivalent "Written FSL". To facilitate research, signs are assigned a label called a GLOSS. It is a word or phrase borrowed from another language. Since many Deaf in the Philippines have Written English as a second language, the GLOSS is borrowed from Written English. It is often written in ALL CAPS to distinguish it from Written English (example: THINK-SKIP-MIND). Note that while the GLOSS is chosen to be as close to the meaning of the sign as possible, this is not a translation. This is one reason why you sometimes see Tagalog used as a GLOSS (example: LOLA).

I think that covers most of it. If you have more questions, leave a comment. Thanks for reading!

Difficulties in Sign Recognition

What makes it hard to do Sign Recognition? I touched this topic briefly in an earlier post. Simply put, there are a lot of things going on in sign language. In spoken languages, you just have to listen to one thing; in sign language you pay attention to the face, the body, the hands and arms simultaneously.

Another part of the problem is the complexity of sign language itself. Facial expressions, and body posture are part of the language. Some form of facial recognition and expression detection is needed (although I ignore this in my research, a topic for another post). The signs themselves vary when used in a sentence, much like the sounds of words change slightly when spoken in different sentences, and in different contexts.

Variation is another source of problems. Each individual performs the sign differently, similar how different people sound different in spoken languages. Even from the same individual, the signs vary slightly when done at different times. And top it off with regional and local variations of the same sign. This is one reason why I restricted my research to signs used in Metro Manila; if I didn't I'll never finish.

The other source of difficulty is general difficulty of computer vision. How do you tell which is the background vs the foreground? How do you distinguish several people in one image/video? Humans have an incredible ability to figure out faces and postures even when viewing from the side, how do we duplicate this ability in computers? To reduce these issues, I recorded one person signing wearing a plain black shirt in front of a plain black background.

Bibliography

This is a partial dump of the references I have used so far.

• Rafaelito M. Abat and Liza B. Martinez. The history of sign language in the philippines: Piecing together the puzzle. In 9th Philippine Linguistics Congress, Diliman, Quezon City, Philippines, 2006.
• Julius Andrada and Raphael Domingo. Key findings for language planning from the national sign language committee (status report on the use of sign language in the philippines). In 9th Philippine Linguistics Congress, Diliman, Quezon City, Philippines, 2006.
• Yvette S. Apurado and Rommel L. Agravante. The phonology and regional variation of filipino sign language: Considerations for language policy. In 9th Philippine Linguistics Congress, Diliman, Quezon City, Philippines, 2006.
• Robin Battison. Lexical Borrowing in American Sign Language. Linstok Press, Silver Spring, MD, 1978.
• Marie Therese A.P. Bustos and Rowella B. Tanjusay. Filipino sign language in deaf education: Deaf and hearing perspectives. In 9th Philippine Linguistics Congress, Diliman, Quezon City, Philippines, 2006.
• Phil. Deaf Resource Center and Phil. Federation of the Deaf. Part 1: Understanding Structure. An Introduction to Filipino Sign Language. Phil. Deaf Resource Center, 2004.
• Phil. Deaf Resource Center and Phil. Federation of the Deaf. Part 2: Traditional and Emerging Signs. An Introduction to Filipino Sign Language. Phil. Deaf Resource Center, 2004.
• Heeyoul Choi, Brandon Paulson, and Tracy Hammond. Gesture recognition based on manifold learning. Structural, Syntactic, and Statistical Pattern Recognition, 5342:247–256, December 2008.
• Philippe Dreuw, Carol Neidle, Vassilis Athitsos, Stan Sclaroff, and Hermann Ney. Benchmark databases for video-based automatic sign language recognition. In International Conference on Language Resources and Evaluation, Marrakech, Morocco, May 2008. http://www-i6.informatik.rwth-aachen.de/~dreuw/ database.php.
• Raymond G. Gordon Jr., editor. Ethnologue: Languages of the World, 15th ed. SIL International, Dallas, Texas, 2005. http://www.ethnologue.com/.
• Sushmita Mitra and Tinku Acharya. Gesture recognition: A survey. IEEE Trans. Systems, Man & Cybernetics, 37(3):311–323, May 2007.
• Phil. National Statistics Office. Persons with disability comprised 1.23 percent of the total population. Special Release No. 150, March 2005. http://www.census.gov.ph/data/sectordata/sr05150tx.html.
• Sylvie C.W. Ong and Surendra Ranganath. Automatic sign language analysis: A survey and the future beyond lexical meaning. IEEE Trans. Pattern Analysis & Machine Intelligence, 27(6):873–891, June 2005.
• World Health Organization. Deafness and hearing impairment. Fact Sheet N300, March 2006. http://www.who.int/mediacentre/factsheets/fs300/ en/index.html.
• Sam T. Roweis and Lawrence K. Saul. Think globally, fit locally: Unsupervised learning of low dimensional manifolds. Science, 290(5500):2323–2326, December 2000.
• Joshua B. Tenenbaum, Vin de Silva, and John C. Langford. A global geometric framework for nonlinear dimensionality reduction. Science, 290(5500):2319–2323, December 2000. http://waldron.stanford.edu/~isomap/.
• Christian Philipp Vogler. American Sign Language Recognition: Reducing the Complexity of the Task with Phoneme-Based Modeling and Parallel Hidden Markov Models. PhD thesis, University of Pennsylvania, 2003.