Computer-assisted teaching of Sign Language using Computer Vision and Machine Learning Erasmus Project

Project Title

Computer-assisted teaching of Sign Language using Computer Vision and Machine Learning

Project Key Action

This project related with these key action: Cooperation for innovation and the exchange of good practices

Project Action Type

This project related with this action type : Strategic Partnerships for higher education

Project Call Year

This project’s Call Year is 2020

Project Topics

This project is related with these Project Topics: Inclusion – equity; Teaching and learning of foreign languages; New innovative curricula/educational methods/development of training courses

Project Summary

According to the World Health Organization up to 5 out of every 1000 children are born deaf or hard of hearing. Those children obviously need a lot of support at school. Therefore, the educators need to receive appropriate training in Sign Languages, the native languages of the Deaf, so that they are able to communicate with them and to teach them how to develop linguistically. Sign Languages in the curricula of Primary Education University Departments or schools dealing with Special Education are very useful and contribute towards this goal. However, it is hard for Higher Education Institutes to find and employ as tutors experts in Sign Language, so many educators are not adequately trained as students to face the challenges of communicating with deaf children in the classroom. Similar considerations apply to other education levels as well.

Objectives
a. To develop an innovative and affordable system/service for interactive SL teaching for students in Special Education/Pedagogical departments and primary school education. It will be based on computer-vision, machine-learning, linguistic technology and avatars developed by all the involved partners in several national and EU-funded projects that are ongoing or recently completed. Thus we aim at an exploitation of reserarch results at European level that would not be possible without the contribution of the EU.
b. To set up the CAT-SL infrastructure in four Higher Education Institutes, and one primary education one, based on open standards, to enable its practical use by tutors and students.
c. To develop curricula and guides for teaching SL using the CAT-SL system for at least two multilingual courses, in Greece, Cyprus, Portugal and Netherlands, to support the social inclusion of the deaf children.
d. To actively disseminate the project results through the promotion of workshops with stakeholders in the EU to raise awareness on the main challenges faced by the Deaf.

Target groups
a. The students of Special Education university departments, aiming to learn how to use and teach SL to deaf children.
b. The university professors teaching SLs in Special Education departments.
c. Deaf children learning SLs.
d. Educators in public or private organizations involved in SL teaching or Deaf education.
e. Any organization, institute, company or individual interested in promotion of the use of SLs and related technology.
f. Local, regional, national and international bodies related to inclusion and citizenship.

The needs:
Curricula on SLs are essential in Special Education University departments, or Special Education schools, since the Deaf are one of the biggest target groups. The universal lack of university tutors with knowledge of SLs may undermine the quality of the provided courses. Therefore the CAT-SL aims to complement the practice sessions of related curricula with automated exercises to be performed by students and automatically evaluated by the system.

Transnational project:
The efforts for SL analysis and systems are fragmented over Europe. There are different groups focusing on machine learning, computer vision and avatar design, SL linguistics, SL didactics etc, however it is very hard for single national groups to encompass all the related know-how. Furthermore, the problem is not of local importance since there are many national SLs, so approaches that apply to a single SL may not be applicable to other ones. Therefore, a more global approach is necessary to create a system that could be extended to more international SLs.

The CAT-SL project is the joint effort of University of Patras (Greece), Cyprus Univesity of Technology (Cyprus), Polytechnic Institute of Porto (Portugal), Hellenic Mediterranean University (Greece) and Royal Dutch Kentalis (Netherlands).

EU Grant (Eur)

Funding of the project from EU: 259601 Eur

Project Coordinator

PANEPISTIMIO PATRON & Country: EL

Project Partners

• TECHNOLOGIKO PANEPISTIMIO KYPROU
• INSTITUTO POLITECNICO DO PORTO
• ELLINIKO MESOGEIAKO PANEPISTIMIO
• Stichting Koninklijke Kentalis