DEMO

To Present an Ear & a Mouth To Those Who Lost Them

What are we dealing with?

(Problem Domain)
The community with issues in listening or speaking abilities faces a challenge in communicating with the rest of society. Their main approach for face-to-face expressions is sign language, which isn't an easy language to learn. That in return results an isolation of those challenged people from the face-to-face communication, and push them behind the screens.

What are we doing?

(Project Overview)
product FingerSpeak is an app for those who became special-need individuals & who want to communicate with them. It captures the PC webcam live-feed and interactively translates or assesses the learning process. This app involves the artificial intelligence (AI) technology of image processing, among others.

To Learn

Some vocabularies and phrases presented in sign language so the user would imitate and the machine would assess.

To Translate

Help the user by translating a preset group of hand signs in case they are new / not understood by the user.

To Listen

Text-to-speech feature to help the user by hearing the pronunciation (in case the user only have a speaking challenge).

To Control

Cool features to control the app by hand signs like controlling the volume and closing windows of the app.

How did we do it?

(Project Process)

Brainstorming

A lot of ideas, the challenge was to choose just one!
Text

Non-Taken Ideas

Help people with learning difficulties learn how to write
letters in the proper way
Text

Adding Features

Practice, Translate, Listen to the word, Control the audio level
Text

R&D

light and efficient
Text

How would you use the software?

(Project App Manual)

App Interface
To Practice the Hand Signs
To Listen to the Word Pronunciation
To Activate Live Translation
To Control the Volume Level
To Lead You To This Page

Use It Now

At the moment you have to go through some painful process to setup the environment to use the app. But we promise in the near future to make it as an .exe file (one file to do all the setup).

Github Repo To Clone Exe File (Soon)

Resources & Links

Software & Libraries

Python

OpenCV

MediaPipe

PyTTSx3

TKinter

Pillow

Others

Documentation

Resource

Resource

Soon-To-Have Features

  • To involve machine learning so the training datasets keeps expanding with every use of the software, and hence, the accuracy of the images processing would improve.


  • Images processing accuracy could also be developed by collecting feedback of a test group of the targeted audience.


  • Rebuild the system as a web app to get rid of the localization issues.

Testimonials

This is amazing work considering what you've been through, be proud about yourselves.

Yahya Albashar

It's a good project, but ours is unprecedentedly the very top best among all.

Zaid Jarrar

THE

The Great Team Behind the Scene

Ghaida Al Momani

Software Engineer

Emad Almajdalawi

Software Developer, Mechanical Engineer

Raghad Abdulhadi

Civil Engineer, Software Developer

Moayad Abukhadra

Mechatronics Engineer, Software Developer

Suhaib Ahmad

Full-Stack Web Developer

Thank you for your support!


All Rights Reserved © | Made with by