ESL - Educate Sign Language
ESL (Educate Sign Language) is a PySide6 desktop application that helps users learn Korean Sign Language (KSL). It captures webcam video, extracts hand landmarks with MediaPipe, feeds the landmark sequence into a Conv1D + LSTM model, and recognizes which sign word the user is performing.
Graduation capstone project. Source code: github.com/sunukkim98/EducateSignLanguage-ESL
Features
- Sign Dictionary — browse and play example sign videos by category
- Learning Mode — follow along with a category’s sign videos (hobbies, personality, family, hometown, age, language, etc.) while the webcam checks whether the gesture was performed correctly
- Quiz Mode — a sentence like “My hobby is (swimming), my personality is (positive), and I have a (brother)” is randomly generated, and the app recognizes each sign in sequence via webcam to complete the sentence
Tech Stack
| Area | Technology |
|---|---|
| GUI | PySide6 (Qt6) |
| Hand landmark extraction | MediaPipe (Hands solution) |
| Gesture recognition model | TensorFlow / Keras (Conv1D + LSTM) |
| Video processing | OpenCV |
| Korean text overlay | Pillow |
Demo
Learning Mode
Quiz Mode
Sign Dictionary
