Lumen

Welcome to Lumen (V 1.9 Beta): The Free Assistive Communication Software!
Access Lumen: https://philipe-pajishvili.github.io/Lumen/
Current Development Stage: Refinement/Testing Stage

The hours on the Hackatime monitor are not accurate for how long I spent on this (about a week).

Please give it a try and let me know how it goes!

A Quick Introduction to Lumen

Uses AI-powered gesture/pose recognition to help users communicate.

Uses a webcam to detect specific movements and translate them into words.

Includes an advanced keyboard for spelling custom messages.

Can speak selected words and messages aloud.

Includes customizable settings such as word detection speed.

Designed to make communication more accessible, independent, and engaging.

Built with JavaScript, TensorFlow, ml5.js, p5.js, and Teachable Machine.

How to Use Lumen

1. Launch Lumen

Open Lumen and allow access to your camera.

Position yourself so your upper body is visible to the camera.

2.Use the Pose Model

Lumen comes with a pose-recognition model trained to recognize your left arm.

You can upload your own pose model from Teachable Machine by pasting the model link below the black box.

Move your left arm into the trained position (shown in the image) to select or communicate a word (in blue) or activate a menu.

Note: You MUST wait for the black box to say “Input Detected” before putting your arm down.
3. Choose Your Communication Mode

Use Basic Mode (Default) for quick, preset words.

Switch to Advanced Mode by activating the menu in Basic Mode to access the keyboard and build custom messages.

4.Speak Your Message

In Advanced Mode, use “Say” to have Lumen read your selected word or custom message aloud.

Adjust Word Speed in Settings to change how quickly Lumen detects selections.

Give it a try and let me know what you think! :)