AirWrite Studio
By
@priyajitk
- 4 Devlogs
- 4 Total hours
A hands-free digital canvas that lets you draw, erase, and annotate using nothing but your hand gestures in front of a webcam.
2h 21m 45s logged
1h 17m 2s logged
Hello Everyone!
I have been working on this project since two months, now. And finally with the help of the Hack Club Tester Team, I am Finally able to make a final release of this Project. I have also updated the demo link so that you all can now directly download a .exe file without any problem. At first you may see a warning as I have not bought a certificate for the .exe program but you can click on Run Anyway to use it. But if you still want a web one navigate here
Ship #1
AirWrite Studio is a real-time, hands-free spatial drawing platform I built using Python, PyQt6, MediaPipe, OpenCV, and offline AI tooling. The idea behind the project was simple: I wanted to make digital whiteboarding and creative interaction feel more natural and frictionless without depending on expensive hardware like drawing tablets, styluses, or touchscreens.
The application transforms a standard webcam into a gesture-controlled digital canvas where users can draw, erase, select, move, and manipulate content entirely through hand gestures and voice commands. Instead of adapting ourselves to traditional input devices, I wanted the interface to adapt to human movement.
One of the biggest quality-of-life improvements is the completely hardware-free interaction system. Users can simply sit in front of a webcam and interact with the canvas naturally in mid-air. I also implemented smart geometry and OCR features that automatically clean up rough sketches into proper geometric shapes and convert handwritten strokes into editable text. During testing, this genuinely made brainstorming and diagramming feel much smoother because users didn’t have to waste time fixing messy sketches manually.
Another major feature I’m proud of is the multimodal accessibility system. Alongside gesture controls, the project integrates offline voice commands using Vosk, allowing users to execute actions like “undo,” “clear,” or change colors without touching the keyboard or mouse. I wanted the experience to feel seamless and accessible, especially for presentations, teaching environments, and users who may struggle with traditional input devices.
The hardest part of the project was making the interaction system feel stable and intentional. Human gestures are naturally inconsistent, and early prototypes constantly suffered from accidental triggers and jittery movement. A huge amount of time went into smoothing gesture input, tuning confidence thresholds, and implementing filters like the One Euro Filter so the controls would feel responsive without becoming chaotic. I also had to optimize performance carefully because the application simultaneously handles real-time computer vision inference, OpenGL rendering, OCR processing, gesture recognition, and UI updates.
What I’m most proud of is how natural the final interaction feels. Gestures like pinching to draw, making a fist to erase, or pointing to lasso-select objects became surprisingly intuitive after enough iteration. I also spent a lot of time polishing the UI because I didn’t want the project to feel like a rough prototype. The interface takes inspiration from modern minimalist tools like Linear and Notion, with a restrained dark aesthetic that keeps the focus on the spatial interaction itself.
The project can be tested using a standard webcam and a Python 3 environment. After cloning the repository and installing dependencies through `pip install -r requirements.txt`, the app can be launched using `python main.py`. Optional integrations include Tesseract for OCR functionality and Vosk with PyAudio for offline voice commands.
Once the camera starts, users can interact entirely through gestures:
* Pinch thumb and index finger together to draw
* Make a fist to erase
* Point with the index finger to select objects
* Use multi-finger gestures to drag content
* Keep an open palm to remain in neutral mode
There’s also a smart shape mode that automatically converts rough freehand sketches into clean geometry in real time, which ended up becoming one of my favorite parts of the entire project.
Overall, AirWrite Studio was my attempt to explore what frictionless spatial computing could look like using only commodity hardware and open-source technologies. I wanted it to feel practical, accessible, and genuinely useful rather than just a flashy computer vision demo.
25m 44s logged
AirWrite Studio — Devlog
Overview
AirWrite Studio is a hands-free digital canvas built with Python and PyQt6. It uses webcam-based hand tracking and gesture detection to let users draw, erase, select, and manipulate canvas content without a mouse or stylus.
Current Status
- Core application structure is in place with
main.py, UI modules, tracking modules, and canvas rendering. - Hand gestures are detected and mapped to canvas actions through MediaPipe landmark tracking.
- Gesture control covers drawing, erasing, object selection, dragging, scaling, and neutral/stop states.
- Voice command support is planned via Vosk and the
tracking/voice_commander.pymodule. - The canvas supports export formats and workspace management via native file handling.
Features Implemented
- Gesture-based drawing and erasing
- Shape recognition and smart geometry snapping
- Text conversion support using OCR integration pathways
- Grid templates and presentation-friendly laser/highlighter modes
- Undo/redo history architecture
- Clean dark UI layout using PyQt6 widgets and custom styling
Project Structure
-
main.py— Application entry point -
config.py,utils.py— configuration and helper utilities -
canvas/— canvas engine, widgets, object rendering, serialization, shape recognition, export -
tracking/— hand tracker, gesture detector, smoother, voice commander -
ui/— main window, sidebar, camera widget, toast notifications, styles -
assets/— model files, Vosk speech resources, hand landmarker task files
Recent Work
- Built the main interactive canvas and stabilized gesture interactions.
- Added smart UI for tool selection and camera control.
- Integrated offline voice command support architecture with Vosk assets.
- Added documentation in
README.mdto describe features, installation, and usage. - Created a web-based demo version using Flask and Hugging Face Spaces Docker deployment. Note: The web demo relies on server-side processing for MediaPipe gestures and may experience latency or glitchiness depending on the user’s hardware limitations, webcam quality, and network speed.
Challenges
- Ensuring reliable hand detection across different lighting conditions and webcams.
- Mapping natural gestures into consistent canvas commands without accidental triggers.
- Keeping performance smooth for real-time drawing and gesture recognition.
- Handling network latency and server-side model processing for the web demo without native client-side binaries.
Next Milestones
- Finalize gesture calibration and user feedback flow.
- Complete OCR text conversion and editable text object workflow.
- Add full export support for PNG, PDF, SVG, and native
.airsaves. - Polish the UI, including toolbar behavior, status indicators, and onboarding hints.
- Add tests for canvas serialization and gesture classification.
Notes for Future Development
- Add a
requirements-dev.txtfor development tooling, formatting, and testing. - Consider adding a
CHANGELOG.mdor release notes for future version tracking. - Keep
venv/and__pycache__/ignored in source control, and preserve only source files and assets.
18m 51s logged
AirWrite Studio - Devlog
AirWrite Studio is a hands-free digital canvas built using Python, PyQt6, and MediaPipe. The idea behind the project is to allow users to draw, erase, select, and manipulate objects on a digital canvas entirely through hand gestures detected via a webcam.
The core structure of the application is now functional. The project currently includes the main application window, canvas rendering system, tracking modules, UI components, and asset management. Real-time hand tracking is handled through MediaPipe landmark detection, which is then translated into different canvas actions.
Several gesture interactions are already implemented. Users can draw using a thumb-and-index pinch gesture, erase using a fist gesture, select objects using a pointing gesture, and drag or scale objects using multi-finger gestures. An open palm acts as a neutral or stop state to reduce accidental interactions.
A major focus during development has been making gesture controls feel natural and stable. Small hand movements, lighting changes, and different webcam qualities often create false triggers or unstable tracking, so additional smoothing and gesture filtering logic has been added to improve consistency during real-time interaction.
The canvas system already supports freehand drawing, object movement, shape recognition, and undo/redo architecture. Smart shape snapping can detect rough circles, rectangles, triangles, and straight lines and convert them into cleaner geometry automatically.
Additional drawing tools have also been added, including:
- highlighter mode
- laser pointer mode
- grid templates
- customizable pen settings
- dark themed UI styling
The project also includes the architecture for OCR text conversion and offline voice command support. OCR integration pathways are connected for future editable text workflows, while the voice command system is being developed using Vosk through the tracking/voice_commander.py module.
The repository is currently organized into separate modules for better maintainability:
-
main.pyhandles application startup -
canvas/contains rendering, serialization, export, and shape recognition systems -
tracking/contains hand tracking, gesture detection, smoothing, and voice command modules -
ui/contains interface components such as the sidebar, camera widget, and notifications -
assets/stores speech models and hand tracking resources
Recent development work focused on stabilizing gesture interactions, improving the canvas workflow, refining the UI layout, and integrating voice command resources. Documentation was also added through the README.md file to explain installation, features, and project usage.
One of the biggest challenges so far has been balancing responsiveness with stability. Gesture systems need to react quickly without triggering unintended actions from natural hand movement. Maintaining smooth performance while simultaneously handling camera input, landmark processing, UI rendering, and drawing operations has also required ongoing optimization.
The next stage of development will focus on:
- finalizing gesture calibration
- completing OCR text conversion workflows
- adding full PNG, PDF, SVG, and
.airexport support - polishing onboarding and status indicators
- adding automated tests for serialization and gesture classification
Future improvements may also include a dedicated requirements-dev.txt, release tracking through a CHANGELOG.md, and additional development tooling for testing and formatting.
The project has now reached a stage where the core systems are functional and connected together. Most upcoming work will focus on refinement, optimization, and improving overall usability.
Which is usually the phase where software projects politely reveal twice as many problems as originally expected.
