Felix is like a magic assistant that lives in your phone and your computer at the same time. You point your phone's camera at something in real life (like a whiteboard, a receipt, or some messy notes), you say what you want out loud, and Felix actually goes and does it on your Mac.
Felix
This past Saturday for TillyHacks. Me and my teammates Ishaan Gulati and Varun Surianarayanan created Felix, an AI assistant that can see the real world through your phone’s camera and actually perform tasks on your Mac. Instead of manually typing things into apps, you can point your camera at something like a receipt, whiteboard, or handwritten notes and simply tell Felix what to do. The app uses speech recognition, computer vision, and AI planning to understand your request, then automatically creates notes, updates spreadsheets, drafts emails, generates presentations, plays music, or opens websites. I built the system using SwiftUI for the iPhone app and FastAPI for the backend, with AI models handling image understanding and task planning. The coolest part of the project was seeing an idea that felt futuristic actually work in real life.
For the future of Felix, me and my team are currently working on deploying the web dashboard to Vercel and the backend to a database like firebase. We also hope to add more features to this later on!
