5h 52m 17s logged
I’ve fixed some bugs and leaks, added more configuration to Android app and created hand tracking vizualization for the PC script. Also, I am working on integrating hands as controllers (because controllers have better support across games).
3h 52m 51s logged
YEAH! I have finally done the PC camera hand tracking! It has much better FOV than phone camera and more computing power.
It’s just Python script with MediaPipe, sending the data to 127.0.0.1 to hand port, and it has same format, so the driver doesn’t even know the data is sending someone other than phone. You will ofc need to disable phone hand tracking, otherwise they will collide.
As you see in the video, I can move the hands far and they still get tracked. (End of the video is when I took off the headset - it was also tracked :D)
1h 18m 41s logged
Now I am making just minor changes to the driver as the base is nearly done. I added screen size synchronization with phone, so the driver now can send right stream. Also I am making small bugfixes and improvements.
1h 44m 7s logged
I added settings on phone, so now you can really configure the ports on both sides!
Also there’s option to disable hand tracking or 6DOF (then it will send only rotation of headset)
3h 55m 16s logged
I created simple terminal installer in python that downloads and setups Monado Phone Driver.
Thanks also for the positive reviews and the 19.12x stardust multiplier — when I started this project, I didn’t think I’d ever finish it :D
Hi! This is my second ship of this project, Monado Phone Driver. It’s bridge between your VR app and your phone in headset (Google Cardboard).
I wanted to make it better than ALVR, because it has only 3DOF tracking, so I created 6DOF tracking and hand tracking, like professional headsets.
The most challenging part of this was debugging all of these. From Vulkan swapchain management and GStreamer pipeline to hand and 6DOF tracking.
I am proud of it, because this will help people without VR headsets to enjoy VR games.
NOTE: Hand tracking is done the best possible way, but I think it still doesn’t compete with real VR headsets. Next ship should add better tracking with PC camera.
27m 42s logged
Last devlog before ship! I just merged upstream monado, built release binaries and created GitHub release.
2h 6m 9s logged
So I finished the hand tracking! I tried to make it as good as possible on a phone, and it looks like it might actually work! Even though I’m limited by the hardware, you can see in the video that it tracks pretty well.
(video from openxr-simple-playground)
5h 38m 13s logged
I am enhancing hand tracking now, and it’s starting to look good!
I use MediaPipe by Google to analyze image from front camera. MediaPipe Hand Landmarker returns hand landmarks in camera-relative format in two dimensions, so I need to make depth (z-coordinate).
I measure distance between wrist and fingers, in 2D space this means distance from camera (the farther away they are, the smaller they get).
Now, they are 3D, but still relative to camera. Then I simply send both hands as UDP packet for the driver.
The driver (PC side) computes real-world coordinates of the hands using saved earlier sent from ARCore and pushes them into Monado.
6h 58m 26s logged
I worked hard on both the Android app and the Monado driver and finally: I have working hand tracking!
It is very laggy, needs many optimizations, and isn’t even in alpha state yet, but I can see that it is possible to do it. This was my dream when I began this project, and finally, it has come true. I hope it will help many people who cannot afford a professional headset :D
2h 14m 48s logged
So today I finished image settings (you can configure them in the config file). They control how the image is rendered, because every headset has different lenses, distance between lenses, distance from screen to lenses etc. To find your headset’s properties, go to https://wwgc.firebaseapp.com/.
3h 32m 23s logged
I created new TCP socket logic for configuration sharing and added logic for loading configuration file.
Now I am trying to change the x265enc software encoding to something faster on GPU. Unfortunately, nvenc (nvidia encoding) is really hard to configure properly, as you can see for yourselves.
9h 15m 50s logged
Phew, I almost went over the time limit (10 hours) :D
I tried to get it running on Windows, and I partially succeeded, but it’s still not stable. Aside from that, I mostly cleaned up the code, improved usability, and removed unnecessary modules from CMake.
Now I’m working on an improved distortion using Monado’s u_distortion_cardboard_calculate (cardboard stands for Google Cardboard btw)
And one more thing: thanks to everyone for the positive feedback on the project — I really didn’t expect that!
1h 37m 53s logged
Since nobody reviewed my submission, I don’t want to add more features, so I only updated the README and made small changes to the app.
If anyone wants to play VR games on their phone, they can just download the files from Releases and run them without hard compiling!
Hi! I ship my Monado Phone Driver! I worked on it for last week and it was really challenging. My vision was to build something with what you can play VR games on phone (with Google Cardboard), like iVRy, but open-source and simple. I worked on streaming pipeline, UDP discovery, pose tracking and gluing it all together with Monado. I hope this will help people that wanted to play VR games without expensive headset.
6h 51m 57s logged
Devlog 4: I replaced boilerplate LLM code with my own. I also wrote README with instructions, so you can now try it!
36m 8s logged
I am making minor adjustments to run with SteamVR. I use opencomposite as layer between OpenXR and OpenVR/SteamVR.
Video is phone screencap from free Steam game “Aircar”.
3h 4m 34s logged
Devlog 3: I made 6DOF (6 degrees of freedom = position+rotation in space) tracking with ARCore. It’s a bit laggy, but it works. This is just test, I plan to optimize it later.
Video is from xrgears with looking around.
1h 30m 21s logged
Devlog 2: Finally I made working streaming pipeline! My driver takes image from Monado, gstreamer encodes it and sends via UDP. On phone, there is AMediaCodec that takes the packets, joins them and renders. Used codec is HEVC (H265).
Video is from xrgears demo.
4h 36m 26s logged
So now I’m basically stuck. Three hours of debugging the streaming pipeline, refractoring with Opencode and trying to make it work. As you see, even Deepseek is struggling with it…
Wish me luck 🤞
