pure hell of trial and error

I am actually unsure if this will even work properly now.

But fear not freinds, “BECAUSE I AM HERE” mha reference btw.

I will actually take it a bit slow now, and ship in 10 days. Promise, no more delays.

bug fixing session, no exciting updates, the more i think im coming close oto finishing it, more the work remains.

Progres Report

• Simulated a hand from data in 2d space
• X axis is found from TDOA cross-correlation
• Y axis is found from doppler shift and EMA decay integration.
  -It’s not the best, but it works 70% of the time.

The project is almost finished, but i’m not happy with the accuracy, so ill work on making it better. As good as it can be.

#Code is not working. After the last post, I made some changes, and maybe messed up. Now something is wrong and im not able to figure it out. Since no AI is used in this project, this is becoming really hard.

Progress Update

• Tried moving hand towards and away from laptop at known velocities.
• Tried to receive doppler shifted audio from the movement of hand.
• Extracted frequency shift from each tone independently using FFT.
• Did some more math and started losing my brain because it wasn’t working at the start.

Now i’ll fuse methods, and get x and y coordinates for the plane.

#Progress so far

played a short pulse first from left speaker alone, then right one. Measured the angles of arrival, then calculated distance of how far apart these speaker and microphones are.

This system can be used for any laptop, allowing for a general approach to be taken further.

Yea next ill just verify two tone doppler separation.

JUST WAIT FOR THE PROGRESS REPORT A BIT MORE, IM HAVING TROUBLE WITH THIS LAPTOP SPEAKER SEPERATION DETECTION.

WHY IS THIS STUFF SO HARD

I was in the middle of making the distance calculator.

So i’ll complete it and post my next devpost explaining it, this time limit of 10h is kinda annoying tbh. lol

Devlog 2, A breakthrough!

Quick introduction

I’m working on a very janky Switch 2 cloudplay setup:

AYN Thor → Windows → XInput → C# → UART → Raspberry Pi Pico → fake Switch Pro Controller → Switch 2

After around 3 days of work the Switch 2 could;

• Identify the controller
• Complete the handshake
• Send bytes and other useful info to my Pico

But it would never actually connect with the Switch 2, I kept messing with the code, debugging, changing bytes I even asked AI but I never got the answer.

Then I had an epiphany,

What if all this time my code was working just fine and I wasn’t testing it correctly, so I sent a singular button press with right_buttons |= (1 << 3); and suddenly. It connected. Apparently, the Switch was waiting for an actual input report before considering the controller fully connected.

All that work and time just to mess up my entire codebase. At least I learned a little more about C and hexes 😓😓

What is next?

I think the project is close for a v0.1 release, I’ll need to;

• Fix the A button input
• Clean up the codebase from all the testing mess
• Test
• Build the application

In the future I want to build a special application instead of relying on OBS, reduce latency and add rumble + gyro support.

https://youtu.be/oTnj4p9ZfQ8

Tried to merge FFT and doppler shift implementations finally

Yea….. This is the only thing done in past 10 hours of work, but lemme explain how hard it is.
This will be a weird analogy but hear me out.

Imagine a robot, it can use sonar to keep a map of things around it and use the change in reflections and finally knows its own change in position. It’s absolute for knowing the surroundings around you, but it drifts overtime.

Meanwhile a simple tracker using wheel can tell you change in location. But you can’t get information about surroundings.
So they both are merged so that the drifting of sonar is stopped.

I’m applying doppler shift in the same way, since it detects only the change, we can use it to correct the positions acquired by FFT implementation.

Hope you all get it, ofcourse its not correctly working right now, so i’ll work on it.

Tried to merge FFT and doppler shift implementations finally

Yea….. This is the only thing done in past 10 hours of work, but lemme explain how hard it is.
This will be a weird analogy but hear me out.

Imagine a robot, it can use sonar to keep a map of things around it and use the change in reflections and finally knows its own change in position. It’s absolute for knowing the surroundings around you, but it drifts overtime.

Meanwhile a simple tracker using wheel can tell you change in location. But you can’t get information about surroundings.
So they both are merged so that the drifting of sonar is stopped.

I’m applying doppler shift in the same way, since it detects only the change, we can use it to correct the positions acquired by FFT implementation.

Hope you all get it, ofcourse its not correctly working right now, so i’ll work on it.

Updates :

Heres another fun photo actually

• Actually tested with multiple laptops to see how much differently the same signals behave. A little bit different yes.
• Tested with 2 other ways just in case they were better. No and kinda??
• 2/3 FFT implementation done
• Yes i am doing that doppler shift thing too now

Hardware obstacle faced

btw heres a cool monster can gun i made in my free time

ive been testing with sounds all this time. But i didnt realise that different laptops have different sound systems. Like downwards firing speakers and upwards firing speakers.
So i have decided that this wont be able to be made for all laptops. So i definitely will have to create a video for this. lol.
Im on fire!

THIS IS HELL

I have nothing new to show, since im still working on the same thing for 2 devposts now. So heres a cool photo

Anyway, after successfully implementing this FFT, I’ll move on to integrate doppler shift as well, like those robots that although have a lidar, still have that internal tracking system. Since lidar drifts, the internal tracking system corrects it. Atleast thats the vision. And implementing doppler shift isnt easy AT ALL, atleast for left to right motion.

im actually tired of failing and trying some solution without using AI.
It would have made it a lot easier but, I’m kinda stupid and egoistic ig?

Made significant progress on FFT implementation

I’m maybe 1/3rd the way to implement FFT ig?
This is getting pretty hard.
I’m spending most of the time just writing, running and testing the code of this implementation.

As it turns out laptops arent really the best at this stuff. And stuff like this being done by actual companies, makes sense now. It’s way too difficult.

Worked more on the implementation of FFT

The picture attached is just when there is forward or backward movement of hand, it glows brighter. Cus I tested with doppler shift a little.

So, I learnt a few more concepts required to make this.

Applied FFT a bit more nicely and neatly.

ALSO I LEARNT COMPLEX FFT WHICH IS AMAZING TBH.

Yea, so far its just applying what I’m learning.

Not easy for a 16 year old to do college shit.

Created a holding structure for the Biosensor

Went through multiple iterations.

Since I was new to 3d printing, my design initally was VERY VERY BULKY, which turned out to cost a lot.

Then went through two more iterations and cut down the need for supports and designed a really light and minimal structure.

For the data collection part, well, we have successfully collected enough data to actually predict in real time what future voltage is.

Will take one more month for this to be completed.

Implemented a basic distance detector using laptop speaker and microphones

Tbh all these difficult concepts, you understand them, do their maths, and then boom, you just implement it using a library. WHY DID I SPEND SO MUCH TIME UNDERSTANDING THEM IN THE FIRST PLACE. I AM A 16 YEAR OLD, ITS REALLY HARD FOR ME TO UNDERSTAND COLLEGE LEVEL SIGNAL PROCESSING, I STILL HAVE FFT AND TO UNDERSTAND!!!! anyway

I’ve been working on a project that basically sees without seeing. So far, i’ve understood concepts like convolution, sampling rate, correlation, stuff to basically find a sound we want from a sample.

I then wrote a code to send out a linear increasing list of frequencies from 10000 to 15000. Then do a convolution of the recieved sound with the target sample to detect where in the recieved sound lies the target sample.

Yea, I updated some minor things, nothing big