@cluelessfr

Devlog 09:
Now the app stores the selected video’s Uri in an activity field, meaning it can be accessed for later processing instead of only the status text. Also frame timestamps can now be generated. This means that you can input the video duration as well as the interval that you want the timestamps to be generated, then you will get a set of timestamps for the video. The video metadata can also now be read. It uses Android’s MediaMetadataRetriever to open the video. Currently only the duration of the video can be retrieved, but later you will be able to decode individual frames. Finally I added tests for all of these. For the metadata retriever specifically I uploaded a test video to confirm that I could get an accurate measurement. Also updated the documentation.

Devlog 08:

I finally added the MediaPipe dependency, as well as a few functions. First of all I also added the Pose Landmarker Full model for it to actually detect poses. Then I created a manager function to initialize and close MediaPipe. I then added a function that could process an already decoded video frame with a timestamp using MediaPipe. I added tests for all of these to ensure they work when the app runs, and updated all the logs.

Devlog 07:

I changed the emulator from being on device to remote via Firebase. This allows me to test the app cleanly without lagging my computer. The only problem is I get a limited 30 minutes per month, so I have to use them wisely. I also updated the Point2D function to reject non-finite points. This prevents errors on the future. I also added a class that combines all the other functionality into one. It returns an OptionalDouble, which is useful when you don’t know if you are going to get a result. This new function accepts three LandmarkObservations, Then if the result is valid, it returns the angle degrees between the points. If not, it returns nothing.

Devlog 06:

I added the landmark quality gate, which helps determine whether the visibility and presence of the point meet the minimum requirements. If it does, then it passes. If it doesn’t, then the point fails. From here there are two options, either the app tries to find another point, or the app says it cannot reliably determine your form from that video

I also added the AI use log. I mainly did this because I will be submitting this to the Congressional App Challenge.

Devlog 05:

Today I added landmark storage and reliability tests. What this is is basically a new object that can store a point, its visibility, and presence. The point is the point itself, the visibility says how well the object can be seen, and the presence determines how much of the object is actually in the video frame. These three factors help determine how well the video can be analyzed. If the video cannot be analyzed well enough, then FormCheck will send a message saying so (This hasn’t been implemented yet, but this is the basic idea of why this is there in the first place).

Devlog 04:

I added functions for measuring the angle based on three points (the joints). Basically how this works is that it first takes in three points, the first endpoint, a vertex, and a second endpoint. Then it uses the dot product formula with the two vectors to find the angle between the two lines.

Devlog 03:

I added the file picking dialogue. Now whenever you click the “Select a video” button the android folder picker opens and you can choose a video. I think next is working on the actual pose detection logic.

Devlog 02:

I added the initial video selection interface. I added some text to describe the app, and a button for the user to be able to select a video to upload. Currently, the button does not do anything. Next I will have to continue working on the button and the actual pose estimation part of this.

Devlog 01:

This is the first devlog of the FormCheck project, an Android app that helps you check your form during workouts. Currently I just set everything up in Android Studio, connected Hackatime, and set up GitHub.

Devlog 39:

I was finally able to get the Linux guards in place, and also packaged the new builds and released them to GitHub has a new version. The guards were mostly simple, as they were just a few checks that asked what operating system the user was on. And depending on that it decided whether or not to install the Spicetify extensions.

Devlog 38:

I worked on functions that helped install the Spicetify extension into the Spicetify configuration. I had to ensure Spicetify was installer, had to find its path, had to find its extensions path, then finally copy over the extension.
I also added system tray behavior and multiple instance detection for the app, so that now when you close the app it doesn’t close but instead goes to the system tray. This allows the Spotify integration to work even if the app itself isn’t visible. The multiple instance detection ensures the app runs properly, and ensures that the app doesn’t try to use the same http port twice, resulting in errors.
This did take a while but now all I have to do is create guards for Linux, so that the Spotify integration only works on Windows and doesn’t create unexpected errors on Linux.

Devlog 37:

I am very close to shipping now. I added the Spotify status/progress update messages, so now you can see when the tracks have finished downloading. I also fixed three (kinda major) bugs. First of all there was an error when searching for the songs on YouTube/YouTube Music. Basically what happened was that the title didn’t follow the same ASCII conventions that was used by YouTube. Because of this the app would reject good title matches even though they were the same. This produced quite a few errors, especially when trying to download songs with apostrophes. The second error I fixed was the some songs would automatically have the clean version downloaded. To fix this I switched to the ytmusicapi python package. This package also returned whether the song was explicit or not. Because of this I was able to check whether or not the song was intended to be that way, and if not I could prevent that match from being downloaded. I did have to edit a few functions to accept the new format, but in the end everything worked. Third I fixed a simple error that messed up the searching. This time it wasn’t really a bug, but instead an edge case that I missed. I was catching keywords like “feat.” and “featuring”, but I didn’t catch words like “From (and tv show, etc.)”. Because of this some matches were getting left out. I fixed this by adding another regex pattern that removes this as well.

Devlog 36:

I am almost done with the first phase of the integration. When the download button is clicked inside of Spotify, Spicetify intercepts it and now posts a request to the open web server. Then the python script gets notified of that request, decodes it, then automatically starts downloading the song/playlist/album. The only thing is that the myMusic app has to be open while doing this, but I will remove that requirement in the future (not now). The next step would be to add status reports to Spotify so that the user knows what the status of their download is.

Devlog 35:

The app now creates its own server on a very distant and far port so that your other local host stuff doesn’t get affected. I did this with uvicorn and threading, so that creating the port doesn’t make the app freeze. With this functionality the app can get the requests made inside Spotify and automatically download them. The next step is to get the app to report the statuses of the download jobs (queued, downloading, completed, failed, etc.). Then I would actually have to connect both parts, let Spotify send messages and the app receive the message.

Devlog 34:

I have been working on creating a queue for the requests. This ensures that the user can download multiple songs at once without having to wait for the previous ones to be finished. So far I have been able to create a request id for each request. The app can update the attributes of each request, such as its status, and error message. On top of this once a request is completed the app also automatically removes that request.

Devlog 33:

The app is now able to understand and automatically download any received links. Now the problem is that Spotify isn’t able to send any normal requests to via the computer. Because of this I need to find a way for Spotify to send and get data. I know that Spotify (using Spicetify) can send and get http requests. Because of that I am planning to use FastAPI and a local http server to be able to send and get data so the app can automatically download tracks. I will ensure to keep the server in a high index to ensure it doesn’t get in the way of any other local servers.

Devlog 32:

(sorry for the late post)
I have been working on a really cool feature for the app. Basically what I am trying to do is integrate the myMusic pipeline into the windows Spotify app itself. When you right click a playlist or album, there will be a menu that pops up. Usually for normal Spotify users (not on premium) there will be a greyed out download button. What I am trying to do is two things: add a download button for tracks (because there isn’t one right now), and make the playlist/album button actually work.

Right now what I have accomplished is adding a new button for the track and making the playlist/album button clickable. I was able to do this by using Spicetify and its framework. Because I already have it installed on my computer I am using it to test everything out but later I will bundle it with my app. What I did was build a custom Spicetify extension (I had to learn JavaScript) that creates a new button.

I was able to make the original playlist/album button work because it was only “aria-disabled”, not actually disabled. This means that it was only disabled at the UI level, but Spotify still recorded clicks. By applying some UI changes I was able to make the button look enabled and still record clicks.

My next step is to actually hook this up to the Python script. I think this is a really cool feature.

Devlog 31:

This devlog was mainly focused on the efficiency and speed of updating the app. Before the data would come in really small packets, meaning the cpu would have to write to the drive about 130,000 times in total (I know 😭). I adjusted this so it now comes in larger packets so the cpu doesn’t have to do more work. I also adjusted the compression and a few other Inno Setup settings (number of threads, etc.) to make the app more efficient in general.

Another issue that I found is that once the app updated (specifically in Windows, idk about other platforms) the settings app would still show the older version for some time. This was an issue with the timing, so I had to adjust some of the delays to get it to work properly. Overall I think this is a pretty good update.

Devlog 30:

I FINALLY implemented playlist and album support. At first I thought it would be simple, just looping though each song in the playlist/album. Oh was I wrong.

First I had to add two more methods that looped though each song, one for playlists and one for albums. There had to be two different ones because each returned metadata differently. Then I had to kind of change the format of the whole thing, because I had to store all of the songs in a list, but currently I was only returning dictionaries from all of my methods.

After getting all of that sorted out I had to actually fix the GUI/downloading part. I had to update the downloading method to handle playlist/album links, then I had to properly handle all of the status message for each song (“Ready”, “Downloading”, “Failed”).

Finally I had to work on fixing the bugs I had introduced and handling the edge cases. After all of that I FINALLY had playlist and album support.

One thing that was sorta tough was handling the status for each track individually. Before I was just looping through the entire list and setting all of the statuses to the same thing, but I was able to create a new method and change each song status as it was being downloaded, not for every song as a whole.