myMusic

Devlog 39:

I was finally able to get the Linux guards in place, and also packaged the new builds and released them to GitHub has a new version. The guards were mostly simple, as they were just a few checks that asked what operating system the user was on. And depending on that it decided whether or not to install the Spicetify extensions.

Devlog 38:

I worked on functions that helped install the Spicetify extension into the Spicetify configuration. I had to ensure Spicetify was installer, had to find its path, had to find its extensions path, then finally copy over the extension.
I also added system tray behavior and multiple instance detection for the app, so that now when you close the app it doesn’t close but instead goes to the system tray. This allows the Spotify integration to work even if the app itself isn’t visible. The multiple instance detection ensures the app runs properly, and ensures that the app doesn’t try to use the same http port twice, resulting in errors.
This did take a while but now all I have to do is create guards for Linux, so that the Spotify integration only works on Windows and doesn’t create unexpected errors on Linux.

Devlog 37:

I am very close to shipping now. I added the Spotify status/progress update messages, so now you can see when the tracks have finished downloading. I also fixed three (kinda major) bugs. First of all there was an error when searching for the songs on YouTube/YouTube Music. Basically what happened was that the title didn’t follow the same ASCII conventions that was used by YouTube. Because of this the app would reject good title matches even though they were the same. This produced quite a few errors, especially when trying to download songs with apostrophes. The second error I fixed was the some songs would automatically have the clean version downloaded. To fix this I switched to the ytmusicapi python package. This package also returned whether the song was explicit or not. Because of this I was able to check whether or not the song was intended to be that way, and if not I could prevent that match from being downloaded. I did have to edit a few functions to accept the new format, but in the end everything worked. Third I fixed a simple error that messed up the searching. This time it wasn’t really a bug, but instead an edge case that I missed. I was catching keywords like “feat.” and “featuring”, but I didn’t catch words like “From (and tv show, etc.)”. Because of this some matches were getting left out. I fixed this by adding another regex pattern that removes this as well.

Devlog 36:

I am almost done with the first phase of the integration. When the download button is clicked inside of Spotify, Spicetify intercepts it and now posts a request to the open web server. Then the python script gets notified of that request, decodes it, then automatically starts downloading the song/playlist/album. The only thing is that the myMusic app has to be open while doing this, but I will remove that requirement in the future (not now). The next step would be to add status reports to Spotify so that the user knows what the status of their download is.

Devlog 35:

The app now creates its own server on a very distant and far port so that your other local host stuff doesn’t get affected. I did this with uvicorn and threading, so that creating the port doesn’t make the app freeze. With this functionality the app can get the requests made inside Spotify and automatically download them. The next step is to get the app to report the statuses of the download jobs (queued, downloading, completed, failed, etc.). Then I would actually have to connect both parts, let Spotify send messages and the app receive the message.

Devlog 34:

I have been working on creating a queue for the requests. This ensures that the user can download multiple songs at once without having to wait for the previous ones to be finished. So far I have been able to create a request id for each request. The app can update the attributes of each request, such as its status, and error message. On top of this once a request is completed the app also automatically removes that request.

Devlog 33:

The app is now able to understand and automatically download any received links. Now the problem is that Spotify isn’t able to send any normal requests to via the computer. Because of this I need to find a way for Spotify to send and get data. I know that Spotify (using Spicetify) can send and get http requests. Because of that I am planning to use FastAPI and a local http server to be able to send and get data so the app can automatically download tracks. I will ensure to keep the server in a high index to ensure it doesn’t get in the way of any other local servers.

Devlog 32:

(sorry for the late post)
I have been working on a really cool feature for the app. Basically what I am trying to do is integrate the myMusic pipeline into the windows Spotify app itself. When you right click a playlist or album, there will be a menu that pops up. Usually for normal Spotify users (not on premium) there will be a greyed out download button. What I am trying to do is two things: add a download button for tracks (because there isn’t one right now), and make the playlist/album button actually work.

Right now what I have accomplished is adding a new button for the track and making the playlist/album button clickable. I was able to do this by using Spicetify and its framework. Because I already have it installed on my computer I am using it to test everything out but later I will bundle it with my app. What I did was build a custom Spicetify extension (I had to learn JavaScript) that creates a new button.

I was able to make the original playlist/album button work because it was only “aria-disabled”, not actually disabled. This means that it was only disabled at the UI level, but Spotify still recorded clicks. By applying some UI changes I was able to make the button look enabled and still record clicks.

My next step is to actually hook this up to the Python script. I think this is a really cool feature.

Devlog 31:

This devlog was mainly focused on the efficiency and speed of updating the app. Before the data would come in really small packets, meaning the cpu would have to write to the drive about 130,000 times in total (I know 😭). I adjusted this so it now comes in larger packets so the cpu doesn’t have to do more work. I also adjusted the compression and a few other Inno Setup settings (number of threads, etc.) to make the app more efficient in general.

Another issue that I found is that once the app updated (specifically in Windows, idk about other platforms) the settings app would still show the older version for some time. This was an issue with the timing, so I had to adjust some of the delays to get it to work properly. Overall I think this is a pretty good update.

Devlog 30:

I FINALLY implemented playlist and album support. At first I thought it would be simple, just looping though each song in the playlist/album. Oh was I wrong.

First I had to add two more methods that looped though each song, one for playlists and one for albums. There had to be two different ones because each returned metadata differently. Then I had to kind of change the format of the whole thing, because I had to store all of the songs in a list, but currently I was only returning dictionaries from all of my methods.

After getting all of that sorted out I had to actually fix the GUI/downloading part. I had to update the downloading method to handle playlist/album links, then I had to properly handle all of the status message for each song (“Ready”, “Downloading”, “Failed”).

Finally I had to work on fixing the bugs I had introduced and handling the edge cases. After all of that I FINALLY had playlist and album support.

One thing that was sorta tough was handling the status for each track individually. Before I was just looping through the entire list and setting all of the statuses to the same thing, but I was able to create a new method and change each song status as it was being downloaded, not for every song as a whole.

Devlog 29:

This is kind of a messy devlog, so I’ll lay it out as best as I can.
First, I update the UI in preparation for playlist/album support. Before it was three static text fields that could only display one song at a time. Now I replaced it with a scrollable frame, with the ability to add more sections inside the frame for more songs. This frame also has the ability to display metadata. I also introduced a status message that shows different messages and colors depending on the status.
While I was doing that (and I am not done with it), I decided to stop in the middle and fix another bug. This bug (that was reported by other people as well as my testing) prevented good matches from ranked properly. This was a problem with how I compared the titles in my ranker. To fix this I added two new functions, one that normalizes the candidate title, and another that normalizes the actual Spotify song name (mainly to exclude features and get just the song name). To do this I messed around a lot with regex patterns, so I learned a lot.
Now I have to go back and continue working on the new UI for playlists and albums.

Devlog 28:

Based on feedback from my first ship, I have decided to introduce Linux support. I thought it would be easy, but it turned out to be a little harder than I thought. The first thing I had to do was rewrite some of the Path code that saves the users download directory. This is because Linux and Windows have different file systems, and I had to account for both. Then came the hard part, Packaging.
For ARM, I decided to do this on my Raspberry Pi, where I downloaded the project files from git, downloaded the OS specific FFMPEG and Deno, then repackaged the app as a .tar.gz. I decided to do it this way because this is sort of simply an experimental release.
For x64 I decided to use GitHub Actions, which allowed me to download the dependencies and compile without having to spin up a new Virtual Machine. I also decided to keep this as a .tar.gz.
Later, once I get more feedback about the Linux versions, I am planning to release a more universal app package (.flatpak).
I also had to change the update checkers, because each OS version has to check its own corresponding file.

In the end this was a very unique experience and I learned a lot of things about packaging apps on Linux.

Devlog 27:

This is a pretty major update. The myMusic can now automatically download and install updates, without the user having to go and download the github repository. Before, the user would manually have to go and download the installer from github, and set it up like a completely new install. Now, the app itself has a “Check for Updates” button. This button checks the github for the latest version, and if there is a newer version available it gives the user an option to download and install that. If they change their mind during the download, there is a cancel option as well.

This works by first getting the github version (latest), then seeing if the current app version matches that. If it does, the app is up to date. If not, then the app can fetch the executable using the github API and automatically download the latest installer. On top of this some inno setup tools allow the app to close the app, install the new version, and reopen the app by itself. The only caveat is that this only works starting from v1.2.0. This means that if you have an older version you will have to manually download v1.2.0 in order to start getting automatic app updates.

Devlog 26:

Added a function to help with auto-downloading new updates. I haven’t hooked it up to the UI yet, but that is my next step. The function takes in the name of the installer and the link to download it. It first downloads the installer to a temp file, then once everything is good it saves it to a real executable.

Devlog 25:

This update was about implementing the update checker from the last devlog. Now there is an in-app button that allows you to check for new updates. The implementation was similar to all the other buttons and didn’t take too long because I already had the function ready. The next step is to allow the app to actually download and install the update, instead of just pointing out it is available.

Devlog 24:

I have started adding implementations that checks for the latest version of the app from github. It uses the github API endpoint for my specific project, then reads the json to get any version information. Using that we can compare the current version to the latest version on the github to see if there are any updates available. Currently I have only created the function and am yet to implement this feature into the actual app/GUI.

Devlog 23:

Today I added the feature that embeds cover art into the mp3 files. To do this I had to first use the requests library in Python to get the image itself. Instead of downloading the image I kept it in memory using the ‘io’ library. This made the metadata tagging safer and faster. Finally the app uses mutagen to embed the cover art into the mp3 file. I used a try and except loop, which ensures that even if the cover art adding fails the mp3 file still gets returned, just without the art.

Devlog 22

This update was about further increasing the matching strength. Before the program would automatically download the highest ranked candidate, even if it was very low. Now the app downloads the highest ranked candidate only if its ranked above a certain thresh hold. If there are no qualifying matches then the app simply returns a no matches found error.

This wasn’t that complicated, but I had to modify a few functions so that the score was returned properly and in a usable format.

Devlog 21: Now added real YouTube (YT) to the match finding instead of just YouTube Music (YTM). Before the songs were found only on YTM, but I realist that normal YT has a larger list of songs, so I decided to add that to the algorithm as well. Now, the program takes 3 songs from YTM and 3 songs from YT, and uses the ranking function I created before to sort them from best to worst. Then the program starts by trying to download the best match. If it can’t, it goes through the list until if finds one it can download. If none of them download, then it sends an error message. One thing with this step was that I tried not to repeat big chunks of code, as a lot of this was similar to the original YTM downloading method.

Devlog 20: Added the Spotify OEmbed fallback. Now even if the SpotifyScraper python package fails we have a fallback to get as much metadata as we can. This wasn’t that hard, as I already had the file from a previous time. The hard part was accounting for all the different cases, such as the title being missing or an empty string. I had to account for all of those while integrating the new file.

Devlog 19: Implemented the ranking feature from the last devlog. Now, the best candidates are ranked in order from best to worst, and that is the order in which they are downloaded.

Devlog 18: I implemented a way to start scoring the candidates returned by YT-DLP. This ensures that even if the first candidate returned is not the best, the program can automatically filter out the bad ones using key words and ensure the best candidate is always selected. I have only coded this function for now, and I have to wire it in to the real app soon.

Devlog 17: Did some major code cleanups. No new features, but made the code more efficient. Also reorganized some files. The hard part was keeping the functionality the same while trying to make the code non-repetitive and more efficient. I also tried to fix any warnings that were present in the code that were caught by the IDE.

Devlog 16: Simple update, but I added a new error message to account for a yt-dlp botting error. Now, when YouTube sends an error saying that the request was blocked due to botting being suspected, it sends a properly formatted error message instead of the raw yt-dlp text. I retested everything to ensure the previous functionality of the app remained the same.

Devlog 15: Improved song selection algorithm, so it selects the next best track if the first selection is invalid. Before if the first track selected by YT-DLP didn’t work/was invalid, then the whole program would fail. Now it moves on to the next if the first one doesn’t work. I also update the progress messages, so it shows which match it is trying.

Devlog 14: I have implemented two new features. First I implemented a feature that allows users to switch the directory where songs get downloaded. This took some time, but I was able to get it in the end. It works by creating a new button which opens a folder picking dialogue. The path that the user seclects inside this dialogue then gets saved to a local json file. Everytime the user changes the path it gets saved to the json, which allows the program to always open up the last directory selected by the user. I had to implement some checks to ensure that the program would work even if the json file was missing or corrupted. The second feature is more of an update. The static “Downloading” progress message is now updated with real time info on what the program is doing. The next step would be adding a dynamic progress bar.

Devlog 13: I added an installer .exe file, added screenshots to the readme, and added a new release on GitHub. Configuring the installer was a little hard, as I was new to this process and didn’t know where to start. I used Codex to help me figure out that I could use a program called Inno. This handled the installer file, uninstall, as well as configuring it with Windows. Codex guided me through how to set that up, and eventually I got everything working. Finally I created a new version on Git and pushed everything.

Devlog 12: I compiled the app into a .exe, updated the readme, and pushed the first version to git. I ran into some issues when compiling the app. Specifically after the initial compiling YT-DLP couldn’t see the location of FFMPEG. Because of this I had to update the “get_app_root.py” file so it accounted for the file changes when compiling.

Devlog 11: I added a preview button to the GUI so that users could see the song details and confirm they were right before downloading the song. also prepared for packaging this file into an executable (.exe).

Devlog 10: Added app.py and introduced a GUI window using Custom Tkinter. Now when users run app.py they see a new desktop window open up, see a text box and a download button. Once they paste in a Spotify link they see status messages like “Downloading”. At the end they also see the downloaded path. On top of this if there are any errors during the download (like fetching metadata), this also displays as a status message. This took a little bit because I get to get familiar with Custom Tkinter, add another function that gets called by the button, and add some safeguards/quality of life updates. The main safeguard was disabling the download button while the song was downloading, which ensured the user didn’t spam processes. The quality of life update was that the download path now wraps the text to multiple lines if the path is very long.

Devlog 09
Simple update, but it now uses Mutagen to add track name, artists, and album name to the downloaded files.

Devlog 08
Introduced FFMPEG to the project to be able to convert downloads from .webm to .mp3 files. Also change the song title to be the title of the song. Introduced a file check to ensure file names are valid and accepted by Windows.

Devlog 07
The program now automatically uses YT-DLP to download the best match found. This works by using YT-DLP’s build in “Download” method. I pass in the number 1 match found by YT-DLP (built in the last devlog), and it downloads it to the users default “Downloads” folder. This is a major step in the project, and I am almost ready to ship the first version.

Devlog 06
Major Update: Now added YT-DLP to the project. Now whenever you put in a valid link you get more than just the metadata. You also get the top 5 best matches from YouTube Music. This took a while because I had to learn to properly call and use the YT-DLP package in python. I created 3 helper functions. The first created the search query using the metadata, the second returned the search results, and the last one formatted and returned the results. I created one last function as a wrapper, such that when I passed in the metadata it returned the YouTube Music search results. When calling yt_dlp.YoutubeDL(), I got a type error when passing the parameters. This was because the package expected its own datatype, while I was simply passing a dictionary. I was able to fix this by setting the type to “Any” when passing in the argument. I also changed main.py to include the YouTube search results. Now, whenever you pass in a Spotify link you get the song metadata as well as well as the top 5 best matches from YouTube Music.

Devlog 05
This is a pretty small update, all I did was add a small test file to be able to consistently check for errors and functionality.

Devlog 04
This was a big change in the metadata retrieving/handling. Last devlog I explained how the new restrictions on the Spotify API made it difficult to fetch the metadata, so I had to use the public oEmbed API. This didn’t give much information and was hard to work with. After researching some more I found a python package that does the metadata fetching for me, “SpotifyScraper”. this package takes in a spotify url (which the user will be providing), and can get information like song names, artists, duration, etc. This was very useful for me when coding the metadata functions. On top of this it supports playlist and album inputs as well, which is great for future updates. I spent this time rewriting the metadata retrieving functions and cleaning everything up. I removed a few redundant functions and introduced proper error messages.

Devlog 03
After getting the link cleaner to work, I wanted to use the Spotify API to get the metadata of the song. The problem with this is that Spotify introduced heavy limitations on its API earlier this year. Because of this I had to fallback to the Spotify oEmbed, which gives only provides the song name. I am still researching for more options.

Devlog 02
This took a while, but I was able to make the program that strips the input down to only the Spotify link. This ensures that (for the most part) that only the Spotify link gets used, and not any other whitespace/extra characters that might have appeared. It took a while because I was researching and learning how to use the ‘re’ library in Python, and it took some trial and error/debugging to get the logic to work.

How this works;

1. The user inputs into the text field
2. First the type of track gets identified (single track, playlist, album)
3. Then it uses the ‘re’ library to detect the pattern of Spotify URLs and URIs. This ensures that the Spotify track is actually valid.
4. It then uses the ‘.group()’ function to capture the matching section out of the input
5. Finally the functions are checked to ensure everything is valid, then the type of track and song url are returned.

Devlog 01

This is the initial commit for the myMusic project. I created the file system locally, then opened the project inside IntelliJ Idea. I kept things simple and only introduced a README for now. I ran into a few problems when pushing to GitHub. Mainly, the .gitignore was being placed inside a .idea folder (I think because I was using IntelliJ). Because of this I had to go back into the IDE and move the .gitignore into the main branch and not into a subfolder.