SarSynth

So in the past half and hour I did a lot and yes this is actually my final devlog this time.

Ok so I finished the css around like 100-200 lines of code but it was light because the color scheme sorta connected with the theme anyway… I ran into a few alignment errors like it was blocking my whole screen but I fixed it so thats solved. I also updatted my readme about the games.

Below I added a picture of my finger guide SarSynth it just shows how all your fingers should be aligned and for what and personally dha and ni is the hardest but with practice (like 5-10min) you can get used to it.

I made it with Canva it took like 2 Minutes because the pictures I didnt really count towards my time just the making and alignments were counted.

Overall thank you and goodbye.

I fixed the Tanpura instrumental background track and now it works more properly because last time it was glitching even when I spammed it so it is fine now just a small JS error.

I also finished the games logic its 3 simple games which you will come to know soon…

In total it is about 250 lines of code which is fairly simple but the games are actually nice in my opinion. And soon I will be making a sort of guide on how to position your fingers for each note which includes Sa, Re, Ga, Ma, Pa, Dha, Ni, Sa. But tbh I have everything thats needed but it doesent hurt to give a bit of teaching because honestly even though I made it, it was pretty hard doing the dha and ni because my hands were keep forgetting the placement but after some time and practice I got it (10 min lol)…

So lets see what happens next…

Byee and ADioss

Ok so my README.md is done after like 1 ish hour in total. I incorporated as much as I could including a license in the bottom and honestly this is my first time not writing it raw and actually using hashtags and asterisks to make the text look better in the readme so it went well.

I hosted it and was going to ship it. I showed it to my sibling and my feedback was “can you make it not that fun its kinda boring” so that obviously struck me inside considering the effort I put into this so what I did next was crazy.

So as you know I was gonna ship it but instead I mapped out a way where I can add small games into it like GUESS THE RAGA and more which I will incorporate later.

So sorry Guys I will need to add a bit more before I can ship it but I got this.

Adioss.

The front page was zoomed in a bit which was because of the css which I figured out after going through the CSS so that I fixed.

Claude helped me a ton just now because I told it to make me 8 different panels in the front page which lead to a page and claude did that which was nice but for some reason it wrote 600 lines of css which is wild because there is not much that was added just colour and a bit of shadows but its fine because its just panels.

OK then off hackatime I did my own little research and basically MY IDEA was to make a panel of each instrument that was incorporated in this project and write a 1 paragraph historical perspective of each instrument and overall it worked out pretty well but AGAIN a sizing issue occurred due to claude and I told claude 4 times to fix it but it could not for some reason And then I went in and figured it out lol it was the most funniest mistake…the css was on 2x zoom and my browser was also on 2x zoom for some reason.

So yeahh… that has been solved now

I also worked on my readme a bit like 2-3 sentences ish.

So overall in these last 25 minutes I got through a lot of progress…

Over the past 45 Minutes or so I have been writing my readme and organizing some files.

In all my other projects my readmes were always the worst part of my project but not anymore starting from now. I learned how to write readmes with more efficiency by adding # and * which basically makes text bigger and also bold or italic.

So far I wrote the overview on what my project actually is and I have also wrote some of the philosophy and design goals which include data in jots. I also completed the tech stack and computer vision data which is in the same section as the design goals.

Overall I am like 20 ish percent done the Readme and my folders are properly organized and I am done that section now.

TODAY WAS CRAZY and this may be my last devlog

I’ve spent today creating the entire visual backbone of the app, and now finally feel that the UI actually has some personality and doesn’t consist of random pieces. To start with, I defined a complete theme in :root – all golds, teals, ink shades, subdued overlays, panel colors, and two font stacks – so all the elements of the app draw colors from one consistent palette. Next, I created the layout guidelines for html, body and #root, and implemented the backdrop system – mirrored camera feed, vignette gradients, and stage layer below the canvas. Finally, I added the top bar styling, dropdowns, hint text, icon buttons, and tanpura toggle styling using the gold/teal accent language. I included the UI for the pose list, the readout section with Devanagari + monospace swara display, and the stat blocks with small capital letter labels. And now, I have the help and settings panels with blurred glass background, gold section headers, chips, toggles, and sliders.

In addition, I designed the footer banner and the whole start overlay consisting of the brand logo, tagline, buttons, loading animation, error message, and legal disclaimer, so that the app can have a nice onboarding screen. Last but not least, I provided the full responsive layout for small screens so that UI elements will be reorganized on mobile devices properly. Overall, this stylesheet became the core of the application UI design.

SO overall it works pretty well and its amazing and I made it indian because I am personally indian. So overall this is prolly note my last devlog but LETS SEE.

Adioss.

HII guys so heres what I did in the last hour…

I spent a good amount of time was spent working on the design of the interface. Finally, I would say that now the interface looks like a finished product rather than some temporary solution. As the first step, I set up a complete theme via CSS variables in :root. In particular, there were colors for the background which should be dark, golden accents, teal expression color, ink text, muted overlays, as well as font stacks of Devanagari and monospace. All those settings in one place immediately gave the interface much better structure and increased convenience of further editing. Further, I adjusted the global layout to make the app take the full screen and have a nice dark mode design, fitting into the overall musical concept of the project. Then, I designed the camera backdrop including the mirrored video feed, vignette gradients, and stage layer. It provides the project with a moody and atmospheric design. After the backdrop was completed, I turned to designing the top bar which includes the design of fields, labels, dropdowns, hint text, and icon buttons.

I added select styles with gold arrows, blurring effect, and transition on hover to make all of these elements responsive and clean. Then, I created the tanpura toggle button with its teal glow and devanagari title, which fits very well into the overall Indian classical theme. Finally, I added styles for the pose list and pose glyphs, which include small circles that represent poses, pose names, and subdued note labels. In short, this was the process that turned the plain UI into an actual design.

And now I also tested it BUT ima keep it a surprise for yall so I guess stay tuned for the next devlog loll adioss. (I pasted a few of the theme colours in the code below…

So heres what happened…

In the last hour Claude generated well over 600 lines of new code, covering roughly 70% of the work, and while it dramatically sped me up the alternative was that my entire project instantly threw errors because none of it was connected together yet. While trying to make some sense of the chaos it dawned on me I had an entirely abandoned folder within my project called Components that was full of React components – HelpPanel, Readout, SettingsPanel, StartOverlay, TopBar – that I’d previously created but never attached to anything, and the kicker was my project wasn’t even set up to buildReact.

My package.json didn’t contain react or react-dom, tsconfig.json had no JSX support, and my main.tsx was just vanilla JS with a button, and a tag masquerading as a UI element. So I decide to just bite the bullet and hook up React to my project for real. I install React, ReactDOM, and the types for both. Add a vite plugin in a new vite.config.ts, update tsconfig.json’s jsx compiler option to “react-jsx”, write an actual App.tsx that owns the top-level application state (my Performer engine, configuration, the live snapshot of what’s playing, and the state of the camera module) and passing that down into all the components so that the top bar, readout, panels and start screen actually did things.

I wire it into my existing drawScene() function via the canvas, I implement some keyboard shortcuts for notes(A-L), octave raises/lowers(Z/X) and tanpura playback(space). I also implement pointer support for playing notes via click and drag without a camera attached. Two things I spotted and removed was a React StrictMode wrap, which would immediately break AudioContext and my handtracking model by remounting them when they just shouldn’t. I also had an import mistmatch between App with braces and a default exported App.

Lastly I added a real index.html with a where React mounts and styled the top bar, readout, panels, and start overlay with a whole new styles.css. I was able to clear down all of Claude’s errors, get TSC output zero errors(tsc –noEmit) and build.

Today’s process of working on scene.ts has certainly been one of the most complex parts of my work, since this file involves practically every visual and interaction mechanism I have designed. The imports for degree poses, snapshots and hand tracking points have been implemented and the constants for melody and expression colors, Devanagari fonts, monospace labels and the whole hand skeleton map were defined. Also, the SceneLabel and SceneInput interfaces were implemented; these two mechanisms serve as a sort of backbone for all the information needed by the renderer, such as hands, poses, thresholds, reachable degrees, video size, etc. Then, the geometry helper methods such as roundRect() and ribbonGeometry() were implemented; these methods determine how the swara ribbon would be drawn and how every cell would scale based on window size. Lastly, the drawScene() method is now partially implemented; at the moment it takes care of projection from camera coordinates, drawing of the octaves ladder, ribbon itself, intensity meter and conditional drawings of both hands with landmarks, tone lean arcs, gamak waves and role tags.

However, since this file is extremely large and complex, it results in numerous TypeScript errors such as undefined functions, type mismatches, unresolved references, and half-written helper methods like drawPoseGlyph(), which are used before the rest of the code is written. Some errors stem from hand tracking processing, some come from the ribbon rendering code, and some are generated by the math code responsible for resizing the video stream. At this point, the code has reached that chaotic state when the structure starts taking form while the compiler throws numerous errors due to the incomplete file. However, despite the chaos, I have managed to make the ribbon rendering code work, the octave ladder operational, and the hand skeleton drawing code properly working, which means that the visual layer is already being created. I am still working on the code, and hopefully, when more code will be written, many of the above errors will disappear automatically.

But yeah overall I fixed them overtime so now I am gonna start the COMPONENTS section around 200+ lines will be written I hope unless an error occurs lets see!!!

So I was busy getting down deep into scene.ts, where I’m about half way done with the code necessary to render the hand tracking visual layer. Scene.ts is huge and highly interconnected, bringing in pose definition data, Devanagari/roman naming, ribbon geometry data, skeleton structuring, projector calculation, and all the UI controls to display swara, articulation, gamak, tone lean, and expression-hand behaviors. I got all the constants wired in, the skeleton mapping, the tip detection set, the margining mechanism, the complete SceneInput interface, and I began wiring up the drawScene() function pipeline responsible for the octave ladder, ribbon, intensity meter, and hands overlays. Despite having a good structural backbone in place, I’m encountering a crazy amount of errors – some from TypeScript whining about type mismatches, some from missing imports, and some because drawPoseGlyph() or some other functions aren’t written yet but referenced. There are just too many red lines at this point that I cannot tell which are legit obstacles, and which are only temporary noise from not-yet-written parts.

However, the basic rendering code is coming together, the projector calculation works, and the skeleton hand-drawing process seems to be halfway there. I’m not finished coding yet, and so I’m hoping that once things come together, a lot of these errors will clear up as the file is completed. Right now, the code is quite a mess, but at least I’m getting somewhere, and the whole scene code system will become operational.

So lets see how it goes see ya’ll later adioss.

The day became one of those big pushes where each individual piece I worked on became part of a much bigger picture, and it truly seems that today marked one of the largest leaps forward in the entire process. Claude assisted me in my work throughout the day not only with small bugs but with several large problems standing in my way and thus allowed me to successfully stabilize the entire workflow. Once I managed to deal with the problems, I returned to main.tsx and properly fixed the rendering flow along with ensuring that the components were mounted correctly. The key accomplishment was managing to get the performer.ts file fully rendered and working, which is a great achievement considering that this particular file deals with handling the camera feed, recognizing finger positions, and turning those positions into musical gestures.

The moment that it actually worked was a feeling of unlocking another level of the project altogether. My entire objective with this application is to create something where one hand can perform the Indian classical notes of sa, re, ga, ma, pa, dha, ni, sa using the finger position to do so, akin to the bansuri or sargam software, and at the same time having the other hand trigger background sounds such as the tanpura drone sound to produce the atmosphere of the Hindustani classical music. The achievement of making the logic of the camera recognize and respond to the gestures makes the entire idea into a reality. This session is definitely one of those milestones since with the main.tsx file sorted out, performer.tsx rendered successfully, and the gesture pipeline behaving the way I envisioned it, there is no stopping what I want to achieve next. And again claude helped soo much in giving big code blocks especially when there were large errors.

SO overall great but it could go better because right now it looks like crap.

Today, I ended up doing more backend setup than what I had expected to do today, in that, after executing the terminal commands to set up the project environment, there were created two files, namely package.json and tsconfig.json, totaling over a thousand lines of configurations. Although mostly being boilerplate, the creation of the two files marked a huge milestone for me in the sense that the project had now become a TypeScript-based one rather than the earlier looser one. After doing that, I then proceeded into the main codebase to modify main.tsx by refactoring the old code. However, my major achievement was the completion of hands.ts, which is one of the harder pieces of codes in the project, in that it deals with hand-tracking, gesture recognition, and all the movement-related logic in the project.

I feel really satisfied in doing this file because it is one of those modules that would allow us to implement lots of other features in the future, and seeing the results of my hard work compile successfully after so much setup was truly satisfying. After implementing all the configurations, making changes in the main entry point file, and completing hands.ts, things went pretty well.

Overall everything worked out well and I finally figured out the issue lol it was because I did not have these packages so thank you to Claude because it told me what to put in terminal.

Today ended up being one of the most efficient, yet very chaotic coding sessions that I’ve had in a while, because I have managed to finish the entire audio file and the entire engine.ts file, which is essentially one of the key aspects of my entire project, and grew to become more than 700 lines long at the end. Even though I was able to get a lot done, the process was far from being easy, as I encountered some really stubborn errors that would stop the project from running entirely, which were mostly connected to some problems with main.ts file and the Vite configuration, which I did not even know was misconfigured and causing silent problems in the process. I spent far too much time on figuring out the real reasons for those errors until Claude helped me to finally get an idea of what’s really going on. In the process of debugging I found some duplicated code in the engine, which made me quite angry, since it resulted in wasting a lot of time chasing some bugs that occurred due to my mistake when copy-pasting code earlier.

Now that I managed to get the engine running at last, I created a little localhost test where I play a sitar sound in order to make sure that the audio stream is functional in real life, and hearing the one little note played made me feel so relieved, knowing that the whole thing really works and not just some theory written in the folder. Now that I have finished all that, I am going to continue to work with hands.ts, which is the part responsible for detecting the hand movements and gestures, and I am excited to see what happens. I also wanna add more instruments which will not be hard but lets see btw I also added a tanpura soo lets see how that goes.

See you guys in the next devlog lets see what happens next…Adiosss.

Today was quite productive – more so than I anticipated at the start – and I finished with something on the order of 400-600 lines of code, most of which were devoted to implementing the entire music module in full with its poses.ts, ragas.ts, and swaras.ts, which I have put together in a clean and working fashion without any odd edge cases and inconsistencies. I think it was very satisfying to see that I had finally managed to put all of the pieces together into one coherent system. Next up was the JavaScript part of the audio pipeline which was much faster to implement than I anticipated because once I figured out the general architecture of it, I managed to do it pretty quickly. Claude helped me a little here and there with some hints and clarifications, but the actual implementation was relatively simple after all that. Today turned out to be one of those productive coding sessions where you just seem to get a lot done and end up with a very well-structured and organized system on your hands.

So yeah overall I am like 15-20 percent done I think but its really based on how the rest goes especially the camera tracking my hands lets see.