@mitul

I blew a capacitor.
For the first time in my life. Since last update, I’ve been trying to set up the demo with perf board, and realized the I2C lines were inaccessible. So giving up on that, I decided to breadboard it out. This would have been fine, if not for the fact that i took apart the entire breadboard for travel, which when I tried to reassembled, I must have messed something up, because next thing I knew there was a short in my circuit and the pwm driver had blown.
From this point, I made a decision to build my own hand, from scratch, that is robust, easy to assemble, modular, and best of all… my very own. Thus, this session was spent researching, how to approach building a robot hand. From prior context, I decided to not build a tendon driven hand, given their abundance in the robotics community, yet lack of implementation in real scenarios, mostly due to wear issues. Looking at the best hands in the world, I noticed most if not all of them were direct drive, and I am particularly a huge fan of the wuji hand. So, I’ve begun the design of a finger, using mini dc actuators, embedded withing the finger, allowing for a thing hand, that is precise and strong.
Of course the major drawback, I am not adressing is that the use of embedded motors makes the hand enormous. Like 6inch pinky enormous. In fact, I searched it up, and turns out the hand might be larger than Wemby’s. But that’s all right with me, as long as it works.
Anyways, I’ve attached a photo of the wuji hand, which I took inspiration from. But thats all for now folks….

HUGE PROGRESS

This session was spent, designing and assembling the electronics enclosure for the hand, and getting the full working demo. to be compact.

When the hand turns I noticed that servo cables would turn with the hand, which made accessing certain wrist positions risky, due to the nature of the wrist design. Thus I needed some way of reduceing the number of wires connecting to the hand, such that it can be more mobile. To do this, I decided to mount the pwm driver and power source directly onto the back, which I designed a bracket for.

Because the hand is shaped so oddly, it took me several iterations to find a mount that would screw readily, and the pcbs, the way I inteded. And while, I would send these designs to print, I worked on the electronics. I put tested the complete human to model to hand pathway, and I realized the whining sound and stunt earlier was likely a result of my battery supply. Thus, I switched out the battery for a wall dc-dc power supply, and was able to sustain the hand without a crash. At it’s peak the servos only drew about 1.2A at 12V, so perhaps the issue wasn’t the battery’s current capability, but instead how quickly it could provide it.

While I printed these iterations, I noticed how the elctronics on the breadboard were functional but fragile, so I decided to take on the UNNECESSARY decision to turn my electronics into a perfboard. Worst mistake ever. Who knew a perfboard could be so troublesome. Almost every 5 minutes, I’d either bridged a connection, soldered ground to voltage, layed out components in the wrong order, or just burnt myself. Even worse, I was laying on the dirty floor, and my shaky hands and burning elbows have left me more sore, than I’ve ever felt.

After some 4 hours recorded, and 3 hours off camera, I got perfboard wired up. Because, I’m almost at 10 hours, however, I’ll share whether it worked in the next log.

Good news and bad news.

Bad news is that this session was supposed to be a recovery log for the 4 hour lapse lookout accidentally deleted, except somehow the recording failed again.

Good news is that instead of waiting on that fix, I’ve kept working, and we’ve made some huge progress. The wrist design you saw last session was pretty neat, except when I finally got access to the hand to screw it on, I realized my screw tolerances were not enough. And worse that caused the screw to drag across the bearing of the wrist leaving that screeching sound, when a chalkboard is scratched.

So I fixed the finicky piece and reprinted it. In the meanwhile, I put together the electronics for the hand, and added the code I’d developed last night. After renumbering the pins, and messing around with stepper speed, I’ve finally gotten to a decent teleoperation phase with the wrist. It’s quite slow, but it works.

The only thing that concerns me, is that when I ask the stepper to move to a position to quickly, I hear a peep, then the fingers don’t move. Most likely somewhere downstream, overcurrent is preventing the stepper from drawing too much torque. This makes sense given how shoddy the wiring is…

Anyways, I’m about to rerecord the alphabet with my new wrist, and hopefully get the full demo working with wrist by next session. I will then hopefully, record a successful recovery session for my precious 4 hours of work ;(

That’s all for now folks…

This session is technically one in the future, because the sesison before this, I plan to log after this post. It’s kind of tricky, but basically lookout’s recording failed because I closed the tab, so now I’m rerecording the old session, while I work on the new one.

Mostly, I spent this time working on 2 things. One I noticed that my wrist, would rotte the entire platform, when rotating the hand, due to the rotational inertia, so I designed a brace to fix it in place. The other thing I did, was develop code for the wrist, so that I could add wrist encoder data to my sign language library, which I actually plan to record in a little bit.

Obviously this implies that last session, I designed the wrist fully, and printed it out, but you’ll find about that, next devlog…

That’s all for now folks.

This log is something I don’t do usually, which is a recovery log. My very first devpost, I ran into the issue on lookout where my video timelapse would save on lookout, but not add to hackatime, since the original recording link got closed. I was very dissapointed with this, as I’d lost my very first work session, which too was four and half hours.

Looking through the slack, it seems other people have had the same issue, and some people indicated that they solved it by playing out the screen capture on slow motion, for lookout. I implemented this, throughout this session, except for an hour of this session, the video was paused, hence the extra time. I hope this isn’t a huge issue; it wasn’t intentional.

To summaraize what the session was actually about, basically, I recreating my original component diagram, within a new project. This was because, the original, I felt, had many flaws, like for example the lack of net classes that were organized. Also, I was reconsidering the layout work of the original pcb. Mostly the work was transitioning my pcb into a new project folder.

Anyways, that is all folks…

This work session I added test points. This was something I noticed on many designs, especially for prototype boards, and I decided to implement them as well. This was tricky however, as my board is already incredibly dense, and test pads can take up considerable space.

After adding the pads, and re-updating my pcb from schematic, I noticed that all the 3d models were gone again. And worse, the custom footprint I designed for my bluetooth chip was also gone. The footprint for the bluetooth chip was especially tedious to make, as I added a bunch of zone rules like no copper fills and components, and such based on the application circuit.

This was and is frustrating, but I think the reason, was because my edits were saved to the pcb rather than to the footprint library, which meant refreshing the pcb page, lead to the changes being lost. I plan to fix this, by getting more familiar with the footprint editor, so my work doesn’t get lost.

Speaking of which, the next journal will actually be a recovery journal, as early on this project, I lost a four hour session due to a lookout recording bug. So, I plan to play out the screen recoding of the 4 hours, which has to be fair game, given how this is the SECOND time, I’ve lost a 4 hour session.

Anyways, that’s all for now folks…

Alright folks…

I know I already shipped this project, but I kind got an unexpected lead. So the reason, I wanted to leave off this project is because additional improvements, such as thumb dexterity or wrist movements would add rabbit holes that deserved projects of their own. But, the person who lent me the hand, also found the missing thumb; it does lack a shell, however. With that, I figured, I’m pretty close anyway, why not add the wrist?

So this work session was spent, predominately, designing a wrist mechanism where a stepper motor could rotate the hand into various orientations. The one thing, that concerned me from the start, was that stepper motors lack positional awareness. At least the kind I’m using does anyway. This means I would need to design a location for an external encoder to fit, that would stay fixed relative to the palm.

This design has been very tedious. The hand is already so compact, that the encoder would not fit, so I am thinking, I can attempt the stepper motor based wrist without encoder values first, and if i detect too much drift, then worry about that addition later. I’ve finished designing the wrist, and am about to send it to print and test fit. Next session, I will find out how that goes.

See yall soon…

Folks, this will be the final devlog. For this project anyway…

This session wasn’t too much involving actual additions to the project. Following last session, I was brainstorming ideas to expand this project, perhaps to make this a more robust system. In this search I encountered IsaacSim, a simulation software that enables the simulation of robots, which could allow for expanded training to grow the model.

But, in all honesty, I wasn’t too interested. Sure, I could add a wrist or thumb, and add more gestures, perhaps train a custom computer vision model, but the point of this entire project was to develop a rapid proof of concept. Given that I’ve achieved that, I’d like to invent my energy into developing a robot hand of my own.

That is what I plan to do next, however, that will have to be documented as a separate project, as it doesn’t exactly correspond with the purpose of this one. The rest of this session, I spent documenting my work, so check out the repo if you’re interested!

Anyways, that’s all for now.

In trying to redesign, this board, I’ve encountered several helpful suggestions in terms of routing and component choice. This session was spent fulfilling them.

First, for the SWD programming interface, I had initially exposed female socket headers, which were layed horizontal, and thus occupied tons of room. Instead someone suggested I use a tc2030 connector, which is significantly smaller footprint-wise, and looks pretty cool.

I also deliberated whether to use a 2 layer or 4 layer board, and after some thought, for this design I’ll be using a 4 layer board. Though a 2 layer board is easier to setup, it is harder to execute well, and certainly suffers from more pitfalls in terms of signal integrity.

Specifically, within the 4 layer stackup, I opted for signal-ground-ground-power stackup (same as last time), because I can dedicate a ground to analog and another to digital, as is common in audio applications.

Finally, I added 3d models that were missing (somehow again) for the footprints, leaving me with board setup for next work session.

That’s all for now folks….

This session completes my base project.

Last time I logged, I’d gotten a chatbot to speak sign language through the robot hand using serial. This means basically half of the interface was done. The other half was for the chatbot to interpret my sign language so it could adequately respond.

In one of my earlier logs, I mentioned finding a repo that supported fingerspelling, so you could spell out any word. This session I forked that repo, and edited the script such that, signs it interprets were spoken to a chatbot, and that chatbot would communicate outward through serial.

However finger spelling is easier said than done. The first difficulty was that letters were often misinterpreted, which was frustrating, but also a limitation beyond my control. The second difficulty was repeating consecutive letters. If I held my hand too long, the model assumed I intended for the letter to be repeated, which lead me to incorporate a cooldown. Finally to adress spacing between words, I set a condition where both hands must be present for over 2.5s, to include a space character.

With this the bridge was effectively complete. What I have left really is optimizing this process, by perhaps improving the hardware design of the hand to grant more freedom, or maybe training a more robust sign language interpreting model.

But all of these options sound boring to me. Instead I want to learn something new, so the second half of this work session, I spent deliberating ideas, and I came across IsaacSim, a simulation software for robots developed by NVIDIA. I plan to learn more about this topic, to allow for the simulation of this hand, such that I can attempt a more sophisticated approach to a sign language robot, than just static recorded symbols.

Anyways, that’s all for this session folks…

HUGE UPDATE.

I think last work session we left off when I had teleoperation working with the hand. Aka I wore some flex resistors, and the robot mimicked my finger movements. The next task, I mentioned, would be to add buttons to allow for recording and replaying symbols. This would allow me to build an alphabet library for the hand.

Today, I did that. I added the button to record and replay movements, and stored the values of the motor positions in a google sheet. And honestly, this actually worked pretty well.

The hand’s method of communication relies on spelling out words, rather than actually signing the words, mostly because there are more words, than there are letters. Also most letters are easy to sign, because they are static. The issue is that when spellling out with letters, spaces needed their own symbol to indicate breaks in the sequence; so I came up with a wave pattern that is unlike any letter, thus can be interpreted as a break.

Also throughout this time, I rewired my servo setup to rely on a servo driver breakout board(PCA9685), rather than manually connecting the PWM lines, and in the process I must have switched a couple channels, and my sign language library was dysfunctional. So, I rerecorded the letters, and finally, I could comfortably type phrases and receive outputs from the hand that were somewhat discernable, with the slight difference that my hand is missing a thumb of course.

Anyway, after this I set up an ollama agent and wrote a script to have the ollama agent output serial messages, which the hand received and interpreted. This was crazy to see in person! I would be talking to ollama, and it would respond through sign language. This project is moving forward significantly faster than I’d expected, so I think I’ll make it better. I still have the sign language to text part to handle, but afterward, I’m thinking of adding a wrist and a thumb to the hand, to improve the clarity of signed letters.

That’s all for now, folks, see you in a bit…

Mini work session. After getting the hand to work with teleoperation, I realized I would need to several things to get this project to work. Firstly, I need to record letters to use, which I plan on doing by incorporating a button system.

I’ve set up 2 buttons that default to a high state. When pressed, the low state indicates that the button is pressed. One button is the recording option, to capture the servo positions, I’m maintaining currently, and store that to the next letter in line. I plan to store these values in a vector, which can be accessed later by the LLM to choose letters to spell out words.

This session, however, I spent finding a camera based sign language model, and analyzing the source code. I am kind of putting off the vector thing, but that’s all right for now. I found this repo online, that works well for my purpose, and I plan to use it, or at least repurpose it.

That’s all for now.

Alright folks, second work session in, and we’ve made huge progress. Last time I left off with a working hand moving demo, where I had the servos go off in sequence to match a pattern. Right after posting that log, I discovered flex resistors, which are basically variable resistors that change value based on the state of their flexion.

These things are similar to ribbon cables, and luckily they came with little rings I could mount to my fingers, essentially giving me an analog method of sampling the flexion state of my finger. I had these resistors form a voltage divider alongside a 10k ohm resistor, and thusly I was able to read the variable voltage formed at the divide.

I routed the voltage values to the analog input pins of the arduino, and guess what? …. 0.0, 0.0, 0.0…..

And literally, I was losing my mind trying to figure out why, because when I check the multimeter, the value seems right. After a bunch of back and forth, I noticed that the read function for the voltage was in setup, and not the loop. So basically, the code was reading the voltage at initialization, and then that value remained static. Stupid mistake, but one I fixed it, it worked.

From there I mapped the resistor values to pwm output, and thus the hand exoskeleton was born. My task for the next work session is to add a recording feature on the exoskeleton, to hopefully capture hand poses, to then save as signs for the ASL library to access.

This session is the start of a new project: building a sign language interpreter using a robot hand. Thanks to a local startup, I got access to a robot hand, which I’ve been getting familiar with. My goal is to allow an LLM to communicate through sign language, while simultaneously interpreting motions made by other users. This would be handy in tasks that involve conversation with computers, which have traditionally from deaf people, as they may involve a high degree of technical and traditional literacy.

This work session specifically, I spent setting up the servos to be compatible with an arduino microcontroller. I’ve produced a script capable of moving the fingers in a sequence. After that, I tried building a c++ library to store the sequences that I would deem a symbol, aka a sign language library. But halfway through, I got off task, and found a sign language interpreting computer vision model, that I wanted to test out. I’m currently trying to install the dependencies, which has proven tedious, as the repo uses an older version of python.

That’s all for this session; this next session, I’m trying to set up flex resistors and get mimicry working…

Alrighty, this will be a pretty dense devlog, but it’s because I’ve been procastinating a lot of the work ;).

For context, the goal of this project is to develop an affordable bluetooth dac, that can also act as a standalone dac, and a bluetooth microphone. A dac or digital analog convertor is a device capable of converting digital signals to analog, which in my case is converting USB-C music to “audio jack” music. A common example is Apple’s dongle.

The point of this device is enable anyone with wired headphones to use them wirelessly. But, most people prefer wired headphones because you don’t need to charge them. So I thought why not allow them to be wireless when I need to(running, activity, etc), and turn them back to wired, when I’m sedentary, thus also charging them during wired use.

I began working on the schematic, and sourcing parts, and planning out the overall circuit around 3 weeks ago. And at this point I’ve probably spent over a hundred hours undocumented. But, better late than never, so here I am trying to catch up.

This work session was spent importing my schematic’s components into the pcb editor, where I noticed a few annoying things. Firstly, a lot of the componenets were footprints imported from EasyEDA’s library, which for some reason didn’t include access to the 3d files. I spent a good amount of time back and forth refreshing KiCad and importing the files, until they appeared adn I was able to import all of them.

Next, I was setting up the parameters of the board before I begin routing; things like layer stackup, differntial pairs, constraints, net classes. Here I found that I kept running into a problem where design rules checker complains about solder mask bridging between pads. After a bunch of research and troubleshooting, I realized I made a stupid mistake. The footprints I used for my resistors were 0201, where I meant to use 0603, wwhich meant they were all way smaller than they needed to be.

So I’ve fixed that, and now I’m deliberating a few things for the next work session. Mainly, I’m split between whether I want to use a 4 layer or 2 layer pcb for my design. A 4 layer design would simplifiy routing, and improve signal quality(and my ealier iteration was 4 layers too), but the main goal of this project from the start was affordablility, so in that regard a 2 layer board is significantly better option.

Anyways, that’s all for this log, future logs hopefully won’t be as dense. Thank you for reading :)

IMPORTANT SECTION BELOW(***for stardance admin):
Earlier today, I spent a 4 hour session(pictured below), redesigning my circuit schematic before I begin another pcb layout. I guess, there is some kind of glitch where lookout sessions aren’t connected to my project if the stardance tab is closed, while the session occurs. If any admin sees this, please comment a fix, because I’d like to claim the hours.