@azure

Devlog 2: Added concurrency for playlist, been getting bored of coding lately so less progress than usual ill try to ship this before thirdspace

Devlog 1: added the frame to text after i took forever figuring out how it works but its actually simple brightness based, did have to figure out how to calc(short for calculate) brightness with rgb but ended up just looking it up fix some bugs in my code(besides me forgetting to type cast in the float), and cleaned the tui up a bit more minor things.

Devlog 0: got inspired by this vid from a video so im making a basic tui set up while i research easy stuff :3 i made a textual basic app with quit and cursor controls, and a skeleton for downloading youtube videos and turning them into text

no more ai css: i re did a lot of the css(colors, some padding, highlights and size) and a good bit of the js(the ai made the skeleton but i changed it for my use so you’ll see bits of it with patterns i did still use ai for the base) + redid the python script that ai made for the reasoning steps for agent, removed unused parts from frontend and minor stuff. less than 30% is js even assuming its all ai made but i did redo it. Only parts ai coded is the sql script, a few lines in docker correcting my mistakes, and some of the js helping me out. Also i added some stuff so scheduling should actually work i noticed there was nothing queued in the logs before hand. back to next ship i made a script to collect me a dataset with both satellites gotta figure out how to label forest vs not forest, and once that’s done ill re do frontend from scratch or ill make it soley an api maybe ill do both either way big changes for ship 2 and no more slop

Devlog 7: its getting hard now
working on scheduler. added a skeleton for get current and desired state, refactored the parsing config. and more. many issues mainly around desired state and designing it, and a few other minor hiccups. errors are from protos not being updated in go yet not a real issue(i think)

Devlog 6: edited tui, and finished server side stuff till i update protobuffer.

No issues this time! i’ve added ttl to redis, ill add more functions, im gonna work on client side stuff then sechdular, after that its time for the binary and probably more interfaces to make sure everything works because i got lazy and didn’t do that for Redis bad SOC but idc!

Devlog 5: gRpc is actually easy
yeah i dunno why i got confused besides refusing to read docs pretty intuitive to use with go Progress: got most of the two internal daemons done server side, scheduler and client side still need to be done, added a way to keep up with heartbeats so it can actually be used. and decided on naming(on parts) and removed redundancy. Some issues i’ve ran into were i was going to recreate etcd but to complex to re do design i had in mind to fit around that and im just to lazy to anyways, and figuring out how redis with go works but didn’t take long. but mainly a me wasting time just to end up removing them for a more efficient way to, and making sure everything matches, and not an issue(as of now) but ill have to deal with it in future, req.id PROBABLY isn’t gonna match container id and stuff and no clue if i still need to pull image, ill work on that but an issue that’s going to get me later. smaller devlog still takes me too long though!

Devlog 4: GRPC IS CONFUSINGGGG
this took way to longer, i thought i knew grpc better because i watched a YouTube video on it weeks ago but i finally understand it now, I added Ymal parsing/config, i refactored file structure some, i plan to replace registry with Redis, and ill add a JSON backup so if daemon/tui/ whatever is restarted or for whatever reason fails ill be able to back up stuff or user will be able to. main hiccups were file structure and figuring out grpc, ymal took some time to figure out docs but relatively self explanatory, but i also haven’t tested it so maybe not . i don’t plan on making a actual etc so like mentioned im going to use Redis in its place.

Devlog 3: Took a bit too long! ive mainly been reading moby/container docs, and kuberenetes docs. because i dont actually use it so i wanted to make sure understating is correct and everything(mostly because im too lazy to do it all) matches. added docker functionality, well not really i added a interface and test ill do real docker once ymal and everything else is set up, i did MAINLY set up ymal parsing but not fully complete. pretty productive, i thought i was relearning go but i guess i didn’t know much about it outside of api so its a good learning experience. no issues on this log, besides it taking a while due to refactoring and wanting everything to fit together nicely + SOC. ill likely start developing faster once im free from this config stuff.

Devlog 2: protobuffer was easier than i thought to set up, i refactored a couple files to match/ separate concerns, no hiccups i just had to switch from pycharm to goland(because im a jetbrains fanboy and wont use vs code or vim) that’s all for now. next ill work on finishing go and node and possibly mounting on to docker or figuring out how kuberentes does that.

Devlog 12: last devlog before i ship!
This was mostly bug fixing and polishing.
I finally tracked down an issue where the model was outputting 0.0. At first I thought the model itself was broken, but it turned out I was double normalizing the Google Earth Engine test data. After fixing that (and retraining the model), accuracy improved quite a bit. I also realized that green ≠ forest and overall more confidence.The permit querying system also had problems. I originally trusted gemini(clanker) because I couldn’t make sense of the IBAMA(its in portuguese) data, but that ended up sending me down a debugging forever. Even after fixing it, the ML still wasnt perfect, so i added a mask(as seen in img 3)and even after retraining the model i noticed the mask was random globs and lines, after 4 hours of trying to debug it was a few things: the model just isn’t meant for it as an u-net and i thought subtracting the two predictions would work but didn’t, so i added threshold, a bit more accurate but not good enough, missed a ton, what really fixed i t was averaging after sigamoid, binary opening and closing, and a big fix for the models accuracy(before threshold-ing) was fixing the payload, i used the wrong cords in payload so the image wasn’t correct. after all of that i finally fixed the prediction!
features i added and fixed:chain of thought, damage %, some frontend overhauls(as you can see), and more. refined prompt for agent, dynamic scanning(no longer hard coded), and chunking for memory. that’s all(all i remember at least)!

Devlog 11: i was right llms were gonna have an issue: tokens. but so far i;ve completely fixed Machine Learning which was always saying it was deforestation due to it being a u-net classifier. which was pretty pointless so i added before and after images and used mask to tell if deforestation occurred and accounting for if trees grew back it wouldn’t just kill system, i added cot functional, and llm was overthinking but i;ve been working on that lately. im in the home stretch for shipping all i want to do is expand where it scans, and chunking the scans so it doesn’t use too many resources, and possibly upgrading to a database instead of json? ill do that in ship #2 as it’ll add more bands and more. but im gonna widen scanning. at mini um. also just identified a bottle neck with my permit tester so ill fix that as well its messing with llm.

Devlog 10.75 had to fix left over placeholder values, gave up on Sentile-1 for this current ship because couldn’t find datasets and i don’t have time or compute to make my own. and I have been working on frontend, and fixing some task. and added a scan button in frontend for interactivity. im now testing the scans and making sure it actually sends over images as i’ve finally debugged ALMOST everything. i had a hiccup with model because i forgot the scope so it got angry there and i have a feeling llm will give me problems so ill need to test and see if it hallucinates.

Devlog 10.5: adding new machine learning, about to train, and added SAR satellite and going to train wont finish till tmrw or 2 days most likely.

Devlog 10: finally go it mainly functional. while it doesn’t look that way im gonna fix that, its just a json not fully adding + API issue on react side and i removed map when removing dummy data so i have to add it back. but other than that fully functional as i know. Some issues i ran into while debugging were, i stupidly forgot to link my .env and google earth engine key to docker, and bigger, i didn’t have a concurrency argument inside of celery workers so i ran into OOM form 96 workers. besides that just some name mix matches, i normalized the time across everything. and just debugged ngnix server to rout correctly, i left it cached and didn’t realize it was broken till i recreated to image. I plan to add sentile-1 SAR satellite to this and cleaning up frontend before i finish the ship. I decied that alerts/messing will come in ship 2 along with a more refined model and possibly more features, contacting NGOs or authorizes is a nice to have but i don’t know how’d i do it? i don’t wanna but a domain for an email and many don’t have apis, slackbot could work if they have any but that will be for ship #2 unless i change my mind.

Devlog 1: Set up the basics and made a rip off logo
nothing eventful, i set up bubble tui, nodes(cats) and pods(paws) types and go, making the protobuffer for hearbeats and stuff next

Devlog 9 : after 10 years of debugging i realized i had to make a sub domain(maybe i should’ve read docs). but after i fixed the nginix server the app was functional i just needed a subdomain to point at it. besides that i had a view errors like apikey instead of api_key, and directors being missing, and a big one requirements.txt being to big, what i did was use a swap file so its not all in memory and build service by service. another issue still on going is celery snit working which is half of my architecture and how i need to test. i think its because i set up config wrong + wrong urls/ ill try to fix it by 4am today because ill miss tmrw and possibly Friday as well(rip to my streak)

Devlog 8: Docker, so i spent a while debugging, fixing some pipelines with machine learning and sending images and more(static files) over to frontend before i added an api instead of fake data, but now the 2GB of ram isn’t looking like its enough. Before that though i had some issues setting up nginx so ill check nest documents to see if I’m doing anything wrong before i request more resources. ill try to fix instillation or find another way

Devlog 7.5: revamped frontend to show Where, and before and after, no point of plugging in api yet since i need to deploy before i can even run backend. also added call llm, and agentic loop so that should now be functional. no hiccups or issues just passing time while i wait for nest approval. And since like i said i have yet to run backend once since my computer cant handle it im expecting 1-2 hours more of debugging after i deploy.