AI agent guyz!!! It uses gemini and I gave it some tools to edit and understand the model. I also added a markdown visualiser to have nice and pretty outputs from the model. It wasnt hard to implement and it’s a great addition to the app I feel.
I’ve added cloud project saving and a project catalog!
You can publish your projects/models for other to see, train and use in their own projects.
It uses a simple s3 bucket for storage and handling the models, nothing too complex.
The catalog has some filters and search function.
I have added what I call an orchestrator system, you can now create a new client instance in the cloud, that can then be used for processing and editing your models, this comes as an automatic substitution to hosting your own local client.
From the home page you can now click on the add new configuration button and choose the cloud option and a automatic connection will be established.
The system uses docker to spin up and manage the instances, there is a service for cleanup and deleting orphan containers.
The orchestrator itself doesn’t act like a proxy, all of the request go directly between the next.js server of the interface and instance.
Okk, so there are now more tools to help you analyse your datasets, that includes charts, data quality, correlation and target analysys.
Another change is to the optimization, more specifically to the optimization results. You can now clearly see what changes where made to the model, and what mutations the network suffered over time.
What is there more to say, than the fact that there is now a working visual editor for creating your own neural networks!!!!
It’s literally unreal engine blueprints but for NN’s, there is a library of nodes, from which you can choose to add, and then you connect those using handles.
There is still some stuff to iron out, but this is looking good!
Ok you guys, we have working neural network optimization! For the moment there is only the option to optimize using a neuroevulution algorithm, but more will come in the future.
You can select the input features, target one, change some settings, lay back and get back an optimized version of your neural network.
This happens using pytorch models, i’m kind of moving away from .onnx, as they represent an “imovable” final version of a model to say so, and using the descriptor model me and my friend made will allow us to make visually editing models easier.
Now, optimizing spits out both a .pt2 and a .onnx file, as the onnx file is directly usable.
These changes required some modifications to the .mylo project file structure, but it’s not hard to add new stuff to it.
Next, I will have to polish the UI, add some more animations and also handle the posibility to upload both a .pt2 for working and a .onnx for visualisation only in the same project.
Besides that I also want to make like a “git” where people will be able to publish their models and stuff for you to use.
I have plenty planned, i really like how this project is coming along.
I got a friend that is more talented in working with AI to help me with this project, together, we developed a optimization tool. It’s currently not implemented in the UI, but it follows.
Optimization is a huge step in the development of this app, as it incorporates, building new models, training and testing them all in one flow, so by getting this working we opened the doors to training and infering tools for the models you build using MYLO.
Also, for this, my friend made a way to describe models in a json format, with the weights held in a seperate files, then, on runtime when needed, using the json, a new pytorch model will be built and the weights loaded, having it ready for any operations needed.
I had also been working a lot on design in figma and from my friends computer, so sadly I am only able to count up 2 hours of work for something close to 10.
Anyways, hopefully, next devlog ill have a working optimization window to show.
Ok, we got dataset analysis tools!
These first tools include a simple table viewer and a check for one-hot encoding, as in certain types of data it can cause excessive memory usage.
Next I want to add graphs for data distribution, missing values, etc.
I’ve worked on polishing what I got until now. I remade the home page to be more clean and to help you understand better what’s going on and what you have to do, basically some UX work. In the attached screenshot is the final version of the home page for now, implemented and fully reactive to the possible states.
I got working project files. While there are still some edge cases to iron out, you can save and load projects into the interface. There is also a system, in case the connection between the client and the interface fails, the data won’t be lost, as there is a copy of the entire project stored on the client, so when the connection is reinstated all of the data can be retrieved. This project appears on a dedicated card when in the interface.
Here is a small demo of the app:
I have spent this time integrating authentication into the app and making a small home page from which to manage client connections and projects.
There has also been a structural change. Until now when loading a file into the client, on request, the client would open a file picker window using tkinter, but this caused a lot of bugs, as tkinter has to run on the main thread, and in general, blocking the entire server until you picked a file was not good, so instead I made it work more traditionally, by having a file picker open on the web interface and uploading a file there before sending it to the client, and its really much better, the client server isn’t stuck waiting for you to pick a file and also it allows for the client to be on another machine, like a powerful server and you just sending the relevant model files to it.
It was still a slow crawl, I had to chew trough a lot of bugs with the authentication, as I implemented a middleware protection system, that blocks any requests that come from an unauthorized user, both for the client and webInterface, making the app fiarly secure, atleast by my standards.
Now that I have client connections done with a nice UI to add new clients and stuff, I will occupy myself with loading, exporting and managing project files.
Ok, so I started integrating the authentication into the app, this meant that now I had to make a way to tie projects and sessions to the user. To do this, I created my own project file, which is basically a disguised .zip file. When you save the project, it bundles all the necessary files into the zip, making it easy to send projects but also “tying” a project to a user is now very simple. Besides that, i began working on the Client system, basically, in the app, you will be able to have multiple clients/workers. Some may be in the local network, right on your device or there will be the option to rent out a server and use that for creating and training your neural networks.
Now that there can be multiple clients, that means that can be many more errors, so I also decided to add a notification system. It was made using sonner.
This devlog I bring you:
-Authentication, made with Auth.js, it has the options to log in using github and google. For now its just the code itself for logging in, retrieving user data and checking if the user is logged in, but it doesnt serve any function.
-Properties panel, now you have the option to open the properties panel. When clicking on a node it shows info about that node, like the inputs, outputs and any attributes it has.
-Data processing, for now only in the API, the panel is coming too! It takes a csv file and checks it’s compatibility with One-Shot encoding, and other basic information about the data, like missing values, encoding size, etc.
Sooo, I have been pretty efficient. We got the basic UI layout done, with the windows and tabs and everything, which have been customized. We also have a logo (sadly I couldn’t log the time working on it) and most importantly we have the first tool!
The visualization tool!
(now I realized I’ve misspelled visualize in the app)
Which also means I got the communication between the web interface and api working.
For now the visualization only works with ONNX files, but I hope to get this expanded to other formats like pytorch files.
The tool works this way. You “point” to the file on your computer, the client on your laptop processes it and generates a list of edges and nodes, where each node carries specific information about itself. Then in the UI I post-process them a bit, as the names are pretty ugly like “/lstm/LSTM|/” and I just trim it. Then based on the category of node, I color it a specific color. The constant nodes are removed for clarity (although I will add a way to re-enable them). Then using ELKjs I order the nodes and then display them using react-flow.
I really like how it all turned out.
Some other notable things are about the meubar, it’s very easy to add buttons, and for those buttons to have nested ones, its just basically a JSON, where each button has a label, an onClick function and children. All of this setup will help me in the future.
This is all for now.
I’ve started with the scaffolding of this project. I want the UI to be IDE like, so I created a window management system using Dockview. I also made a Client class. The idea is for there to be two ways to process your neural networks, either locally or in the cloud, thus I made this client class that should make it very easy to switch between the two on the Frontend side. In the client itself, where all the processing happens, I made a very basic fastapi server and some endpoints for testing, as well as laid the brickwork to “load” local files into the client, without having them uploaded to the Website UI and then sent over to the MYLO client on the PC, basically creating a duplicate just to read from it. It works by instead of opening a file dialog in the browser, opening one from the python client installed on the computer, thus allowing me to access information such as the absolute file path, which would otherwise not be possible trough the webinterface.
I have been working on making racing ai for the game. As a small intro, i have worked on this project before, and what I got is a working car controller, and dialed in physics to something I like how it feels.
The ai represented a difficult problem. If I want this to be open world, and for there to be police chases, I needed the NPC car to be able to go offroad, so basically, of a set path. There would be the option to make a hand tailored map and even for the offroad sections to be counted as roads, but that would be hard for one developer to do and test, so I opted for this. When the car is not on a road network, it fallbacks to the built in navmesh system in unity. It then uses it to calculate a path as a list of point and then drives between them. On the road, we just need to sample points from the spline of the road and then drive between them.
The hard part was this driving part :D
While it might seem simple, just point the car to the next point, and then have it move forwards, there is much more nuance to this. First, the NPC car should behave just like the player car, it should have the same turn angle, and thus I must now convert a simple direction into a steering angle. Another difficult part is when racing. I can’t just have it accelerate always, the car must brake when a sharp corner is coming up, and for really tight one, maybe even drift. So how do you do that? Well, i dont know yet, this past 4 hours I have been working on the same script and same section trying to figure that out. I really hope that next devlog I can come with a solution, but i cant promise anything.
My plan is to calculate the curve of the upcoming curve, and then somehow spit out a good speed for that, but there are a lot of factors to account for, like the grip of the car and actual speed, distance to corner etc.
