By
@ravi10
A research-grade cognitive runtime for autonomous robots that dynamically schedules reasoning, reflex, memory, and recovery modules based on urgency, safety, and resource constraints.
3h 22m 6s logged
I just wanted to add a way for robot to effectively store and fetch its memory.
I expected this phase to be fairly straightforward.
The runtime already had State Estimation, Planning and Scheduling. Memory sounded like another subsystem to plug into the pipeline, expose a few interfaces and move on.
It didn’t quite go that way.
I started with what seemed like the obvious design:
A common memory interface with multiple interchangeable strategies. Instead of committing to a single implementation, I built:
behind the same abstraction. I knew I’d want to compare different approaches later, so making them interchangeable from the beginning felt like the right decision.
Most of the implementation went smoothly 🔥.
Unlike the other strategies, SPSCA wasn’t just storing records. It was trying to represent them. My first implementation followed the algorithm closely. Every piece of content was recursively encoded using semantic pointer operations before being stored. At the time, I was pretty happy with it.
Then I ran the benchmarks.
The larger scenarios became painfully slow. Some didn’t even finish. I was upset that this whole work and time for nothing and just ruining the project? :-(
At this point, I feel like giving up on this and do the next thing or just implement this in the simplest way possible. I actually started making the next phase idea and then while doing that, suddenly out of nowhere, I got an idea about how to find the exact cause of failure 🥹.
So, I just went to see that was my hypothesis was correct about the error.
I spent a whole evening looking for small optimizations. Once I did, I found the issue with it!. The encoding stage itself had become the bottleneck.
That forced me to see the bigger picture of what’s happening during that stage.
Instead of trying to optimize the existing implementation, I redesigned the encoding pipeline completely. Memory content is now serialized into a deterministic representation and converted into fixed-length vectors using SHA-256 before participating in similarity operations. The semantic pointer operations are still implemented, but they’re no longer part of every storage operation.
That one decision simplified the whole implementation very much.
Watching the benchmarks finally finish without timing out was a huge relief. 😭
With the implementation stable again, I turned my attention to testing. I added benchmarks for insertion, retrieval latency, sustained memory growth and strategy comparisons, along with a much larger test suite covering storage, retrieval, forgetting behaviour, vector operations and runtime integration.
By the end of the phase, the runtime could finally retain experiences across execution cycles.
But, stopped myself from immediately moving on to the next feature as there were still a few loose ends I wanted to clean up before building anything on top of Memory.
Nothing in the runtime actually depends on memory yet. Planning doesn’t query it, scheduling doesn’t use it, and no module learns from previous experience. I considered pushing further, but decided against it. Changing the foundation becomes much harder once other components start depending on it.
As a small side quest during this phase, I also redesigned the project homepage. Hopefully it finally looks the way I originally imagined it.
For now, memory exists for one reason: to give the runtime a place to store or fetch it in a very efficient way.
4h 6m 17s logged
I thought it would be an interesting day today, as I was going to add something important that will help a robot perceive and predict the world.
Instead, I spent most of my time reading research papers.
At first I thought I could just implement a World Model and move on.
After reading more robotics papers, I realised everyone seems to solve the problem differently.
I couldn’t find a single approach that worked well across every scenario.
That changed my view of how I wanted to build this part of CORES.
Instead of picking one approach that works only in one scenario, I started implementing multiple approaches instead of one.
While doing this, I also realised that calling everything a “World Model” was a bit misleading.
The runtime isn’t trying to store the world.
It’s trying to estimate what the world currently looks like based on observations.
That made “State Estimation” feel like a much better name for the implementation.
The architecture didn’t really change.
Only the implementation became clearer.
It was mostly about changing how I think about this part of the project.
I think spending a day reading papers and redesigning things will save me weeks of rewriting later.
I’ve mentioned this before, but I think it’s important to build the clearest picture possible of a feature before implementing it.
Just to be clear, the papers and images below are existing robotics research.
I’m not claiming them as my own.
I’m studying different approaches before deciding how to implement this part of CORES.
3h 5m 13s logged
Today was a difficult day as i was working on too many things but i tried to simulate what i have implemented till now.
It was all about making it easier to understand.
Until now I had graphs, benchmark results and logs. They were useful for me, but if someone else looked at them, they probably wouldn’t understand why the runtime made a particular decision.
That made me think.
If a robot suddenly stops or changes its plan, how do I show why that happened instead of just showing numbers?
So today I spent most of my time working on the replay system.
To test everything, I created a simple Mars mission.
The rover starts driving normally.
Halfway through the mission a rockslide blocks the path.
Instead of continuing, the runtime reacts.
The rover stops.
Safety takes priority.
Navigation calculates another path.
Then the mission continues.
Watching that happen felt much better than looking at benchmark graphs.
For the first time it actually felt like I was watching the runtime make decisions.
I was happy at this point that all this work actually getting showed in a sim
(┬┬﹏┬┬)
I also spent time improving the replay itself.
The rover now has moving wheels, suspension, antenna, brake lights, warning lights and dust effects.
I also added different camera angles to make it easier to follow what is happening.
One feature I enjoyed building was the cognition timeline.
Instead of showing every module sitting there doing nothing, I can now see when modules become active, when they pause and when another module takes over.
That made the scheduler much easier to understand, while replaying a mission.
The most difficult part was finding a simple way to show what the runtime was thinking. Trust me it was difficult, putting everything into a presentable demo, Trying to fit weeks of work into a small demo meant deciding what to show and what to hide. I had to remove a lot of unnecessary details while still making it clear how the runtime actually works.
I think that’s something I’ll keep improving as the project grows.
Right now the replay still uses temporary modules.
The next step is replacing them with actual runtime components like Perception, Planning, Navigation, State Estimation and Safety Monitor.
After that I’ll connect the replay directly to the Python runtime so every event comes from the real scheduler instead of prerecorded data.
The replay is still pretty simple, but it finally feels like the project is becoming something people can interact with instead of just reading about.
I finally achieved something what i wanted to when i started the project, a simulation that is demoable.
It’s still far from finished, but seeing the runtime make decisions inside a simulation feels like a huge milestone for me.
CORES is a research project exploring how autonomous robots can schedule cognition instead of simply executing tasks.
This update introduces the first public version of the CORES experience. I built and deployed a homepage that explains the runtime architecture, scheduling, benchmark overview, and cognitive module system. A major focus was making a complex robotics runtime understandable through clean visual design rather than long technical documentation.
The next milestone is an interactive runtime simulator where visitors will be able to inject hazards, change mission conditions, and watch the scheduler reprioritize cognitive modules in real time. Exact visualizations and benchmark charts will be added.
I'd love feedback on:
- Is the core idea easy to understand?
- Does the homepage clearly communicate the purpose of CORES?
- What sections are confusing or missing?
3h 27m 1s logged
hmm, So when i started the project i thought i would build an actual simulation where i could show everyone how it works, its still far from that.
So i focused on doing that,
I took a break from working on the runtime itself and started building the website.
Until now, everything was happening inside the codebase. The runtime was working, benchmarks were running, and tests were passing, but if someone visited the repository they still wouldn’t understand what CORES actually does.
I just wanted it to explain the project.
I started building the first version of the website using Next.js.
Most of the time today went into trying different layouts and figuring out how to present the project in a simple way.
The homepage now has:
One sentence stayed with me while building the page:
The project should focus on making robots think better in emergency situations and have actual knowlege of their resources and their environment.
I think it explains the project much better than a long paragraph.
After changing the layout a few times, I removed most of the unnecessary elements and kept the design much simpler.
I’d rather have a clean page that explains the project than something flashy that distracts from it.
Getting the website online wasn’t completely straightforward.
Since the repository contains both the Python runtime and the Next.js frontend, I had to configure Vercel to build only the frontend.
A couple of module import issues also showed up during deployment.
After fixing those, the website finally deployed successfully.
While working on the homepage, I realised something.
The website shouldn’t just tell people what CORES is.
It should help them understand it.
Instead of reading pages of documentation, I want someone to interact with the runtime and see how scheduling decisions change under different situations.
That’s probably where I’ll spend most of my time next.
I also added around 80 more tests while working on the project.
The runtime is getting bigger every day, so I’d rather spend extra time writing tests now than spend hours debugging later.
Today wasn’t about adding another algorithm.
It was about making the project easier to understand.
The runtime is only useful if people can understand what it’s trying to do, and I think the homepage is a good first step toward that.
Tomorrow I’ll start working on the simulator so visitors can interact with the runtime instead of only reading about it.
3h 0m 47s logged
Welcome to my another devlog!!
Today was all about the scheduler.
Yesterday I finished a greedy scheduler, but I wasn’t convinced it was the best option. So today I implemented an exact 0/1 Knapsack scheduler expecting it to perform much better.
After running benchmarks, it honestly wasn’t what I expected.
Most of the time both schedulers produced almost the same schedule.
For a while I thought I had implemented something incorrectly, so I kept checking the code and rerunning the benchmarks.
Turns out the implementation wasn’t the problem.
The greedy scheduler was already finding the optimal solution in most of the scenarios I generated.
The only noticeable differences appeared when resources became really limited.
That’s where I found something that bothered me.
During a simulated sensor failure, the Knapsack scheduler decided to stop localization because it increased the optimization score.
From a mathematical point of view, that decision was correct.
From a robotics point of view, it was terrible.
A robot shouldn’t sacrifice safety just because it saves a little more battery.
That’s when I stopped thinking about improving the algorithm itself and started thinking about what I was actually asking it to optimize.
Instead of using one objective function, I switched to a lexicographic approach.
Now scheduling decisions happen in this order:
This means lower priorities can never override higher ones.
While changing the scheduler, I also realised I had been treating every module exactly the same.
That didn’t make much sense.
A battery monitor shouldn’t be treated the same way as an exploration module.
So I divided modules into three groups.
Those groups are now used directly by the scheduler.
I also added a module dependency graph so relationships between modules can be represented instead of assuming every module is independent.
After finishing the scheduler changes, I spent quite a bit of time expanding the benchmarks.
I added more scenarios, an ablation framework, and graphs so I could compare different scheduling policies instead of relying on a few manual examples.
The result I cared about the most was the original failure case.
The previous scheduler reduced safety coverage during a sensor failure.
The new scheduler kept safety at 100%.
That was enough to convince me this direction makes more sense.
83 tests passing.
Today was a good day, I completed and learned so many things from different algorithms and how each behave differently from other in eveyr situation. And it was fun. Not anything cool yet but it felt like i achieved something here.
1h 2m 14s logged
Date: July 15, 2026
Just started the project and it already feels ike theres too much too learn beofre starting. I know it would be hard designing the whole architecture and system and oh god it was!
So as I was saying, I started with the most important steps designing the whole system and architecture of the project.
The usual stufff!! :
The first few commits were just getting the project into a shape where I could actually start building.
.gitignoreTill this point I hadn’t really started implementing the actual idea. It was just a setup, that was going to be base of the whole project.
The first actual component I worked on was the event system.
I didn’t want modules calling each other directly because that would make everything tightly coupled very quickly.
Instead, I added an event-based communication layer.
At this point it was very simple, but it gave me something to build on.
Once events were working, I moved on to the scheduler.
At first I wasn’t sure how much responsibility the scheduler should have.
After a few iterations I settled on keeping it responsible only for deciding what runs next. The ExecutionLayer should only execute that plan.
I wanted scheduling decisions to exist independently from execution, so I introduced an ExecutionPlan instead of letting the runtime decide everything on the fly.
After that I implemented the runtime itself along with the execution layer.
The runtime finally had a complete execution cycle.
Too late feeling sleepyyy (┬┬﹏┬┬)
however i moved on from my sleep with a coffee but the headache wasnot over yet.
While connecting everything together I realised one part of the architecture wasn’t sitting right.
ModuleResult was trying to do too much.
Instead of letting it continue to grow, I redesigned it into a cleaner boundary between modules and the runtime.
That change also let me decouple the ExecutionLayer from the EventBus.
The code became much easier to understand what it does.
Here i learned before coding any feature you have to have clearest picture of the feature you want to implement or you will end up debugging and redesigning the architecture whole night like me ಥ_ಥ
I’m pretty sure this won’t be the last redesign, but it already feels cleaner than the first version.
After getting the runtime working, I didn’t want to assume it was correct.
I added integration tests for the runtime cycle instead of only testing individual classes.
It’s still a very small project, but having tests this early makes me much more comfortable refactoring things later.
I’ve never really documented architectural decisions properly before.
Usually I only remember what I changed.
A month later I completely forget why I changed it.
So this time I documented everything.
By the end of today, System Design and Architecture was complete.
There isn’t much to show visually yet.
No robot.
No simulator.
No fancy AI.
Just a runtime that can execute modules, schedule work, and pass integration tests.
Honestly, I’m happy with that.
The interesting features can wait.
I’d rather spend time getting the foundation right than spend the next few months constantly rewriting it.
So at last I went to sleep ofcourse it was important to me, I love my sleep. GoodNight ;-)
