Scope Leaking Bug Fixed, Mouse Input, Scripting Improvements, ECS & Events Scripting Improvements and Fixes, Cameras

There was an issue where I used a single variable to hold the last used scope before and forgot to change it to support multiple scripts. That is fixed now and scopes aren’t mixed weirdly anymore.

There is now mouse functions and events

Events can be called like part.physics:on("touched") instead of part:on("Physics.touched") for clarity

Cameras are added, game wide function to change active camera, and camera system that updates the camera things bidirectionally supporting multiple camera types.

Script Scheduling Engine Revamp

The entire scripting engine has been redone. It was a lot of cascading changes again, with now each “Scope” having it’s own scheduler. A “scope” is a single logic component that gets attached to entities.

How it works now

The scheduler handles the Lua engine stuff. It wraps the main script around into a coroutine automatically now, and extracts the coroutine into a thread. It then creates a scheduler job object, pending into the queue. The scheduler then gets stepped, and initializes the jobs into the main queue and steps them. By stepping them, it resumes the coroutine until the script either ends or manually pauses the coroutine itself (usually done with a predefined “wait” function). I think this is very similar to how Roblox handles it’s script scheduler, although Roblox’s is of course much more complicated. Handling the events and callbacks was much more difficult though. Each of those ALSO need it’s own thread to run to access coroutine related functions and to access data (otherwise very weird errors will happen, like calling code over a boundary or something). This all gets added as a scheduler job, and that allows us to record telemetry data. Eventually, threads can be assigned a “budget”, where they can be stalled if they consume more than the allotted time or even do things like infinite loop detection, etc. There was just generally tedious work that was annoying.

Added CSG Operations

I added CSG operations, using the Manifold library. It makes it a bit simpler, making watertight meshes and giving back the geometry for me to make a custom mesh in ThreeJS. The reason it took so long was because I had to add cascading changes to a lot of the data structures to support this custom geometry. I also added a little loading thing at the bottom right for CSG operations. As of right now Union and Subtract are supported. Materials are not preserved as of right now, due to the simplistic way I’m handling materials but when I finish building out the materials system I basically just need to get the triangle references out of Manifold and relink them to the correct materials.

Engine Events System & Starting Significant Scheduler Engine Revamp

So I made the events system. It works by a class on the JavaScript (engine) layer, and is attached to all entities. There are a couple of functions:

• on: fires on event
• once: fires on event only once
• off: clears event callback
• clear: clears all callbacks

Other entities are allowed to access another entitys events, being able to listen or even emit. A game architecture could work like having “manager” entities at the top level of the game which hold different events. For example, an “InventoryManager” entity could have an event for when an item is picked up, it itself can listen to it, as well as any other things that may find it useful.

I’ve also cleaned up the code, and now there is a “game” reference. Eventually, it will have access to more things but as of right now you can access an “inputManager” which holds functions and events for user inputs. It works by attaching a metatable to it and exposing global hidden functions to call to my JavaScript engine code.

But a huge problem showed up, the way I was running scripts. Before I thought just a simple coroutine running with a timeout for delays was just fine, but I was super wrong. If a callback from an event is fired and the function wants to do something, like call an external function or wait, it completely breaks. Why? I didn’t know and it took me a lot of time to find out. It’s because the callbacks aren’t a coroutine too. This is when I realized why something like Roblox has a full blown scheduler.

So I’ve been planning and starting on the scheduler engine. It has to handle a lot of stuff, including managing when scripts run, including WHEN they want to run (next frame, in 3 seconds, etc) and track if they are causing issues by keeping track of their budget. Everything will need to be in a coroutine coordinated by the scheduler. Including callbacks from events too. It’s going to be pretty complicated I think, requiring a rewrite of the entire scripting system.

I also just encountered a lot of bugs. And I fixed the errors from Wasmoon including the entire JS stacktrace for no reason too.

Scripting System Affecting the Game

Scripts now can change parts. There are getEntityById and getEntityByName functions now, and a bunch of metatables and stuff to get it working. Now you can query for an entity, and then modify it’s component data with metatables. For example to access or set the position of a part, you can do part.transform.position. The transform finds the transform component and position finds the data entry of position inside the component. This is all done on runtime, with component queries happening dynamically so no manual programming for new components is required. This may affect performance a little bit though. There is a custom Vector3 object now too.

Error messages are also now a little less vague but still pretty confusing. I don’t know how to solve that.

Logic & Scripting Prototype Working

this was so difficult

basically a TON of things needed to be changed for this to work. i had to rework the entire scripting system. i had to also think about how the system is going to work

the system i landed on, basically is you can create nodes in the state machine editor, and attach scripts on those states. instead of scripts, im calling the whole package of state machine + scripts a “Logic”. i havent reflected the name everywhere else yet though, and that is probably going to take a really long time as its a major architectural shift that i didn’t plan for beforehand.

i made custom nodes in the editor as you can see, a start, end and the script node. there is now a nice dropdown to select the script (a searchbar will be added later) and you can create new scripts. i fixed the code editor to actually load/save code properly.

i had to redo how it saved data, so i could communicate the scripts, nodes, connections, etc all to the ScriptingSystem, which starts off at the “Start” node and steps the state once, going into the first node. This part of the code is an absolute MESS.

i had a ton of bugs to fix, including state and Svelte firing state over and over again for some reason, Lua issues, dealing with coroutines, dealing with external functions which would change the state and exit the code, etc

To add the script, there is now a “Add component” button which you can select the script and select the logic.

The state machine handles direct to end, looping nodes, multiple handles, infinite loops going into other nodes, etc.

At the end of the video, you can see that i created two states and the second state had two outputs, and the path is a 50/50 random decided by the Lua code to loop back into the first state or exit.

Overall, this was really tiring and I’m glad it’s done. The amount of time thinking and debugging isn’t included in the time tracked so yeah :(

Scripting System

I added Svelte Flow and the Monaco editor
Basically, I think that the system will work mainly off of state machines. Each state will have an attached script. The scripts can decide when to change states.

For example, the scripting for a bee in a game could have states for wandering, eating, etc and the scripts handle only it’s own state which could make the code a little cleaner.

I also tried making the script editor to look really good. I spent a bunch of time trying to add LSPs and intellisense/autocomplete and syntax and formatting but honestly I give up for now.

Fixed Constraints

Getting constraints working was super annoying. Basically, there is a problem where the engine is built around models, and you can spawn in multiple models. When models are spawned in, the parts inside are cloned including their IDs to not lose reference, but this comes with the huge issue as in the backend, the engine actually flattens this structure for more efficient processing of entities, meaning that there will be multiple entities sharing IDs, which is really bad.

But creating new IDs is also a huge problem because when we reference it, the ID is not final and changes all the time on load. So instead it does <model_id>:<entity_id>, for targeting.

The constraint creator lets you select two points (snapping will be added in the future) and it creates a constraint. This is later processed by the PhysicsSystem detecting the ConstraintComponent. Later there will be debug viewer for constraints.

Anyway it was just a lot of annoying work.

Opening Tabs Working

There is now a flow for opening and creating tabs, now new tabs can be created and new models in the same way. Opening a tab that is already open just switches to it.

Runtime working

This took so long

There was just so many things that just weren’t working well together. Especially the data loading code. I initially just made the code use and modify the gameData directly, meaning that if I stop and play again the game would not reset and would probably completely break. But cloning is so difficult. Since the gameData is pretty complicated with nested classes and data, simply JSON.stringify and parse, and even structuredClone didn’t work. I had to use a function to manually copy a lot of things which was really annoying.

Another thing that was weird was that parts would only have physics if it was dictated by the parent model, which wasn’t what I wanted. Now there is a function to flatten models, since I want them to kind of just be fancy folders at the end of the day anyway.

There was just generally a lot of rework to get everything working, with systems and components working now with a setup phase, and a general frame tick function for systems to process. I also had to add dirty flags to all ComponentDataEntries so other components actually notice changes by OTHER components which was so annoying :((

Model Thumbnails, Better Highlighting, Reactivity, Sidebar, etc

The models now have a place to store a thumbnail, and on save, it autogenerates a new thumbnail. It creates a new ThreeJS scene in the background and adds all the entities to it. For now it only cares about the Transform component, so that’s why all the parts don’t have the correct colors.

The face highlighting also is fixed to use ThreeJS’s Line2 so an adjustable line thickness can be used. The face edge highlights are now thicker and a lot easier to see.

Reactivity is now fixed, so dragging an object in the editor updates the UI, and the other way around. In the video, you can also see the “–” when multiple objects were selected. It means they have unique values, but they can still be all updated. The system works by storing all seen components in a JS map, and then only filtering for components that appear in ALL of the selected parts. I might change that functionality later but it works for now.

I also added the sidebar to the model workspace. I also did work on the ECS system and just engine code in general.

Fixed ECS Data Storage and UI Displays ECS Data

I got the UI to display the data for the ECS components. I had some weird bugs, also I messed up earlier on the code to store the data entries for the components (stores things like data type and stuff) and then now it displays on the sidebar.

For now I temporarily disabled the code to handle multiple different objects that could have differing component values or different components altogether, because it was causing a bunch of problems cuz the code is not great :(

Started Data and ECS Code

I started working on the ECS system, and now the parts spawned are “entities” and they have “components” like a mesh, a transform, and a physics component. The components are queried and cached by the game engine and passed to the “systems” that actually do work with the component data of the entity.

I also dealt with a stupid tab bug :(((
And generally other engine code.

Worked on Model tab and Working on Refactoring Backend & Renderer Component

There’s now a model tab, also I made some more animations and things to make the app more fun to use. I’m also in the process of rebuilding the data types and completely redoing the reusable Renderer component since it was an absolute disaster before.

Completely Revamped UI and Working On Refactoring Entire Engine

I really tried making the UI look a lot nicer than the generic UI it was before, and I think it looks pretty nice. I might need to simplify some stuff, but I spent a long time trying to make the UI more welcoming and easy to use and for the UI to seem more playful in a way I guess, than the more mono font boring gray it was last time.

I also have started on completely refactoring the engine. The previous data stores and data types were too limiting and I made some bad decisions early on so starting from scratch is kind of necessary

Added constraints, which took a while because the way I store objects just isn’t sustainable

I’m going to redo the entire architechure of this app, mostly the backend because it’s just such a mess. I’ve been piling on features and the technical debt and poor early decisions are catching up to me and it’s making development slower so I’m going to start refactoring a lot of stuff. This should mean everything will be less buggy and make it faster for me to develop new features

Now that I know how the app will be I can make smarter decisions

Reworked script system for better and easier scripting and added some basic part modification functions

Basically, before you would need to type something like find(“stud”):setVelocity({10, 10, 10}) or something like that, but with metatables

Metatables are kind of weird, but they basically let me return a special object from the find() function, and modify that object to override functions and add new ones. For example, you can override the metatable of custom Vector3 objects to handle addition, so you can do v1 + v2 instead of addVectors(v1, v2) and it’s a lot cleaner and straightforward.

This means injecting a bunch of Lua code to set these metatables, and exposing global functions from JS to handle the metatable calls.

I also added some event handling (very basic for now) but you can see in the video the part moving from me using WASD

You can also see a ton of errors, and I’m trying to fix those right now. I really don’t know why they are here

Part sidebar shows parent/child relationships now (also had to change backend)

I also got save data to serialize, but loading seems to break the other components that rely on the state

I also added the cool little part add thing. I changed the font of the app too and stuff. I tried different search things but this one from shadcn-svelte seemed to work the best. I also picked the Lyra theme from shadcn-svelte

It’s been a while, I kind of forgot to log in the middle

I reimplemented parts of the backend to support play testing and handling the state of the current game, keeping track of parts and bodies and scripts

Added physics engine (Rapier), requiring keeping track of colliders and physics bodies. Right now only convex meshes for performance meaning concave colliders might be a little broken for now so I plan to add something like VHACD or a more modern alternative to split up complicated meshes into smaller convex and simplified colliders.

Added Wasmoon (Lua interpreter) and CodeMirror 6 (code editor component) and some old language parser for Lua to be supported properly, this was also a challenge because Wasmoon just wants to run all the code in one go, so instead there has to be a coroutine setup where the code returns back out temporarily (with a custom wait function that utilized the coroutine function) where then my code ticks every frame to tick the code (while loop) again and also adds the delay if applicable

The renderer also needed a few tweaks to get this to work properly

Also something kind of weird was I couldn’t get the playTest state to update and propagate throughout the application (thats why the top bar button doesn’t work for now) and honestly I have no idea why and I spent a lot of time debugging it

Can now apply textures to objects with a new Texture Studio tab :)

It’s still really buggy and doesn’t work on CSG objects. It also ignores the coplanar face detection I have right now for simplicity so it only works on ThreeJS primitive face groups as of right now.

I added the framework of a UV atlas editor, and also the painting studio for the texture editor

I’m working on the texture system. I’m still working on UVs. It’s honestly really complicated, I have no idea so far how to make it intuitive for users. Auto unwrapping it would not give easy to use results because most assets in this engine are going to be simple. Showing all the vertices on a single atlas also would be very overwhelming. I’m trying to figure something out. It could be determined on a face by face basis, but some faces could be super super weird due to imported meshes or weird CSG operations, which would lead to a mess of a UV unwrap

I’m also going to make an editor to assemble the texture atlases by stitching together the individual smaller textures.

Finally got the world studio to work properly, it was because I didn’t assign an ID to the spawned objects

And I added the premade things in the scene also be added to the store

I finally got model saving to sort of work. This required revamping the backend stores to support multiple tabs, and updating the sidebar to actually use the correct tab state data.

Moved 3D rendering code to be shared between both studios via creating a component that applies the shading and stuff and exposing the scene for the individual studios to add objects

And added an isolation mode that allows the user to focus a single object or group of objects (whatever the user is selecting) and pressing i to toggle. Useful for like creating constraints and stuff more easily

Transform controls (now can switch to rotate and scale)

And basic undo/redo system that basically just takes the ThreeJS properties and saves it and reapplies it back on undo

And delete

Face selection (took so long) and with face selection I added constraint creation system that just creates a constraint in the center of the faces selected

I had to add another face renderer, and then also refactor the stores and how I stored data to get this to work and to keep track of faces selected and stuff which took a long time

The CSG works really well now. Union operations now maintain materials by keeping track of the mesh IDs and then reassigning the materials afterwards manually. Also some clean up and general improvement

I’ve fixed the face selection code to check the dot product of the faces to make sure they are actually coplanar.

I also added some cleanup functions to perform after the CSG operations to clean up geometry to make it more usable

Face detection and highlighting working

This was pretty complicated because 3D objects are made up of triangles, but the user probably does NOT want to select triangles, but the faces themselves. So basically, I got the face the raycast hit, and then looped through all the triangles of the mesh they are hovering and got the vertex indices and got their positions and turned that into a normal and compared it with the other faces and decided if the normals were close enough

Added retro shader that quantizes the colors to look more like a limited color space

Fixed CSG errors and fixed gizmo disappearing over and over again

Added another render pass and raycast to add outline to hovered objects

I switched the CSG library to Manifold3D and it’s better but the meshes are a bit weird. Hopefully I’ll figure it out later. Also for some reason selection of these meshes and their transform gizmos are really difficult and buggy so I have to fix that.

Also I know that there are errors I’m going to fix that now

CSG is being very annoying.

I’m using three-bvh-csg but it’s not really what I need I think. There’s some subtraction artifacts, and most importantly, materials are not preserved

I think I’m going to switch to a different library because this one isn’t really working great for me but it is pretty nice and simple

Added multi part select. Trying to debug why CSG operations not working

I honestly don’t know what’s wrong because I followed the example code from the library but it’s not working :(

The sidebars are now synced, but I still need to make them sync correctly to the current tab. Right now they sync exclusively to the model studio. The raycasting & selection of parts work now and I fixed the tabs to not unload and reload on each tab switch which caused data sync issues and lag and instead all tabs stay loaded.

I made the part selected have an outline and made the gizmo work, but currently I can’t select other parts for some reason and I’ve just been debugging that

I started work on the “World Studio” and “Model Studio” of this game engine/fantasy console. The model studio is where you create the models that you drag into the world studio. I’ve added a gizmo and a way to place the parts and also made a more stylized rendering thing to make it feel more retro in a way.

I started the project and installed SvelteKit and added ThreeJS