Sigma Peak Editor Frfr

The editor is turning out beautifully. Like I love it so far. Tons and tons of improvements to mainly the editor, but a few pivotal improvements to the compiler itself.

Website / GUI Improvements

• File uploads — no longer are your costumes handwritten svgs :P AND you can import .knip files!
• QOL improvements — draggable files, auto-focus editor on openning of website
• Terminal prettification
• STAGE BUTTONS! — start / pause / stop + TurboMode
• Flag functionality rework
  click => compile + run
  shift+click => run without compiling
• Settings file actually does something

Compiler improvements

• Costumes implementation (new syntax!)
• Forever block + more math blocks!
• Zip compression level compiler flag

Costumes + Sounds!

Finally, after a long time of putting this off, Katnip finally has syntax to import sounds and assets to use for your projects!

stage {
    costume "./sky.svg";

    events.onFlag() {
        looks.switchBackdrop("sky");
    }
}

sprite Cat {
    costume "./cat.svg" as idle;
    costume "./cat-blink.svg" as blink;
    sound "./meow.wav";

    events.onFlag() {
        motion.goTo(-100, 0);
        looks.switchCostume("idle");
        looks.say("costumes + sounds work", 2);
        forever {
            wait(0.5);
            looks.nextCostume();
        }
    }
}

Changelog

Compiler changes

• Added costume and sound keywords
• Added stage keyword for stage override (previously auto-gen’d)
• Added forever keyword (cuz I forgot to add it somehow)

Website changes

• Added upload icon for uploading files
• Added support for the new costume and sound required fs callbacks
• Added “rename” in dropdown right-click context

Notes

Not sure how the forever block and rename button took this long to add, both seem like something I could’ve adressed earlier, but—better late than never :P

Fleshing out the GUI

Finally got a working prototype! Check it out at https://katnip.org

Components added:

Stage

This was honestly a pretty hard part of the process. I am using tw-scaffolding, the core vm from TurboWarp. But it has no types, no docs, and the docs that exist are outdated. So I resorted to having Claude make a .d.ts file for me so I could fuzz my way through instead.

This process wasn’t that bad once the functions were clear, and I hooked it up to my compiler—intentionally designed to output Uint8Array.

Toolbar

This was fun since I don’t know svelte yet that well. I had a state originally stored in my state.svelte.ts, but I figure out I could refactor and not need it.

Then I just did some vm tomfoolery to ensure the threads stay open on pausing, and it was good to go.

Terminal

This is a super rudamentry approach. I plan to have an input method at some point, but we will see. Hopefully I implement a better technique, currently I just have a string array storing the logs. Makes it easy to make it stateful in svelte ig

Katnip GUI!!!

Ahhhh im so excited.
I’m building the gui for Katnip right now—built off a small foundation done by a friend.

Currently repo is private, but I’ll open it once my computer’s not about to die and I’m on stable internet.

Worked on the following things:

• Re-binding TextMate grammar tokens
• File editor popup
• TurboWarp scaffolding library for the running
• Pulling in my compiler through npm for compilation

Yikes.

So this is a step I didn’t really want to take.
I had to take away a part of the language that I always wanted to exist.

The existence of the temp keyword.

I mean this thing was literally covering my demos and other scripts in the docs and whatnot. But alas, it was not properly implemented, and I have no forseeable good solution to working it out.

So its removed for the time being.

Brighter note

On the flipside, I did finalize and fully implement zip, enumerate, and range!

These functions now properly parse the list arguments and let tuple unpacking do its amazing goodness.

Cooking up straight peak

Great new features. Genuinely super cool.
This new bout of progress that led to end-to-end compilation has given me so much motivation.

Enums

So, this already existed. But I realized that they are so much more than just a way to create states to check against, or store values in. Ever since I allowed scalar values to be the represenation of enum members, this has become more powerful than I realized.

The key was implementing the type interpretation of an enum. I already was kind of doing this, but when you have an input param for a function, you can assign a type of enum FlowerType for instance. So the input only accepts FlowerType.x where x is the type.

But thats long. Look at a function call:
sniff(FlowerType.DAFFODIL, backup = FlowerType.ROSE);

Instead, I like what typescript does with its discriminant types, allowing a type to represent a set of possible values. So, it was born.

sniff("daffodil", backup = "rose");

In order to use this, just assign values to your enum:

enum FlowerType {
    DAFFODIL = "daffodil",
    ROSE = "rose",
    ...
}

STDLib improvements

Prior, a lot of stuff including type casting was a mess. I just recently got this up and running, so it was hard to even try to think about this.

Scratch only has a few types: round and angled and block. Thats it. And types really only apply to round/angled.

So I implemented small blocks that allow scratch’s oddities and allow you to tell the compiler to “shut up”.

Want to get the string of a bool? No problem.

looks.say( Str(var == 2) + "apple")

And many many other use cases.

Made some marginal improvements to the CLI to allow parsing

Made some commands to run with command pallete for the vscode ext

Holy Guacamole I’m So Excited

Insane progress over the past few days has occurred, in large part due to my travels on train, and my apparent ability to lock in when I start seeing things.

Codegen

Probably the most recognizable part of the project, Katnip can now generate scratch code. Blocks are being made, and complex patterns written in the IR are finally revealing themselves.

It is so important that you look at the pictures, as the improvements made here are hard to put into words.

Writing this part was arguably the most fun part of the entire project. This part was more scratch-like than any other part so far (“well duh, its the closest stage” you might be saying. And you’d be right.)

IR

Writing a majority did not come without its need for the IR to step up its game. I was quite fed up with 0 visual progress, so I jumped to codegen to boost my morale for the improvements made to the IR.

This includes dict and list slicing, and a proper handling of booleans.

I’m so excited

The light

I can see it… (what is it?) … the light at the end of the tunnel… (what are you talking about?) .. its almost here

Yea no it ain’t

Though the project isn’t done, I will admit I am nearing the closing moments of the IR stage, with only 1 last hurdle to overcome before I can transpile projects beginning -> end.

Here’s a small overview of what’s been done in the last two weeks (nah who am I kidding its all last night, I haven’t touched this in a bit)

Procedures

Katnip is scratch. Kinda. It transpiles to scratch. So it needs procedures.

I took a giant bite out of the huge problem that is procedures. It is very complex to try and emulate a return system in a program that just doesn’t support it.

Having taken a DSA class, this was a problem that I took to paper, and got the following solution:

1. Stacks are beautiful.
   Let me elaborate. A stack is a structure that follows FiLo, or first-in, last-out. This is super important for returning from procedures, when considering the fact that a single variable will not be enough for systems when/if the function recurses. So this way, you push and pop on the stack to keep the order, and all return values are kept track of nicely!
2. Mangled vars.
   Scope is a really hard concept to grasp, but even harder to emulate in a non-scoped language. For the most part, I want to keep Scratch’s inherent scope-less design, in the sense that variables can exist in similar ways. But I do want scopes. So I take back the before-last sentence.
   To do this, I mangle vars, allowing scopes to exist, multiple instances of the same code to run, and to have variables not collide.

For loops

A smaller sections, but still cool.

A for loop in scratch is a hidden–but supported–block. You can only iterate a variable, lets call it x, from 1..y inclusive, where y is the max range.

So for lists, this means lowering this:

for (x, myList) {
    looks.say(x);
}

into this:

for (x, listLength(myList)) {
    x = myList[x];
    looks.say(x);
}

Additionally, if doing a range function like this:

for (x, range(4, 10, 2)) {
    looks.say(x);
}

it lowers into this:

for (x, 3) {
    x = x * 2 + 4;
    looks.say(x);
}

Project depth increase

Retrospectively, this probably should’ve been multiple seperate posts. But I forgot. Again.

Return statements

For compiling to scratch, a variety of different appracohes have been done. Personally, these fall under a few distinct approaches: variable and list. Variable is simply setting a variable to the output, whereas a list is just a stack following LiFo rules.

A variable should be used for single value, non-recursive functions. It’s pretty naive actually. The better alternative way is to use a stack, in which you push and pop from it. This way, you know the exact offset that correlates to the return value in recursive functions.

The stack also allows for easy nesting of functions next to each other, like foo(1, 2) + foo(3, 4), which is only possible in variable form with temporary vars.

Vscode Autocomplete

Less said here, but this was actually pretty simple. I just ran some regex to get all the identifiers, and then seperated them by type. Pretty rudamentry, but nice.

Scratch Defs

This was a huge first step in the IR design. Basically, I need a good way of storing metadata. Inventing whole new katnip syntax for this would:

a. Look bad, but also
b. Be horrible to implement

So I wrote a typescript file that takes care of the json metadata, allowing only rudamentry tags and opcode to be stored in Katnip portion.

Imports

This was a fun portion to tackle. Because I am targeting 3 deployments: vsc, cli, and web, I need a design that covers all 3. A friend suggested callbacks. This allows me to assume the user will define a function that allows filesystem interactions, without me touching a thing.

So all I did was bring in the code from the imported file, and ran compilation over it too. I pulled it in during the semantic analysis phase as that was the correct timing; after verifying the import statement; before the codegen.

Structs

Lots of deliberation occured here. It was very crucial to implement this, personally, because I use this type of feature (self-implemented, each time) in scratch. It always felt like a thing that I needed.

I was torn between strided and parallel list storage for the pieces of it. Strided was cool, and was more efficient with space and project list bloat, but in the end parallel lists were just SO much cleaner.

Bugfixes + webprep

This will be a shorter devlog, because, though there is a lot of time logged, much of this was just troubleshooting the few things I was adding.

Bugfixes:

• In the lexer, there existed some issues with my fsm, and the way I processed escaped characters
• Added some protections in the parser instead of semantic analysis so enforcement was throurough and non-manual

Webprep:

• Removed external dependencies that only npm could run. This enabled the beginning of development of a web editor gui by my friend in Svelte
• Rewrote dependencies and npm-specific stuff to my npm-agnostic, using my own implementations (mostly just pico-colors tbh)

The photo attached is it showing how another runtime (in this example, bun) can run the project that originally only worked with npm.

Getting ready!

Getting ready to ship the semantic analysis. General config stuff, npm registry, vsc ext, open vsx, all those fun things.

1. https://www.npmjs.com/package/@katnip-org/compiler
2. https://marketplace.visualstudio.com/items?itemName=Katnip-org.katnip-vscode
3. https://open-vsx.org/extension/Katnip-org/katnip-vscode

Phase 2 complete (plus a lil extra)

Phase 2

Function overloads

This was an interesting task to take on. I had to first figure out how to match a function’s signiture to its call. I couldn’t simply check if a signiture existed or not, I had to check which variation it satisfied. This involved reworking the way I stored signitures, and allowed a single object to hold them all for a certain function name. This way, it was a lot cleaner and put together.

STDLIB

After working through the project, I realized that I needed a way to define the primitive blocks used in Scratch. I didn’t think adding them in as a Typescript object was elegant, and it definently wasn’t extendable. Instead, I opted to expand the language syntax to allow generics, and therefore enable modular and clean code in the form of a katnip-defined STDLib.

Katnip LSP

This section is less in depth, because the setup wasn’t as hard. I look up some docs, and most of it was drag and drop. There exists a single file for defining the regex expressions with which to color the words on the page, and then I connected my error reporter to the lsp’s standard error reporter to further enhance the experience. Every 300ms or so, it updates, and tells the user where the errors are, just like any other language supported in text editors.

Almost completed phase 2 of the Semantic Analysis

Today I built a wide variety of things. But 3 specific features.

1. visit(node: StatementNode): void
   This function parses all of my nodes. It covers variable assignment, sprite declaration, etc etc. It has many jobs, including entering scopes, declaring bodies, and following each AST-node stack recursively. Some are left stubbed out, as I spent most of my time on #2:

2. inferType(expression: ExpressionNode): InternalType
   This function is ~200loc. It covers a wide variety of cases, all pertaining to the type inference of an expression AST. It not only infers the type, but recursively builds an internal model of what the user has written. So a type of list<dict<num, str>> becomes the stored internal representation of:

{
    kind: "list",
    element: {
        kind: "dict",
        key: {
            kind: "primitive",
            name: "num"
        },
        value: {
            kind: "primitive",
            name: "str"
        }
    }
}

Though it is a lot more complex than that, that’s what it boils down to. And lots and lots and lots of error statements.

1. InternalTypes itself
   I built a system that stores the type state, with a few extra helper functions: isAssignable and typeToString. The former recursively traces both types and ensures total match. This means that a variable of type to could be assigned to a value of type from. The latter is just a pretty-print function for pretty errors to see what types were being looked at/expected at certain parts of the code.

Finished phase 1 of the Semantic Analysis

Implemented Scope-Analysis checks. It can correctly identify and point out issues with scopes. It understands a variety of different scopes, from for-loops to functions to Sprites.

Also added support for switch-statements. Importantly, I chose to force only 1 default, and force it to be only at the end.

I also implemented true and false as a new Boolean primitive type, alongside a true return statement

2. Building Semantic Analysis

This will be shorter, since most of what I say is better just written in code.
The main idea is two passes:

1. Hoist
   Bring procedures to the top to allow forward references. I explain why I did so above.
2. Visit
   Acctually go through and validate. This is the main core bit of the semantic analysis portion.

I am a little stuck at a desgin fork. Should I allow scripts inside no ‘Sprite’ Scope to be there? Or should I force them into the Stage. Or do I error?
Scratch-like behaviors tell me that it should go to stage, but that feels tacky and wrong.

• Can functions be public? I think so. Can’t be temp.
• Need to make start and end possibly null
• Make function be able to be public/private. Assumed private
• Everything at the root of a file should be invalidated. The space should be reserved for imports, functions, global stuff.

Overall, I worked on building semantic analysis, at least the basic parts. I added variable decleration handling in scopes, and all of expressionstatement parsing.

Semantic Analysis Planning

Currently trying to think about how best to implement semantic analysis. I can’t lie, I am just getting back into this project, so I am rediscovering what I had previously written.I guess I have to start at the beginning. What is semantic analysis?

1. Type Checking   
   • Operations need to be with compatible data types   
   • One note to myself here is that I have reserved functions because it’s Scratch, so those need to have a designed store for their types2. Scope Resolution   
   • Make sure variables and functions are designed in the right parts of code   
   • I think this will cover imports too, but its worth thinking about   
   • Undeclared and redeclared variable checks
2. Function Call Validation   
   • I’ll have to go back and check if I accidentally covered that in the parser, but I believe this should go here
3. Flow Control   
   • Switch case validations, make sure fall through keywords and default cases etc. are all in valid places5. Reserved Keyword   
   • Check if variable and functions defined are conflicting with Katnip’s

First I’m going to do symbol collection and scope resolution.
One important nuance I discovered was whether or not to enforce scope declaration. Should there be an inferred scope?
In Scratch, this inferred scope is mostly ‘Public’, as variables are shared across sprites. However, I think I am going to flip this, as it feels wrong to teach the user that variables work across sprites.

Thought dump:

• ‘Private’ kward can’t be used at root level (stage). Can’t be used as modifier for sprite either, tho neither should any modifier.
• Scope-less definitions are okay. Scratch does this, just opposite of me.
• I am going to raise function declarations so that they can be referenced before making. Requires 2 passes, but seems simpler and more intuitive in my opinion.
• I am going to allow function overloading. It seems useful, and I like the way Java does it.