Project depth increase

Retrospectively, this probably should’ve been multiple seperate posts. But I forgot. Again.

Return statements

For compiling to scratch, a variety of different appracohes have been done. Personally, these fall under a few distinct approaches: variable and list. Variable is simply setting a variable to the output, whereas a list is just a stack following LiFo rules.

A variable should be used for single value, non-recursive functions. It’s pretty naive actually. The better alternative way is to use a stack, in which you push and pop from it. This way, you know the exact offset that correlates to the return value in recursive functions.

The stack also allows for easy nesting of functions next to each other, like foo(1, 2) + foo(3, 4), which is only possible in variable form with temporary vars.

Vscode Autocomplete

Less said here, but this was actually pretty simple. I just ran some regex to get all the identifiers, and then seperated them by type. Pretty rudamentry, but nice.

Scratch Defs

This was a huge first step in the IR design. Basically, I need a good way of storing metadata. Inventing whole new katnip syntax for this would:

a. Look bad, but also
b. Be horrible to implement

So I wrote a typescript file that takes care of the json metadata, allowing only rudamentry tags and opcode to be stored in Katnip portion.

Imports

This was a fun portion to tackle. Because I am targeting 3 deployments: vsc, cli, and web, I need a design that covers all 3. A friend suggested callbacks. This allows me to assume the user will define a function that allows filesystem interactions, without me touching a thing.

So all I did was bring in the code from the imported file, and ran compilation over it too. I pulled it in during the semantic analysis phase as that was the correct timing; after verifying the import statement; before the codegen.

Structs

Lots of deliberation occured here. It was very crucial to implement this, personally, because I use this type of feature (self-implemented, each time) in scratch. It always felt like a thing that I needed.

I was torn between strided and parallel list storage for the pieces of it. Strided was cool, and was more efficient with space and project list bloat, but in the end parallel lists were just SO much cleaner.

Bugfixes + webprep

This will be a shorter devlog, because, though there is a lot of time logged, much of this was just troubleshooting the few things I was adding.

Bugfixes:

• In the lexer, there existed some issues with my fsm, and the way I processed escaped characters
• Added some protections in the parser instead of semantic analysis so enforcement was throurough and non-manual

Webprep:

• Removed external dependencies that only npm could run. This enabled the beginning of development of a web editor gui by my friend in Svelte
• Rewrote dependencies and npm-specific stuff to my npm-agnostic, using my own implementations (mostly just pico-colors tbh)

The photo attached is it showing how another runtime (in this example, bun) can run the project that originally only worked with npm.

Getting ready!

Getting ready to ship the semantic analysis. General config stuff, npm registry, vsc ext, open vsx, all those fun things.

1. https://www.npmjs.com/package/@katnip-org/compiler
2. https://marketplace.visualstudio.com/items?itemName=Katnip-org.katnip-vscode
3. https://open-vsx.org/extension/Katnip-org/katnip-vscode

Phase 2 complete (plus a lil extra)

Phase 2

Function overloads

This was an interesting task to take on. I had to first figure out how to match a function’s signiture to its call. I couldn’t simply check if a signiture existed or not, I had to check which variation it satisfied. This involved reworking the way I stored signitures, and allowed a single object to hold them all for a certain function name. This way, it was a lot cleaner and put together.

STDLIB

After working through the project, I realized that I needed a way to define the primitive blocks used in Scratch. I didn’t think adding them in as a Typescript object was elegant, and it definently wasn’t extendable. Instead, I opted to expand the language syntax to allow generics, and therefore enable modular and clean code in the form of a katnip-defined STDLib.

Katnip LSP

This section is less in depth, because the setup wasn’t as hard. I look up some docs, and most of it was drag and drop. There exists a single file for defining the regex expressions with which to color the words on the page, and then I connected my error reporter to the lsp’s standard error reporter to further enhance the experience. Every 300ms or so, it updates, and tells the user where the errors are, just like any other language supported in text editors.

Almost completed phase 2 of the Semantic Analysis

Today I built a wide variety of things. But 3 specific features.

1. visit(node: StatementNode): void
   This function parses all of my nodes. It covers variable assignment, sprite declaration, etc etc. It has many jobs, including entering scopes, declaring bodies, and following each AST-node stack recursively. Some are left stubbed out, as I spent most of my time on #2:

2. inferType(expression: ExpressionNode): InternalType
   This function is ~200loc. It covers a wide variety of cases, all pertaining to the type inference of an expression AST. It not only infers the type, but recursively builds an internal model of what the user has written. So a type of list<dict<num, str>> becomes the stored internal representation of:

{
    kind: "list",
    element: {
        kind: "dict",
        key: {
            kind: "primitive",
            name: "num"
        },
        value: {
            kind: "primitive",
            name: "str"
        }
    }
}

Though it is a lot more complex than that, that’s what it boils down to. And lots and lots and lots of error statements.

1. InternalTypes itself
   I built a system that stores the type state, with a few extra helper functions: isAssignable and typeToString. The former recursively traces both types and ensures total match. This means that a variable of type to could be assigned to a value of type from. The latter is just a pretty-print function for pretty errors to see what types were being looked at/expected at certain parts of the code.

Finished phase 1 of the Semantic Analysis

Implemented Scope-Analysis checks. It can correctly identify and point out issues with scopes. It understands a variety of different scopes, from for-loops to functions to Sprites.

Also added support for switch-statements. Importantly, I chose to force only 1 default, and force it to be only at the end.

I also implemented true and false as a new Boolean primitive type, alongside a true return statement

2. Building Semantic Analysis

This will be shorter, since most of what I say is better just written in code.
The main idea is two passes:

1. Hoist
   Bring procedures to the top to allow forward references. I explain why I did so above.
2. Visit
   Acctually go through and validate. This is the main core bit of the semantic analysis portion.

I am a little stuck at a desgin fork. Should I allow scripts inside no ‘Sprite’ Scope to be there? Or should I force them into the Stage. Or do I error?
Scratch-like behaviors tell me that it should go to stage, but that feels tacky and wrong.

• Can functions be public? I think so. Can’t be temp.
• Need to make start and end possibly null
• Make function be able to be public/private. Assumed private
• Everything at the root of a file should be invalidated. The space should be reserved for imports, functions, global stuff.

Overall, I worked on building semantic analysis, at least the basic parts. I added variable decleration handling in scopes, and all of expressionstatement parsing.

Semantic Analysis Planning

Currently trying to think about how best to implement semantic analysis. I can’t lie, I am just getting back into this project, so I am rediscovering what I had previously written.I guess I have to start at the beginning. What is semantic analysis?

1. Type Checking   
   • Operations need to be with compatible data types   
   • One note to myself here is that I have reserved functions because it’s Scratch, so those need to have a designed store for their types2. Scope Resolution   
   • Make sure variables and functions are designed in the right parts of code   
   • I think this will cover imports too, but its worth thinking about   
   • Undeclared and redeclared variable checks
2. Function Call Validation   
   • I’ll have to go back and check if I accidentally covered that in the parser, but I believe this should go here
3. Flow Control   
   • Switch case validations, make sure fall through keywords and default cases etc. are all in valid places5. Reserved Keyword   
   • Check if variable and functions defined are conflicting with Katnip’s

First I’m going to do symbol collection and scope resolution.
One important nuance I discovered was whether or not to enforce scope declaration. Should there be an inferred scope?
In Scratch, this inferred scope is mostly ‘Public’, as variables are shared across sprites. However, I think I am going to flip this, as it feels wrong to teach the user that variables work across sprites.

Thought dump:

• ‘Private’ kward can’t be used at root level (stage). Can’t be used as modifier for sprite either, tho neither should any modifier.
• Scope-less definitions are okay. Scratch does this, just opposite of me.
• I am going to raise function declarations so that they can be referenced before making. Requires 2 passes, but seems simpler and more intuitive in my opinion.
• I am going to allow function overloading. It seems useful, and I like the way Java does it.