Finished lexer :P

Changes

• Mini cli (kindaaaa just for testing atm)
• Finished implementations of token handling for numbers, comments, and the other stragglers
• Removed the raw"..." keyword in favor of a simpler-looking `...`
• Smol error reporter because … its necessary idk 😭

Lexer pt. 1

Background

After messing around in my other projects with regex, I came to the conclusion that regex is hard to read, and hrad to write. Lets fix that shall we?

Design

I want the language to feel simple, and ideally be simple under the hood. simple == !regex. For the whole run down on all of my keywords and symbols, check out the Token definition file.

But we both know you’re lazy, so heres a tl;dr rundown:

• Define functions for sub patterns using parameters alongside a standard function shape: def dotted(part) = { part repeat { "." part } }
• Define variable-esque patterns to reuse, so you don’t bloat your regex:
  let buildId = repeat 1.. { num}
• Capture groups using the capture kward and use that output directly by importing the .grex file using a typescript import (yes really, no nrn, this lang is a WIP)
• And SO SO SO much more. I highly recommend you read the token file.

Comparison

Don’t trust me? Adament on writing your own regex? Tell me what this regex means in 10 seconds:
^(?=.*\d)(?=.*[a-z])(?=.*[A-Z])(?=.*\W)(?!.*(?i:password))\S{12,}$
..
..
Yea no, thats near impossible. Now try this:

/** Secret must contain each class, avoid the obvious, and be 12+ non-space. */
pattern StrongSecret {
  start
  ahead { repeat { any } digit }
  ahead { repeat { any } ['a'..'z'] }
  ahead { repeat { any } ['A'..'Z'] }
  ahead { repeat { any } not word }
  notAhead { repeat { any } ignoreCase { "password" } }
  repeat 12.. { not space }
  end
}

You may be saying: “you cheated, you used a comment!”. Well, exactly. Regex doesn’t have comments. G-Rex does.