Starweb dev log #8

Page scripts can fetch() now.

fetch("/api/time", function(err, res) ... end) runs on a worker thread and calls you back when it’s done, so the page never freezes. You get res.status, res.ok, res.body, res.headers, and res:json(). POST works too: fetch("/api/echo", { method = "POST", json = { hi = "there" } }, cb). There’s a json global with it (json.encode/json.decode).

Limits: 6 fetches at a time, 1MB request body, 8MB response, caps on headers, URL length, and JSON depth. Cross-origin requests are CORS-gated by browsers; a fetch to another host only returns a body if it sends Access-Control-Allow-Origin. The Python server got a per-route cors= option to match.

Also refactored the fetcher: one perform_request with RequestOptions (method, headers, body, timeouts, max size), and perform_fetch is a thin wrapper for navigations on top. Scripted fetch and the address bar share code now instead of two separate socket loops.

StarWeb dev log #7

You can write StarWeb backends in Python now.

Everything so far has been C++. Writing a page that did anything meant static files or a Lua script in the browser. So there’s now a starweb Python package that speaks STWP, in two halves.

The server half looks like Flask. Make an App, decorate functions with @app.route("/api/time"), return a dict, and it goes out as JSON, or a Response for control over status and headers. Routes take params (/api/greet/<name>), the request has a query dict, and app.mount_static("/", "www") serves files with routes winning over static. app.run(scheme="both") hosts moon:// and star:// at once, thread per connection, same as the C++ server.

The client half looks like requests. starweb.get("moon://localhost/"), .post(...), a Session to reuse a connection, and TLS for star:// with each failure getting its own exception (TLSVerificationError, MixedContentError, ALPNError).

it only speaks STWP

The interesting part is what it refuses. Send the server an HTTP request line, and it answers 505 and hangs up; curl and a browser can’t wander in by accident. The client trusts only the StarWeb root CA, never the system roots, and it’s pinned to TLS 1.3 and nothing else. ALPN is required: no stwp/1.0, connection dropped.

the CLI

starweb get moon://localhost/index.html fetches a URL, -v for headers and TLS info. starweb serve app.py runs a file defining an App, with --scheme, --no-tls, --log. curl and a dev server in one command.

It’s a proper pip package too: README, license, pyproject.toml, pip install starweb and you get a starweb command. And there are tests: interop against the C++ server, URL and message parsing, schemes, isolation, which caught a couple of parser mismatches between the two.

StarWeb dev log #6

StarWeb has secure connections now: star://.

star:// is to moon:// what https is to http: same STWP messages; the transport underneath is just TLS 1.3 (OpenSSL 3) instead of plaintext TCP. Plaintext moon:// stays on port 8090, encrypted star:// runs on 8490, and the server serves both at once. There’s a tools/make_certs.sh that spins up a local root CA and a server cert so you can test it. The CA only signs localhost, .local/.star, and private IPs (10.x, 192.168.x, etc.), so it works on your LAN but not the public internet. (I it will be public after i make DNS)

The address bar got a lock icon. Click it, and you get a cert info popup: who issued it, who it’s for, that kind of thing. If the handshake fails, verification fails, cert’s self-signed, expired, wrong host, you get a full-page interstitial instead of the page, same as a real browser yells at you.

the security rules

A star:// page can’t pull in moon:// content. Mixed content is blocked; no loading plaintext resources into a page you’re being told is secure.

Downgrades are blocked too, and stricter than the normal web; here, a script can’t navigate you from star:// to moon://. You can still type a moon:// URL yourself; that’s your call, but a page can’t quietly drop you off TLS.

TLS also enforces ALPN and checks the hostname, and there’s a session resumption cache so reconnecting doesn’t redo the whole handshake every time.

Also fixed URL parsing to handle IPv6 addresses in brackets ([::1]), which it just choked on before.

StarWeb dev log #5

Lua support!

Every tab gets its own ScriptEngine, a sandboxed Lua 5.4 interpreter. A custom allocator caps its memory, an instruction hook kills it when it runs past its time budget, and require, dofile, and loadfile don’t exist in there. It’s untrusted code from whatever page you loaded.

Scripts get a DOM API: getElementById, querySelector, innerText, get/setAttribute, addEventListener, console.log, alert, setTimeout/setInterval, requestAnimationFrame. There’s a location object too, so scripts can navigate, but only to moon:// URLs.

Yes, there is <canvas>. getContext("2d") gives you fillRect, paths, arcs, fillText, but scripts never touch the screen directly; every call gets recorded into a list of CanvasOps and the renderer draws the list. To test all of it, I wrote a raycaster (www/game.lua), a small maze you can walk around in, running entirely as a page script.

external scripts and keyboard input

<script src="..."> works, not just inline scripts. The fetcher downloads them so the engine sees them in document order. Keyboard input works too; keys get polled from GLFW and sent to the active tab, so document.addEventListener("keydown", ...) does something. That’s how the raycaster gets its controls.

The main loop also stopped redrawing at 60fps when nothing’s happening. It sleeps until something needs a frame: a fetch, a playing video, a pending timer, or requestAnimationFrame. Good for the battery. (My MacBook 2015 was on fire when the browser ran. That’s how I noticed lol)

Also, from now on I will be posting more devlogs because im done with the core stuff. Rupnil () said not to make them that big and commit more.

StarWeb dev log #4

CROSS-PLATFORM SUPPORT AHHH!!! (BOTH MICRO SL.. WINDOWS AND LINUX! and yk it was working before on MacOS)

The socket code was the first problem. Everything was written against BSD sockets, which Windows doesn’t speak. I pulled all of that behind a net.hpp wrapper: a socket_t type, a kInvalidSocket constant, and helpers like net::close, net::is_valid, and set_recv_timeout that pick BSD sockets or Winsock2 depending on platform. Small thing, but an invalid socket is -1 on POSIX and an unsigned INVALID_SOCKET on Windows, so you can’t just reuse the same sentinel value everywhere like I was doing before.

Media playback was the other one. AVFoundation isn’t available outside macOS, so VideoPlayer now has two backends: AVFoundation on macOS, and a FFmpeg + miniaudio backend for Windows/Linux. Same play/pause/seek/volume API.

Alongside that: a CMake build, a BUILDING.md with setup steps per platform, GitHub Actions building targets on push, and a LICENSE + third-party notices file, and that needed to be documented properly instead of just… not. (FFmpeg licensing is so confusing)

Then: flexbox, and input types

Once the cross-platform stuff was done, I went back to layout, which had been held together by manual cursor-position math since day one. I imported Yoga (the flexbox engine Meta built for React Native) and wired it into the renderer through a compute_flex_layout call, so display: flex, gap, and the flex-* properties are now working.

The parser also handles text a lot better now. Before, it just dumped every raw character into text_content, so something like & showed up on the page as & instead of &. Now it decodes entities and splits text into its own #text nodes instead of gluing everything onto the parent tag.

Forms are also better now because of new input types: name, min, max, step, checked, and gave the renderer form controls instead of the placeholder boxes, plus calendar and clock widgets for date/time inputs. Submitting one actually resets and collects values now.

And we know what happens now…

StarWeb dev log #3

Finally made a media player!

The fetcher now actually looks at what it downloaded instead of assuming everything’s HTML. It checks the Content-Type header first, falls back to the file extension if the server didn’t send one, and sorts the response into HTML, image, video, or audio. Anything that doesn’t match any of those (a plain .txt file, for instance) just gets rendered as text on the page instead of the browser trying to parse it and displaying nothing. Images and media get pulled down recursively too. The fetcher walks the page for <img>, <video>, <audio>, and <source> tags and fetches each one alongside the page itself, the same way it already did for stylesheets.

Images go through stb_image, decoded from memory into an OpenGL texture. For video and audio tho, I built a small VideoPlayer wrapper around AVFoundation, so playback is currently macOS-only (sorry again, but I didn’t add support for Linux/Windows yet; just wait for next devlog, I promise). Fetched media gets cached to a folder on disk under a hashed filename, since AVFoundation wants an actual file to point at rather than a memory buffer.

resolve_url was improved. It now knows moon:// defaults to 8090 and star:// to 8490, so URLs don’t end up with a redundant :8090 attached onto every link.

@Rupnil sorry man, I ran into issues ;-;

StarWeb dev log #2

The client could fetch static files but couldn’t render them. Starweb finally has a browser called Starmap! The first version wasn’t much: a minimal parser splitting tags and attributes on whitespace, and a couple CSS properties. Only one page could be loaded at a time; it wasn’t much of a browser, but I wanted to at least render something.

The whitespace-splitting parser didn’t last long. It broke when an attribute value contained a space, which style="..." does constantly. So I rewrote it into a proper scanner that walks the tag character by character and actually respects quotes. That also made it easy to start picking up attributes I wasn’t tracking before: id, style (parsed into real properties instead of just stored as text), and type, value, and placeholder for form inputs. Nothing renders them yet, but the data’s there.

CSS grew from a couple properties to something you can work with: padding and margin per side, width, height, border-width, border-color, font-size, display, each with its own inheritance rule so one style doesn’t just blindly overwrite another. Headings finally scale too; h1 down to h6 each get their own font-size multiplier instead of rendering at the same size as everything else. Pages could finally pull in external styling too, via <link rel="stylesheet">: the fetcher walks the parsed page for stylesheet links, resolves them against the current URL, and fetches each one as a secondary request before the CSS gets parsed.

The layout change with the biggest visible impact was inline flow. Up to this point, every tag forced a new line, so a paragraph with a link in the middle of it rendered as three separate lines. Spans, links, buttons, and form controls now sit on the same line instead. <input> also stopped incorrectly behaving like a container element, since it shouldn’t be able to hold children.

Then I added tabs. Each tab now owns its own URL, page, style, and fetch state. The window title and each tab’s label now follow the page’s <title>, falling back to the host and path if the page didn’t set one. I also swapped the default OS window frame for custom traffic-light controls (because I’m on a Mac and that’s how Chrome does it; sorry if it looks wrong on Linux or Windows, I haven’t tested yet). And also, resizing works correctly since the first version.

By this point, the browser source had grown to 1700+ lines of code, holding everything: parsing, fetching, theme, and rendering. I split the browser into modules, leaving the main file responsible only for booting up and driving the UI loop.

Woah, that was a lot, but I managed to do it in 3 days? idk how to count it because my sleep schedule is so cooked that I often worked at night and went to sleep at 4 am.

StarWeb is my attempt at building an internet of my own. A custom protocol, a custom URL scheme, and a client and server that communicate using them.I put together STWP, my own protocol with request and response parsing, structured similarly to HTTP but built from scratch. The server runs multithreaded, serving files from a www folder, blocking path traversal attempts, and returning proper error codes when something’s missing or wrong. The client connects using its own URL format, moon://host:port/path, sends a request, and prints back whatever the server responds with. Essentially my own version of curl made for testing; I’m planning to make a browser later. Right now it only handles fetching static files.