Dev Log #1 — Why ThinTensor Exists

SafeTensors is already a very good format for what it was designed to do: store tensors safely. It keeps tensor metadata simple. For normal model distribution, that is enough.

The problem starts when the tensor file becomes the runtime boundary.

That is enough if another framework already knows the model class and execution graph. But for an inference runtime, especially one focused on single-token decode, those fields are not enough. The runtime needs to know what each tensor means.

The .thin Format

The first .thin archive is intentionally simple:

.thin = fixed header + JSON manifest + page table + raw page data

The Rust implementation defines the archive identity and header size directly:

pub const MAGIC: &[u8; 8] = b"THINv0\0\0";
pub const HEADER_LEN: u32 = 88;

The header stores the physical structure of the archive:

pub struct Header {
    pub header_len: u32,
    pub version: u32,
    pub manifest_off: u64,
    pub manifest_len: u64,
    pub page_table_off: u64,
    pub page_count: u64,
    pub data_off: u64,
    pub archive_hash: [u8; 32],
}

This lets the loader open the file without guessing. It reads the header, jumps to the manifest, then reads the page table, then maps the raw page data.

The manifest is JSON because the early format needed to stay inspectable. It contains the model-level information needed to reconstruct decode:

architecture
hidden size
number of layers
attention heads
KV heads
head dimension
vocabulary size
RoPE settings
tensor names
tensor shapes
tensor dtypes
tensor roles
quantization metadata
runtime/backend hints
validation metadata

The page table connects the manifest to real bytes inside the archive:

pub struct PageTableRecord {
    pub page_id: String,
    pub offset: u64,
    pub stored_size: u64,
    pub raw_size: u64,
    pub flags: u32,
    pub checksum: [u8; 32],
}

A page record gives the runtime a verified byte range, not just a tensor name. This matters because pages can later be scheduled, streamed, cached, kept on CPU, moved to GPU, or assigned different execution policies.

Packing and Verification

The packer does not blindly write files into an archive. It validates the manifest first:

let report = validate_manifest(&manifest);
if !report.is_ok() {
    bail!("manifest invalid:\n{}", report.errors.join("\n"));
}

Then every page declared in the manifest must exist and match its declared size:

if bytes.len() as u64 != page.size {
    bail!("page declared size does not match file size");
}

It also verifies the page checksum:

let checksum = blake3::hash(&bytes);
if checksum.as_bytes().as_slice() != expected.as_slice() {
    bail!("page checksum does not match manifest");
}

Only after validation does the writer emit the archive:

write_header(&mut file, &header)?;
file.write_all(&manifest_bytes)?;
for record in &records {
    write_page_table_record(&mut file, record)?;
}
for page in pages {
    write_page_source(&mut file, page)?;
}

So the final file is structured as:

[ header ]
[ manifest JSON ]
[ page table records ]
[ raw page blobs ]