first: why this is one giant devlog instead of a bunch of small ones 🤔

ok so quick context before the actual product dump

ive been shipping this under hack club macondo this whole time. i had multiple journals (devlogs) posted there everything was fine until i tried to ship it which then hit me with the ‘u r ‘old’ unc’ so i had to shift programs and here i am

that means im not gonna re-post every old macondo update one by one. this single post is the catch-up. everything we built so far in one place. after this normal session-by-session posts continue on stardance

live: https://slopmark.vercel.app
repo: https://github.com/nothariharan/benchmark

---

what slopmark even is

slopmark = the honest slop detector

public ai benches are cooked. contamination. saturation. goodhart. vibes scoring. llm-as-judge lying to itself

so we built an open eval platform where every model eats the same harness and a rule-based verifier decides pass/fail. never another llm judging another llm

spine is dumb on purpose:

novel task → fixed harness → behavioral verifier → score + receipt

if a model sounds smart but misses the contract it fails. that is the whole product

---

architecture real quick

next.js on vercel · eval in lib/harness + lib/eval · providers: openrouter / aiml / fireworks / byok · ~364 seed tasks · rule verifiers only

deploy trick: vercel has no sqlite so curated results are committed as results.json and the site just reads them. challenges = curated forever sprints. sessions = ad-hoc byok runs that dont pollute the scoreboards

ok architecture done. sessions are the real work

---

the sessions

revisitable sprints. fixed tasks. fixed models. fixed harness. run the grid. commit the receipt. anyone can reopen later and see who failed what

1. niche sprint v1

https://slopmark.vercel.app/challenge/niche-sprint-v1

6 aiml models x 10 mixed niche tasks = 60 runs. zero_context. procedural + extract + html + 8-word constraint + json + regex

gemini 2.5 flash 90% · deepseek v3 70% · gpt-4o mini + llama 3.1 8b 60% · ministral + qwen 50%

every model ate dirt on the 8-word constraint. that one task justified the whole sprint

2. drawing contest v1

https://slopmark.vercel.app/challenge/drawing-contest-v1

text models only. raw svg. no image models. html_contract verifier + live gallery

10 draws (house cat snowman bicycle rocket etc). 5 fireworks models

gpt-oss 120b + glm 5.1 = 100%. rest at 90%. common miss: rocket flame color. looking fine is not passing

3. game night v1

https://slopmark.vercel.app/challenge/game-night-v1

same fireworks lineup. party-game instruction contracts: no-e pizza lipogram, pirate wifi, questions-only db explain, taboo cat, 12-word black holes, printer haiku with jam, ALL CAPS chef, haunted house review, internet in one word

gpt-oss 90% · deepseek 60% · kimi 30% · glm 5.2 20% · glm 5.1 10%

drawing said everyone can draw kinda. game night said almost nobody can shut up and follow a party rule under zero context

---

everything else around it

bench + byok · build your own sprint (/challenges/new) · sessions wall · realshot duels · goal games · hall of shame · leaderboard

started as arena, pivoted to verifier-first slopmark, stripped auth so vercel just works

---

where we at rn

shipped: verifier engine, 3 public sessions with committed receipts, byok + ad-hoc sprints, realshot/goal/shame/leaderboard, vercel json deploy

this post is the macondo → stardance handoff. product is slopmark. program is now stardance. building continues from here