What did you make?
An assistant for a salesperson, not another lead list. You give it a customer profile, it walks the Czech public registers, and once a week it hands back a handful of companies with something that actually changed this week — each with a full dossier where every claim carries the exact quote it came from and a link to the page it was found on. Built as a take-home assignment for RTsoft (they sell custom shop-floor management software), and it’s live in production, not just running on my machine.
What was challenging?
Hallucination is the whole risk of this task, not a side concern — an invented fact about a company that a salesperson pastes into an email is worse than no output at all. So the model never gets to assert anything; it proposes a claim + a quote, and my code searches for that exact quote in the page text I archived. First time I ran a verifier that rechecked stored claims against their own archived pages, 100 of 172 supposedly-verified facts failed — all from one code path, the part reading the state register, which had been marking its own claims “verified” without ever passing them through the check. The other hard part: I assumed a good weighted score could rank “why this company.” Two separate experiments — a permutation test and a 1536-dim embedding of company sites — both came back at chance. The site text just doesn’t say who’s buying software. Had to throw the score out and rebuild ranking around what’s actually provable instead.
What are you proud of?
One line in the schema: claim.snapshot_id NOT NULL REFERENCES snapshot(id). A claim about a company physically cannot be inserted into the database unless it points at a stored page snapshot — “nothing without a source” isn’t a rule I have to remember, it’s a foreign key. Also proud that manual review keeps finding real bugs the statistics don’t: a solar-panel subsidy misfiled as “scaling up production,” a company ranked #1 whose own grant paperwork said they’d already bought the system — both caught by reading five actual cards, after two rigorous experiments said everything was fine.
What should people know so they can test it?
It only works for Czech companies — the sources are Czech state registers. python -m pipeline.run –check tells you exactly what’s missing before you run anything. The number that matters on every dossier is the last line — “N verified facts · N inferences · N discarded” — click a claim’s source link and check the quote is actually on that page; that’s the whole trust model, so it should be checkable, not taken on faith.
- 5 devlogs
- 34h