You are browsing as a guest. Sign up (or log in) to start making projects!

Open comments for this post

1h 7m 44s logged

I caught a bad bug where the extractor was hallucinating numbers: I ran the same AP unemployment article three times and the reported rate came out as 4.2%, 4.1%, and 3.9% across the three runs when the article clearly says 4.2%. The model was filling in from training data instead of reading the text in front of it, which is a subtle failure mode because the numbers look plausible enough that you would never notice without explicitly checking. The fix was adding a verbatim extraction instruction to the system prompt and setting temperature to 0.

0
3

Comments 0

No comments yet. Be the first!