I caught a bad bug where the extractor was hallucinating numbers: I ran the same AP unemployment article three times and the reported rate came out as 4.2%, 4.1%, and 3.9% across the three runs when the article clearly says 4.2%. The model was filling in from training data instead of reading the text in front of it, which is a subtle failure mode because the numbers look plausible enough that you would never notice without explicitly checking. The fix was adding a verbatim extraction instruction to the system prompt and setting temperature to 0.
Comments 0
No comments yet. Be the first!
Sign in to join the conversation.