hi, I have some questions: How do you handle non-deterministic nature of LLMs? How do you handle that LLMs can halucinate on big codebases? How do you handle that big apps will just eat the entire context of the model?
Non-determinism: AiRY Core is a strict intermediate representation, so even if the LLM generates slightly different code each run, the Rust interpreter executes it deterministically. The non-determinism is contained to the translation layer.
Hallucinations: The prompt is very constrained — the LLM only has a small set of instructions it can output (set, show, read, if, loop, infloop, etc.). If it hallucinates something outside that set, the lexer/parser rejects it immediately with an error. It’s not generating free-form code.
Context window: You’re right, this is a real limitation right now. AiRY is designed for small scripts, not big codebases. For now the natural language input is expected to be short. In the future I plan to split programs into chunks and translate them separately.
Comments 2
hi, I have some questions: How do you handle non-deterministic nature of LLMs? How do you handle that LLMs can halucinate on big codebases? How do you handle that big apps will just eat the entire context of the model?
Hey! Great questions. Honest answers:
Non-determinism: AiRY Core is a strict intermediate representation, so even if the LLM generates slightly different code each run, the Rust interpreter executes it deterministically. The non-determinism is contained to the translation layer.
Hallucinations: The prompt is very constrained — the LLM only has a small set of instructions it can output (set, show, read, if, loop, infloop, etc.). If it hallucinates something outside that set, the lexer/parser rejects it immediately with an error. It’s not generating free-form code.
Context window: You’re right, this is a real limitation right now. AiRY is designed for small scripts, not big codebases. For now the natural language input is expected to be short. In the future I plan to split programs into chunks and translate them separately.
Sign in to join the conversation.