GAI

finally I made it v0.5.0 is finish!!

v0.5.0 is released

• Simpler context.Builder API
• Turn-aware history with summarization
• Richer OpenTelemetry tracing + debug events
• Cleaner agent and model APIs
• Better token accounting
• Legacy prompt, RAG, and session abstractions removed

https://github.com/lace-ai/gai

What I updated:

• The history source to include summaries
• summary agent
• Prompt Renderer

The history source now summarieses the past conversation if it doesn’t fit into the token budget.

You can configure it like this:

history, err := history.New(sessionID, store, &history.SummarizerDefinition{
	Model:   model,
	Enabled: true,
})

or create a custom summarizer and pass it in the SummarizerDefinition

The Summary agent got some bug fixes and better error handling

It doesn’t panic if no model is configured :)

The Renderer and the whole Prompt builder is updated with the focus on better renderer final prompts

• there is now a Render Part function that turns a part into a RenderNode
• Part and RenderNode can be recursive / hierarchical

the default XML Renderer renders something like this:

<system>...</system>
<history>
    <user>...</user>
    <assistant>...</assistant>
    <user>...</user>
    ....
</history>
....

Hi, everyone 👋

This is the Devlog of what changed in gai v0.5.0

I have updated the prompt builder and the budget handling for gai, the last couple of days

Before there was a AI generated, huge and complex builder full of bugs and problems.

Now gai has an intuitive simple and flexible to use prompt builder with build in budget managing

The whole builder is build on the concept of an stack that is separated in three parts:

• System Instructions
• Context
• User Prompt / Current Loop

System Instructions:

These are just static parts that could be just text but also something else, doesn’t matter.

There is a helper for loading instructions from a file, that turns these directly into a part that can be used as system instructions.

systemInstructions, err := gaictx.LoadPromptFromFile(pathToFile)

Context

Context is special, it is used for dynamic sources, like history. These sources get rendered at the start of the loop and are just functions that return a part.

A source is just an interface:

type ContextSource interface {
	Name() string
	Function(ctx context.Context, TokenBudget int) (Part, error)
}

Currently there is just the history source, but in the future I will add some more, but they are meant for the user to implement them self.

Current Loop

The last section is just the user prompt and the last assistant massages or tool calls (if the assistant calls tools)
These are rendered per Iterations.
Nothing more.

Token Budget

The token budget is pretty simple, you pass in a token budget, and reserved output tokens. and the BuildContext function iterates over the sources and passes in the remaining token budget.

So Yes, if the first source needs all the budget, the last sources don’t have any budget left.

The internal flow is like this:

First you pass in the system instructions, context sources, and user prompt:

gaictx.New(gaictx.Definition{
	SystemInstructions: []gaictx.Part{
		systemInstructions,
		toolInstructions,
	},
	ContextSources: []gaictx.ContextSource{
		gaictx.NewHistory(sessionID, store),
	},
	UserPrompt: input.Text,
})

Now the stack looks like this: {[part,part], [source], [part]}

Then one time at the start of the loop the sources are build: {[part,part], [part], [part]}

parts, err := a.PromptBuilder.BuildContext(ctx)

On each iteration, the current iterations parts get added to the end, and the prompt gets rendered (default XML render)
{[part,part], [source], [part, part, part, part]}
=> <system>...</system><history><user>...</user></history><userPrompt>...</userPrompt><assistant>...</assistant><tool>...</tool>....

I also updated the way how to build reusable agents.

here is a complete example:

agent.New(agent.Definition{
		Name:  "user-assistant",
		Model: model,
		Tools: []loop.Tool{webSearch},
		Prompt: func(ctx context.Context, input agent.RunInput) (gaictx.PromptBuilder, error) {
			return prompt := gaictx.New(gaictx.Definition{
				SystemInstructions: []gaictx.Part{
					systemInstructions,
					toolInstructions,
				},
				ContextSources: []gaictx.ContextSource{
					gaictx.NewHistory(sessionID, store),
				},
				UserPrompt: input.Text,
			}), nil
		},
	})

Work that needs to be done until I publish the new version

• the history needs to use the summarizer
• some small bug fixes
• summery should be more dynamic
  if it summaries, the summary of turns 1-3 and turns 4-6, this should be somewhere saved, and persisted.
• maybe some more issues that I find while using the library

if someone is rely curios here is the pr: https://github.com/lace-ai/gai/pull/28