GAI v0.6.0 – what changed since v0.5.0

Hi everyone 👋

My last devlogs were about GAI v0.5.0 and the new prompt builder, history system, and token budgeting.

I originally thought the next release would mainly contain a few fixes. But while using GAI in a real application, I kept finding places where the abstractions were still too simple.

Around 280 commits later, this turned into a much bigger release.

The most important changes

• Added full OpenAI and Anthropic providers, alongside the existing Gemini and Mistral integrations.
• Added provider-native multi-turn messages instead of putting the whole conversation into one large rendered prompt.
• Added Workflow.RunEvents, which returns one ordered stream for tokens, retries, iterations, errors, cancellation, and completion.
• Added model capability descriptors, so applications can check whether a model supports tools, structured output, reasoning, tokenization, and native messages.
• Added local request validation, so unsupported combinations fail before making a provider request.
• Added an optional OpenAI Responses API transport with streaming, tool continuation, reasoning context, usage, and better error handling.
• Added workflow middleware for things such as memory extraction, auditing, formatting, and evaluations.
• Added OpenTelemetry tracing and an optional Langfuse exporter.
• Reworked the README and added a complete runnable order-support agent example.
• Added a lot more provider, workflow, tool, streaming, and end-to-end tests.

One important problem I fixed

Previously, a model could receive tool definitions twice:

1. Through the provider’s native tool API.
2. As a JSON tool protocol inside the rendered prompt.

This wasted tokens and gave the model two different ways to call the same tool.

GAI now decides between native and text-based tool transport once when the workflow is created. Native models receive AIRequest.Tools, while other models receive the text fallback. The runtime tools remain executable in both cases.

The hardest part

Honestly, adding more providers was not the hardest part.

The difficult part was keeping one shared Go API without hiding useful provider-specific features.

OpenAI, Anthropic, Gemini, and Mistral all represent tools, history, reasoning, streaming, and errors differently. Making everything look identical would remove useful features, but exposing every provider detail would make the abstraction pointless.

The new capability descriptor system was my solution. A feature can be supported, unsupported, or unknown, and GAI can validate requests based on the information it actually has.

Smaller API changes

Because GAI is still pre-v1, I also cleaned up some APIs:

• NativeMessageBuilder now returns both the compatibility prompt and native messages together.
• NativeToolModel is deprecated in favor of the more complete ModelDescriber.
• Workflow.RunEvents is now the recommended API for primary-agent streaming.
• The older three-channel workflow API remains for middleware compatibility.

What I learned

The biggest lesson from this release is that an abstraction should not pretend every provider works in exactly the same way.

It is better to expose capabilities clearly and fail early than to discover an unsupported combination after opening a stream or calling a tool.

GAI is still pre-v1, but it now feels much more like a real agent runtime instead of only a collection of prompt and loop helpers.

Repository:

https://github.com/lace-ai/gai

finally I made it v0.5.0 is finish!!

v0.5.0 is released

• Simpler context.Builder API
• Turn-aware history with summarization
• Richer OpenTelemetry tracing + debug events
• Cleaner agent and model APIs
• Better token accounting
• Legacy prompt, RAG, and session abstractions removed

https://github.com/lace-ai/gai

What I updated:

• The history source to include summaries
• summary agent
• Prompt Renderer

The history source now summarieses the past conversation if it doesn’t fit into the token budget.

You can configure it like this:

history, err := history.New(sessionID, store, &history.SummarizerDefinition{
	Model:   model,
	Enabled: true,
})

or create a custom summarizer and pass it in the SummarizerDefinition

The Summary agent got some bug fixes and better error handling

It doesn’t panic if no model is configured :)

The Renderer and the whole Prompt builder is updated with the focus on better renderer final prompts

• there is now a Render Part function that turns a part into a RenderNode
• Part and RenderNode can be recursive / hierarchical

the default XML Renderer renders something like this:

<system>...</system>
<history>
    <user>...</user>
    <assistant>...</assistant>
    <user>...</user>
    ....
</history>
....

Hi, everyone 👋

This is the Devlog of what changed in gai v0.5.0

I have updated the prompt builder and the budget handling for gai, the last couple of days

Before there was a AI generated, huge and complex builder full of bugs and problems.

Now gai has an intuitive simple and flexible to use prompt builder with build in budget managing

The whole builder is build on the concept of an stack that is separated in three parts:

• System Instructions
• Context
• User Prompt / Current Loop

System Instructions:

These are just static parts that could be just text but also something else, doesn’t matter.

There is a helper for loading instructions from a file, that turns these directly into a part that can be used as system instructions.

systemInstructions, err := gaictx.LoadPromptFromFile(pathToFile)

Context

Context is special, it is used for dynamic sources, like history. These sources get rendered at the start of the loop and are just functions that return a part.

A source is just an interface:

type ContextSource interface {
	Name() string
	Function(ctx context.Context, TokenBudget int) (Part, error)
}

Currently there is just the history source, but in the future I will add some more, but they are meant for the user to implement them self.

Current Loop

The last section is just the user prompt and the last assistant massages or tool calls (if the assistant calls tools)
These are rendered per Iterations.
Nothing more.

Token Budget

The token budget is pretty simple, you pass in a token budget, and reserved output tokens. and the BuildContext function iterates over the sources and passes in the remaining token budget.

So Yes, if the first source needs all the budget, the last sources don’t have any budget left.

The internal flow is like this:

First you pass in the system instructions, context sources, and user prompt:

gaictx.New(gaictx.Definition{
	SystemInstructions: []gaictx.Part{
		systemInstructions,
		toolInstructions,
	},
	ContextSources: []gaictx.ContextSource{
		gaictx.NewHistory(sessionID, store),
	},
	UserPrompt: input.Text,
})

Now the stack looks like this: {[part,part], [source], [part]}

Then one time at the start of the loop the sources are build: {[part,part], [part], [part]}

parts, err := a.PromptBuilder.BuildContext(ctx)

On each iteration, the current iterations parts get added to the end, and the prompt gets rendered (default XML render)
{[part,part], [source], [part, part, part, part]}
=> <system>...</system><history><user>...</user></history><userPrompt>...</userPrompt><assistant>...</assistant><tool>...</tool>....

I also updated the way how to build reusable agents.

here is a complete example:

agent.New(agent.Definition{
		Name:  "user-assistant",
		Model: model,
		Tools: []loop.Tool{webSearch},
		Prompt: func(ctx context.Context, input agent.RunInput) (gaictx.PromptBuilder, error) {
			return prompt := gaictx.New(gaictx.Definition{
				SystemInstructions: []gaictx.Part{
					systemInstructions,
					toolInstructions,
				},
				ContextSources: []gaictx.ContextSource{
					gaictx.NewHistory(sessionID, store),
				},
				UserPrompt: input.Text,
			}), nil
		},
	})

Work that needs to be done until I publish the new version

• the history needs to use the summarizer
• some small bug fixes
• summery should be more dynamic
  if it summaries, the summary of turns 1-3 and turns 4-6, this should be somewhere saved, and persisted.
• maybe some more issues that I find while using the library

if someone is rely curios here is the pr: https://github.com/lace-ai/gai/pull/28