Hi, everyone 👋

This is the Devlog of what changed in gai v0.5.0

I have updated the prompt builder and the budget handling for gai, the last couple of days

Before there was a AI generated, huge and complex builder full of bugs and problems.

Now gai has an intuitive simple and flexible to use prompt builder with build in budget managing

The whole builder is build on the concept of an stack that is separated in three parts:

• System Instructions
• Context
• User Prompt / Current Loop

System Instructions:

These are just static parts that could be just text but also something else, doesn’t matter.

There is a helper for loading instructions from a file, that turns these directly into a part that can be used as system instructions.

systemInstructions, err := gaictx.LoadPromptFromFile(pathToFile)

Context

Context is special, it is used for dynamic sources, like history. These sources get rendered at the start of the loop and are just functions that return a part.

A source is just an interface:

type ContextSource interface {
	Name() string
	Function(ctx context.Context, TokenBudget int) (Part, error)
}

Currently there is just the history source, but in the future I will add some more, but they are meant for the user to implement them self.

Current Loop

The last section is just the user prompt and the last assistant massages or tool calls (if the assistant calls tools)
These are rendered per Iterations.
Nothing more.

Token Budget

The token budget is pretty simple, you pass in a token budget, and reserved output tokens. and the BuildContext function iterates over the sources and passes in the remaining token budget.

So Yes, if the first source needs all the budget, the last sources don’t have any budget left.

---

The internal flow is like this:

First you pass in the system instructions, context sources, and user prompt:

gaictx.New(gaictx.Definition{
	SystemInstructions: []gaictx.Part{
		systemInstructions,
		toolInstructions,
	},
	ContextSources: []gaictx.ContextSource{
		gaictx.NewHistory(sessionID, store),
	},
	UserPrompt: input.Text,
})

Now the stack looks like this: {[part,part], [source], [part]}

Then one time at the start of the loop the sources are build: {[part,part], [part], [part]}

parts, err := a.PromptBuilder.BuildContext(ctx)

On each iteration, the current iterations parts get added to the end, and the prompt gets rendered (default XML render)
{[part,part], [source], [part, part, part, part]}
=> <system>...</system><history><user>...</user></history><userPrompt>...</userPrompt><assistant>...</assistant><tool>...</tool>....

---

I also updated the way how to build reusable agents.

here is a complete example:

agent.New(agent.Definition{
		Name:  "user-assistant",
		Model: model,
		Tools: []loop.Tool{webSearch},
		Prompt: func(ctx context.Context, input agent.RunInput) (gaictx.PromptBuilder, error) {
			return prompt := gaictx.New(gaictx.Definition{
				SystemInstructions: []gaictx.Part{
					systemInstructions,
					toolInstructions,
				},
				ContextSources: []gaictx.ContextSource{
					gaictx.NewHistory(sessionID, store),
				},
				UserPrompt: input.Text,
			}), nil
		},
	})

---

Work that needs to be done until I publish the new version

• the history needs to use the summarizer
• some small bug fixes
• summery should be more dynamic
  if it summaries, the summary of turns 1-3 and turns 4-6, this should be somewhere saved, and persisted.
• maybe some more issues that I find while using the library

---

if someone is rely curios here is the pr: https://github.com/lace-ai/gai/pull/28