Devlog 7: I optimized streaming! Not only that, my mini PC has integrated graphics that apparently makes LLM go zoom, so I got a really large unanticipated speed boost :0. I got to increase the generated tokens all the way to near max, so y’all can get more gibberish faster! Going to ship this version :)
Comments 2
This is peak
sliding windows
Sign in to join the conversation.