RAG pipeline!
Finally migrated the old notebooks to a rag-based one. Most of the changes are backend, so theres not much to show :(
- Files (PDFs, slides, lecture notes) were directly used as sources for notebooks before. This was slow, inefficient, and wasted tokens. Now, files are converted to markdown on upload.
- This markdown is then processed, and broken down into chunks in the rag pipeline, making chat faster and more accurate!
- Additionally, chat uses the new AI gateway/Openrouter provider to ensure fallback and consistent speeds regardless of provider downtime (plagued the app during flavourtown)
On preliminary testing, generation speeds jumped from 30s ttft to 10s ttft, with chat seeing the most improvements.
The next update should improve UI and refine subject prompts (a lil too robotic rn)!
Comments 0
No comments yet. Be the first!
Sign in to join the conversation.