Devlog — Document Ingestion
Started the document ingestion pipeline for Athenaeum.
Completed:
- Added PDF text extraction using pypdf
- Added PDF upload endpoint
- Connected document extraction to the Knowledge System
- Automatically created knowledge items from uploaded PDFs
- Preserved authenticated user ownership
- Stored extracted document content in PostgreSQL
Current API:
POST /api/v1/knowledge
GET /api/v1/knowledge
GET /api/v1/knowledge/{KnowledgeItemId}
PATCH /api/v1/knowledge/{KnowledgeItemId}
DELETE /api/v1/knowledge/{KnowledgeItemId}
POST /api/v1/knowledge/upload
Next:
- Add document chunking
- Generate embeddings
- Store vector representations
- Implement semantic search