Devlog — Document Ingestion

Started the document ingestion pipeline for Athenaeum.

Completed:

• Added PDF text extraction using pypdf
• Added PDF upload endpoint
• Connected document extraction to the Knowledge System
• Automatically created knowledge items from uploaded PDFs
• Preserved authenticated user ownership
• Stored extracted document content in PostgreSQL

Current API:

POST /api/v1/knowledge
GET /api/v1/knowledge
GET /api/v1/knowledge/{KnowledgeItemId}
PATCH /api/v1/knowledge/{KnowledgeItemId}
DELETE /api/v1/knowledge/{KnowledgeItemId}
POST /api/v1/knowledge/upload

Next:

• Add document chunking
• Generate embeddings
• Store vector representations
• Implement semantic search