Alot of updates this time
Updates
Brand new scraper crate separate from app, dk if I have mentioned it before in devlogs.
It does slack and hackatime scraping all in one so we can have the main app deploys not interrupt scrapes.
Updates^2
- Added “a” keyword - Alias for “all time”
- Updated settings.rs and example.env to fit new defaults
- Fixed refresh_daily_stats failing on a second
- Fixed a bunch of ::bigint needing to be everywhere a sum() was decoded
- switched scraper to mimalloc allocator
- Replaced ts with real message_tis / 1000000 (I dont know if i put enough/too many zeros here)
- Fixed slack time recomputing in new postgredb
- Precomputes page stats now
- Removed a useless half-hourly job
- ON CONFLICT now only fires when counts actually change
- Added DB available probe to tests
- Better logging (disabled timestamps in logger)
- Added a mark_seen dedupe
- Updated hackatime sync logging to be less taking up all logs
- Added zstd + drop duplicate tables
- Removed zstd as postgre18-alpine does not come with it and I am not rescraping slack
- Only the scraper can init tables now
I don’t think im missing anything else i did,
If I am though then sorry gois 
Also see in one of the images below is total ram and cpu usage. somehow the postgre db is using over 4.5G of ram 
Comments 0
No comments yet. Be the first!
Sign in to join the conversation.