4h 59m 27s logged

Events endpoints and display

problem i faced

wow this was a pain T___T
so basically when i started working on the backend to receive events from lilbro i planned something like this

lil bro -> gets logs -> normalizes and understands it -> sends to backend -> dashboard displays it

and i created the neccessary endpoints and database tables schemas what not bruh and halfway through i realized: “damn what about logs??” and it snapped. i realized i wasted like 2 hours on this cuz this system was absolutely trash and not afforadable on resources wise cuz we cant shove a whole detection engine in agent, and we gotta think about it from analyst side too cuz they gonna see logs for investigations and cant just depends on pre-build event types

better system

so then i decided to go with this kinda system:

agent will see logs -> normalizes them -> sends to backend with logs and events in this kinda structure:

        {
            "id": 1,
            "host": {
                "id": "",
                "hostname": ""
            },
            "log": {
                "source": "",
                "raw": "",
                "ingested_at": ""
            },
            "event_type": "",
            "context":{},
            "created_at": ""
        },

this would allow the backend to store the events and also the logs related, it also allows saving events with logs cuz not every event is made from a log

and i think this is a nice system, cuz now we can show logs, events, and have a detection engine in backend to create alerts based on rules

so to build this i created:

Logs table, stores logs
Events table, stores events with log ids
/events/ - global event feed
POST /agents/id/event - endpoint for agents to send events

and on the dashboard side i have made /events which displays all events from all hosts in a table, it also has a filter for selecting hosts and/or event types, and also a limit.

next i will be working on sending events from lil bro