- Rotate logs periodically - 5mins, 1hour etc
- Push roasted logs to say s3.
- Have a process that listens on s3 notifications, and on receiving one, pulls given log, extracts required info and inserts to DB. Depending on scale, could be a lambda doing this too.
With this basic set up, we should think about load pattern, SLA requirements, cost erc