Logging is always a compromise between storing maximum amount of context information and maintaining good performance + saving disk space. In other words some errors occur only in production environment or are very difficult to reproduce locally, but you can’t store all debug messages on production systems.
I’d like to propose an alternative solution to this dilemma: BurstLogging.
The idea is simple – log only some debug messages that were created shortly before an error message was logged.
This is achieved by storing all logs in buffer, logging only informational messages during normal operation and dumping all messages when an error occurs. Chronological order of messages is preserved (debug messages that are older than currently logged info message are dropped) and there is no huge performance penalty (messages are formatted only when they are emitted).
Current implementation is written in Python, but it shouldn’t be difficult to port it to other programming languages or implement a language agnostic solution communicating via a port or a pipe.
The project is really new and it hasn’t been used in production yet. Please tell me what do you think about this idea.