Ticket #679 (defect)
Opened 2 years ago
Last modified 1 year ago
log file handling with standard tools
Status: closed (toolfix)
| Reported by: | jtate | Assigned to: | rdelon |
|---|---|---|---|
| Priority: | normal | Milestone: | 3.1 |
| Component: | CherryPy code | Keywords: | restsrv |
| Cc: |
logrotate, or external log rotation scripts are frequently employed to keep logs to a reasonable level. The python logging module keeps an open file descriptor for the duration of the process, causing data loss if the log file is removed.
Some sort of signal handler (apache uses HUP, but USR1 is nicer) could be employed to close all the file objects held by the various log handlers, and reopen them, though a fully self contained rotation mechanism that provides compression (based on RotatingFileHandler??) could be employed instead.
Note that all log rotation outside of the process itself has race condition potential, but catching a signal, plus using the "delaycompress" logrotate option reduces the chance of this happening to near 0.
Some sort of registration for "extra" logging Manager classes would be needed so that all loggers could be handled.
Documentation should be updated to describe methods to rotate log files.
Attachments
Change History
03/30/07 15:22:04: Modified by jtate
03/30/07 15:26:54: Modified by jtate
Note that for the above, I recommend adding a signal handler on POSIX systems. For Windows I suppose you'd be using the Windows Event Logger instead.
Some other notes: Log something after reopening, e.g. "SIGUSR1 handled". If using logrotate, use the "delaycompress" option to reduce the chance of a race condition.
03/30/07 22:38:03: Modified by jtate
- attachment testloglock.py added.
Test case proving that child threads aren't interrupted by signals.
03/31/07 16:41:51: Modified by fumanchu
[1646] adds this to trunk by subscribing to the 'graceful' channel.
06/22/07 10:18:58: Modified by fumanchu
- status changed from new to closed.
- resolution set to toolfix.
def reopenLogFiles(manager): ''' This method iterates through all of the logger objects held by the provided manager, then iterates through all of the handlers of the loggers looking for FileHandler types. A lock is acquired, the underlying stream closed, reopened, then the lock is released. This method should be called when logs are to be rotated by an external process. The simplest way to do this is via a signal handler. ''' # If there's more than one manager, have to inject one more loop for x in manager.loggerDict.itervalues(): try: for h in x.handlers: if isinstance(h, logging.FileHandler): h.acquire() h.stream.close() h.stream = open(h.baseFilename, h.mode) h.release() except AttributeError: pass # I call the above as follows # log = logging.getLogger('foo') # reopenLogFiles(log.manager)