Fancier logging in Python (with module and method/function names)

As Python programmers most of us know the old good logging module in Python standard library. This is somewhat really good and useful library for debugging and logging the behavior of the program, but I found it somewhat lacking.

What I needed was to be able to log what function or class method issued the log.debug() and in what line. This was needed by me for asynchronous and multithreaded networking library project I’m working on.

The solution was to write my own custom logging.Logger class implementation that overrides it’s debug method with some fancy inspect module tricks. Works like charm! Try it by yourself.

import logging
import inspect

logging.basicConfig(level = logging.DEBUG)

class MyLogger(logging.Logger):
    
    def debug(self, msg, *args, **kwargs):
        method = inspect.stack()[1][3]
        frm = inspect.stack()[1][0]
        if 'self' in frm.f_locals:
            clsname = frm.f_locals['self'].__class__.__name__
            method = clsname + '.' + method
        if not method.startswith('<'):
            method += '()'
        msg = ':'.join((method, str(frm.f_lineno), msg))
        self.__class__.__bases__[0].debug(self, msg, *args, **kwargs)

logging.setLoggerClass(MyLogger)
getLogger = logging.getLogger

Voila! Now, after instancing this new MyLogger class, you can use it just like normal Logger class, except for the bonus of having module name, function or method names and line number in your log.

Here's a simple debug output from this class:


DEBUG:network.connpool:Connection.analyze():187:Found packet: [0, 1, 0, 42, 0]
DEBUG:network.connpool:Connection.analyze():187:Found packet: [0, 1, 1, 69, 0]
DEBUG:network.server:PacketHandler.handle_close():102:Closing connection id: 139697871065392
DEBUG:core.event:emit():153:Emitted net.conn "CONN_DROP"
DEBUG:core.event:EventManager.process():194:Sending event net.conn (CONN_DROP) to ConnectionPool
DEBUG:core.event:ConnectionPool.notify():86:ConnectionPool got event of type "net.conn"
DEBUG:network.connpool:ConnectionPool.remove():294:Closing connection id: 139697871065392
DEBUG:network.connpool:ConnectionPool.parse_packets():332:Removing connection id: 139697871065392