Skip to main content

python and the logger

I've been working with logging in Python for the last few weeks. For some reason I ended up having to log to a MySQL (MariaDB?) server and I hadn't done that before. Turns out that a lot of people want to log MySQL queries to the same logging server as Python, but not a lot of people want to log to a MySQL server. The Python logger is actually quite a nicely documented piece of software if your needs are basic, but want something more and you might end up between a suggestion and a hard place...

Next to logging to MySQL I also wanted to configure the loger using code, not an ini file or a YAML file. Enter DictConfig. The example in the documentation is massive, does much more than anyone needs and is too complicated for when you are starting out with logging. So here's a basic example:

logging_config = {
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {
        'standard': {
            'format': '%(asctime)s [%(levelname)s] %(name)s: %(message)s'
        },
    },
    'handlers': {
        'default_handler': {
            'level': 'DEBUG',
            'formatter': 'standard',
            'class': 'logging.StreamHandler',
            'stream': 'ext://sys.stdout',
        },
    },
    'loggers': {
        '': {
            'level': 'NOTSET',
            'handlers' : ['default_handler']
        },
    },
}

import logging.config
logging.config.dictConfig(logging_config)
logger = logging.getLogger(os.path.basename(__file__))
# Now you can use logger to log
logger.error('Panic')

This should be sufficient for a simple logger to stdout (the CLI, console, DOS box whatever you call it). See the documentation if you want to change something or make it more complicated.

When you want to log to a MySQL database, you need to write your own custom handler class, inheriting from logging.Handler. Or you could google for an existing MySQLHandler, like this one I found on Github.

I did bring that handler up to Python 3 specs, and followed whatever Pycharm wanted me to fix on this code.

The one "big" thing I did, was to change the constructor of the handler class to the following:

def __init__(self, host: str, port: int, db_user: str, db_password: str, db_name: str):
    pass

I'll leave the implementation of this function up to you, but the signature is really important when you want to use this custom handler in combination with DictConfig.

logging_config = {
    'version': 1,
    'disable_existing_loggers': False,
    'handlers': {
        'default_handler': {
            'class': 'my_custom_handler.MyCustomHandler',
            'level': 'DEBUG',
            'formatter': 'standard',
            'host': "localhost",
            'port': 3306,
            'db_user': "user",
            'db_password': "password",
            'db_name': "Log",
        }
    },
    'loggers': {
        '': {
            'level': 'NOTSET',
            'handlers' : ['default_handler']
        },
    },
}

import logging.config
logging.config.dictConfig(logging_config)
logger = logging.getLogger(os.path.basename(__file__))
# Now you can use logger to log
logger.error('Panic')

The custom handler in this example has the class name MyCustomhandler and can be found in the my_custom_handler module. As you can see the parameters for a custom handler in DictConfig reflect the parameters of the constructor of the custom handler. Note also that a database handler doesn't really need a formatter when your datastructure looks like the format of log messages.

Vakantie 2019

In 2019, I spent a lot of holiday time on recording various events with C3VOC, so I had very few days left to go on a cycling holiday. At the end of September we tried to ride as close to the Dutch-German border as possible using (mostly) the "knooppunten"-network. We started in Bad Nieuweschans and got as far as Sittard in just over a week. We had a surprising amount of nice weather, but one day was very, very wet and cold.

Vakantie 2019

Vakantie 2018

For some reason I never added the summer holiday 2018 photograps to Flickr, nor did I write a quick post here. So, here it is. Enjoy our route from Zaltbommel to Berlin via Cologne, Paderborn, Magdeburg and Brandenburg.

Vakantie 2018

Wijk aan Zee

Last Saturday I spent the afternoon at the beach, mostly watching loads of people paragliding along the dunes. The Flickr album has the photos. The album has had two more batches of photographs added to it, so if you've watched them ealier, check again for more colourful photographs.

Wijk aan Zee 2020-05-17

Finally an update

Hi there! It's been a while ;-)

For various reasons I haven't been writing here as much as the (last/first) few years. But this week I felt the need to write a blog post and I noticed that the build environment for this site had been completely destroyed. Fortunately I had the sources in git, but the rest was broken beyond all recoginition.

So after quite a few hours, this site is back with a new theme, built with the latest Nikola and has SSL support!

The blog post that I talked about will probably appear here in the coming week or so.

Removing stale entries from "Open With" menu

For the last few years I had (for some unknown reason) three entries for VLC in my "Open with" menu in the Finder. There was no way to remove them from VLC itself and I couldn't find a way to do it myself. Until today. The following steps are also valid for other application that have stale "Open With" entries in the Fidner on a Mac running MacOS (nee Mac OS X)

The first step is to close the offending application, just to make sure nothing is still open that could block our attempts). Then open a CLI terminal (Terminal, iTerm, etc) and go to "~/Library/Application Support/". You will find loads of qpplication related directories (maps, folders) here. Remove all the directories that have something to do with your application ("rm -rf VLC" etc).

The last step is to recreate the "Open With" menu and restart the Finder. This might take a few seconds ("/System/Library/Frameworks/CoreServices.framework/Frameworks/LaunchServices.framework/Support/lsregister -kill -r -domain local -domain system -domain user" and "killall Finder")

SQLAlchemy and UML

I've started a new "project". I had a need to convert some UML diagrams to Python code and foudn that there was no support for the OpenModel XML fileformat, so I created it. See Github for more information. It is a prototype for now, but with a little help (from you?) it could be made much more useful, even support for other UML XML fileformats?

tiler.py

So I had to create a 1024*1024 background image consisting of 2 different 64x64 tiles. The script had to choose randomly between both tiles, with a preference for one over the other. Below is the resulting code. Enjoy!