Classic tailing

In my day to day activities, there is one thing I do everytime and I think every developer does : I tail the logs. The problem with logs checking is that you are rapidly facing a huge file, and the classic tail -f is not enough. The sad part of log tailing is that you only have monochrome logs, without any colouring of any kind helping you identify critical lines against more common ones.

Colored logs

A good thing would be to have colorized logs. I used ccze a few years ago, which combined with the command you want provide a great output. But, as I am a Mac user, I can't use ccze anymore as it does not work on my platform.
I looked for another existing alternative and I found log4tailer, now Alerta which is a nice ccze equivalent.
By testing it, I discovered a few problems, but as it is an open-source software, I forked it (on github, don't have a bitbucket account and was lazy having another DVCS to learn).

Log4tailer / Alerta features

This is the list of features from the original author's page

  • Multitailing capability. It can tail multiple logs at a time
  • Colors for every level: warn, info, debug, error and fatal
  • Emphasize multiple targets (log traces) given regular expressions
  • Follow log upon truncation by default
  • User defined colors for each level
  • Silent (daemonized) mode
  • Throttling mode. Slow down the information being printed in the terminal
  • Inactivity log monitoring
  • mail notification
  • Pause Modes freezes output for a limited period of time depending on level found.
  • Analytics. Makes a report of each level found in logs when finished.
  • Different color schemes for each log
  • Tailing logs over SSH

The key features for me are the SSH mode (it requires python-paramiko module) and the inactivity log monitoring (it handles log-rotate automagically). The good part is that everything can be customized by the user in a file, provided in the command line or taken from ~/.log4tailer (you can check mine here).
This way, when I want to check the log from my second computer, I just have to launch Alerta, it autoconnects via SSH to my remote machine and tails the log I want, magic inside !

Improvements ?

Phase 1 : Bug fixing

The main thing that bothered me was that color matching was done against log4j's log levels and I am using Java Util Logging, so my first modification was made to add JUL logging levels matching.
The second problem I had to face was another incompatibility with my Mac environment (readline in python does not behave the same as on other systems). The bug made the log tailer not follow the logs (embarrassing isn't it ?).

Phase 2 : Improvements

When I fixed the things that annoyed me, I realized I could go a little further by implementing a really flexible coloring system. I inspected the code and found it was a little messy, at least for a Java developer looking at python code. I refactored it first to make it more readable to me.
I then started to add this flexible log coloring, and a little extra feature, the ability to skip line matching a certain pattern. In my case I have a useless log message showing once every page hit, but I don't want to turn off the log, I just wanted to strip out a line including a specified pattern.

New configuration options

Here is an extract from the README file describing new configuration options :

You can add specific logging levels by using the following syntax color.${level} = ${color} where ${level} is the level pattern you want to match (or any other pattern...) and ${color} is the color you want to use. Valid colors are, their background counterparts are prefixed with 'on_' : * white * cyan * magenta * blue * yellow * green * red * black * skip Levels with the skip color will not be printed out in your console (fast and easy way to disable an annoying message)

Sounds nice ? Download it !

If you're interested in this project, don't hesitate to download the package on github and install it, a simple untar and you're ready to go (assuming you've got a python set-up). If you need a full manual, read the great manual available here !

comments powered by Disqus