Searching compressed Apache logs

Posted by Tariq • Tuesday, June 23. 2009 • Category: One liners, Tidbits

From time to time things tend to go wrong and systems tend to go crazy. Sometimes these errors are more of nuisance than anything else, an intermittent annoyance you'd prefer not to investigate -- trawling through logs is a pain in the backside. Now you need to take a peak at those compressed Apache error log files, ugh! Well before you look for suitable places to hang some rope try running some expressions across a certain number of files. Normally you'd only want to look at logs for the last 7 days, so lets give that a go.

The error messages I am interested in all contain the text Healthcheck so I will use trusty grep find matching lines in the logs. I will use find to get me all the files modified in the last seven days. A program called xargs will then be used to call cat which will pipe to a compression program which finally pipes to grep. Put this all together and you get something like this:

find /var/log/apache2 -type f -name "error_*.bz2" -mtime -7 | sort | xargs cat | bunzip2 | grep "Healthcheck"

If your files are tar.gz then you'll need to replace bunzip2 with gzip -dc. So that would look like:

find /var/log/apache2 -type f -name "error_*.tar.gz" -mtime -7 | sort | xargs cat | gzip -dc | grep "Healthcheck"

Have fun!