Howto: Use logrotate to manage log files

Logrotate is a great utility to manage and adminster log files in a system. It provides you with a lot of options, namely, automatic rotation, compression and mailing of log files. Each log file can be rotated hourly, daily, weekly or monthly. In this article, I will show you how to manage your log files using logrotate.

Any type of logs can be managed using logrotate. It runs as a daily cron job. These days most of the operating systems come with logrotate pre-installed so you probably won't have to deal with the installation. Lets start with looking at the files and directories used by logrotate.

Files and Directories used by Logrotate

The 2 important files in case of logrotate are

/etc/logrotate.conf

/var/lib/logrotate.status    or    /var/lib/logrotate/status

The first file is where the whole configuration and default options of logrotate is stored. The second file contains the information about the last rotation times of different log files.

The directory thats usually used by logrotate is

/etc/logrotate.d

The options specified in /etc/logrotate.conf are used as default for rotating and managing log files. But if we want to specify separate settings for a set of log files then we can add their config files in this directory /etc/logrotate.d. Although we can even use another directory to save config files by changing an option in /etc/logrotate.conf.

Now, enough of the theory lets see how these config files really look and what all the different options in it mean.

Logrotate Options

Let me first show you some commonly used options of logrotate and then we will see how to configure various options in the config file.

rotate <num> Log files are rotated "num" times before getting deleted or mailed.
daily When used this means log files should be rotated daily
weekly Log files should be rotated weekly
monthly Log files should be rotated monthly
notifempty Don't rotate the log if its empty
compress Compress the files after rotating them
delaycompress This options means to delay the compression of a log file to the next rotation cycle. This is used in combination with compress.
missingok If the log file is missing, go on to the next one without issuing an error message.
create <mode> <owner> <group> After rotation create a new empty file with the following properties. e.g create 640 root adm. Just "create" will ensure to inherit the properties of previous log files.

Logrotate config files

As stated above, /etc/logrotate.conf is where the default options to handle logs are specified. A typical log file will look something like this.

# rotate log files weekly
weekly

# keep 4 weeks worth of backlogs
rotate 4

# create new (empty) log files after rotating old ones
create

# uncomment this if you want your log files compressed
#compress

# packages drop log rotation information into this directory
include /etc/logrotate.d

# no packages own wtmp, or btmp -- we'll rotate them here
/var/log/wtmp {
    missingok
    monthly
    create 0664 root utmp
    rotate 1
}

/var/log/btmp {
    missingok
    monthly
    create 0660 root utmp
    rotate 1
}

The options mentioned in the config file above means that the logs are going to be rotated "weekly" and only 4 copies of backlogs will be maintained. On every rotation "create" a new empty file with properties inherited from the previous logs. Compression of backlogs is disabled.

The "include" directive is used to mention the directory in which the package specific log config files will be saved.

Managing Package specific log files in /etc/logrotate.d

Now, if there is an application which generates logs how would it be able to use logrotate to control and manage its logs. This is all done using its /etc/logrotate.d directory. This is where applications drop the information required by logrotate to manage their logs. e.g consider the example of apache logs. My apache logrotate config file looks something likes this (its /etc/logrotate.d/apache2).

/var/log/apache2/*.log {
    weekly
    missingok
    rotate 5
    compress
    delaycompress
    notifempty
    create 640 root adm
    sharedscripts
    postrotate
        if [ -f "`. /etc/apache2/envvars ; echo ${APACHE_PID_FILE:-/var/run/apache2.pid}`" ]; then
            /etc/init.d/apache2 reload > /dev/null
        fi
    endscript
}

Now, cross checking  the options from the table mentioned above the following properties of the logs can be inferred.

  • Its going to be a weekly rotation
  • 5 backlogs will be maintained in compressed format. But since we are using 'delaycompress' as well, the compression of a backlog will take place in the next rotation.
  • No rotation will take place if file is empty.
  • No error in case no log is found.
  • On every rotation a new file will be created with owner root, group adm and permission 640

We haven't discussed anything about 'sharedscripts' or 'postrotate' yet. Lets begin with prerotate/endscript and postrotate/endscript.

Prerotate and Postrotate

Both of these options control the scripts that will need to be run either before or after rotating a log file. Lets take the above example.

postrotate
        if [ -f "`. /etc/apache2/envvars ; echo ${APACHE_PID_FILE:-/var/run/apache2.pid}`" ]; then
            /etc/init.d/apache2 reload > /dev/null
        fi
endscript

'endscript' is mandatory to be mentioned in both prerotate and postrotate to specify the ending of the script.

You would have also noticed the sharedscripts option in the apache2 logrotate config file. Now, if sharedscripts options is not specified, prerotate and postrotate will run the scripts on each log file which is rotated. But if you look carefully at the file logs mentioned in the apache2 logrotate conf file, its a wild card /var/log/apache2/*.log i.e. not just a single file but a bunch of files matching a pattern. These files in our case will be /var/log/apache2/access.log and error.log. So, for every rotation we will have 2 log files.

But look at the script that we want to run after rotation. It reloads the apache server and there is probably no need to run this script twice. So, by using sharedscripts we are ensuring that the script doesn't run on every rotated log file but only once.

I hope this howto will be helpful for a lot of you. If you have any doubts or want something to be corrected here, feel free to comment.

4 Comments

Anon Linuxer (not verified)
March 29th, 2010 11:16 pm
Great explanation, thanks. One quick question. I have this setup on my Linux server, but it doesn't appear to be working. The access.log and error.log file keep growing instead of rotating weekly. Is there a log file that stores logrotate error messages that I can examine?
Rod Elias (not verified)
April 30th, 2010 05:37 am
really useful howto. it'll help me for sure. thanks for writing it and posting it here. [ ]'s
Anon Linuxer (not verified)
February 10th, 2011 06:14 pm
I use logrotate to "rotate" my directory backup tar.gz files that are created daily. Very useful backup tool too.
Anon Linuxer (not verified)
February 25th, 2011 12:13 am
Can both the monthly and size options be used together for a single file? That is if the file is over the size OR the time criteria (such as week, daily, monthly) then it will rotate. I tried to setup a quick test, but it doesn't appear to be working. The syntax I used in my logrotate.d file is: /var/log/messages { rotate 3 daily size 50M sharedscripts postrotate /bin/kill -HUP `cat /var/run/rsyslogd.pid 2> /dev/null` 2> /dev/null || true endscript } Thanks

Post new comment

  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <img> <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd> <blockquote> <h1> <h2> <h3> <h4> <h5> <h6> <p> <br>
  • You may post code using <code>...</code> (generic) or <?php ... ?> (highlighted PHP) tags.
  • Image links with 'rel="lightbox"' in the <a> tag will appear in a Lightbox when clicked on.

More information about formatting options

Type the characters you see in this picture. (verify using audio)
Type the characters you see in the picture above; if you can't read them, submit the form and a new image will be generated. Not case sensitive.