Using logrotate to rotate log files

By on January 7, 2013
tux

 

Logrotate is a great utility to manage and adminster log files in a system. It provides you with a lot of options, namely, automatic rotation, compression and mailing of log files. Each log file can be rotated hourly, daily, weekly or monthly. In this article, I will show you how to manage your log files using logrotate.

Any type of logs can be managed using logrotate. It runs as a daily cron job. These days most of the operating systems come with logrotate pre-installed so you probably won’t have to deal with the installation. Lets start with looking at the files and directories used by logrotate.

Files and Directories used by Logrotate

The 2 important files in case of logrotate are

/etc/logrotate.conf

/var/lib/logrotate.status    or    /var/lib/logrotate/status

The first file is where the whole configuration and default options of logrotate is stored. The second file contains the information about the last rotation times of different log files.

The directory thats usually used by logrotate is

/etc/logrotate.d

The options specified in /etc/logrotate.conf are used as default for rotating and managing log files. But if we want to specify separate settings for a set of log files then we can add their config files in this directory /etc/logrotate.d. Although we can even use another directory to save config files by changing an option in /etc/logrotate.conf.

Now, enough of the theory lets see how these config files really look and what all the different options in it mean.

Logrotate Options

Let me first show you some commonly used options of logrotate and then we will see how to configure various options in the config file.

rotate <num> Log files are rotated “num” times before getting deleted or mailed.
daily When used this means log files should be rotated daily
weekly Log files should be rotated weekly
monthly Log files should be rotated monthly
notifempty Don’t rotate the log if its empty
compress Compress the files after rotating them
delaycompress This options means to delay the compression of a log file to the next rotation cycle. This is used in combination with compress.
missingok If the log file is missing, go on to the next one without issuing an error message.
create <mode> <owner> <group> After rotation create a new empty file with the following properties. e.g create 640 root adm. Just “create” will ensure to inherit the properties of previous log files.

Logrotate config files

As stated above, /etc/logrotate.conf is where the default options to handle logs are specified. A typical log file will look something like this.

# rotate log files weekly
weekly

# keep 4 weeks worth of backlogs
rotate 4

# create new (empty) log files after rotating old ones
create

# uncomment this if you want your log files compressed
#compress

# packages drop log rotation information into this directory
include /etc/logrotate.d

# no packages own wtmp, or btmp — we’ll rotate them here
/var/log/wtmp {
missingok
monthly
create 0664 root utmp
rotate 1
}

/var/log/btmp {
missingok
monthly
create 0660 root utmp
rotate 1
}

The options mentioned in the config file above means that the logs are going to be rotated “weekly” and only 4 copies of backlogs will be maintained. On every rotation “create” a new empty file with properties inherited from the previous logs. Compression of backlogs is disabled.

The “include” directive is used to mention the directory in which the package specific log config files will be saved.

Managing Package specific log files in /etc/logrotate.d

Now, if there is an application which generates logs how would it be able to use logrotate to control and manage its logs. This is all done using its /etc/logrotate.d directory. This is where applications drop the information required by logrotate to manage their logs. e.g consider the example of apache logs. My apache logrotate config file looks something likes this (its /etc/logrotate.d/apache2).

/var/log/apache2/*.log {
weekly
missingok
rotate 5
compress
delaycompress
notifempty
create 640 root adm
sharedscripts
postrotate
if [ -f "`. /etc/apache2/envvars ; echo ${APACHE_PID_FILE:-/var/run/apache2.pid}`" ]; then
/etc/init.d/apache2 reload > /dev/null
fi
endscript
}

Now, cross checking  the options from the table mentioned above the following properties of the logs can be inferred.

  • Its going to be a weekly rotation
  • 5 backlogs will be maintained in compressed format. But since we are using ‘delaycompress’ as well, the compression of a backlog will take place in the next rotation.
  • No rotation will take place if file is empty.
  • No error in case no log is found.
  • On every rotation a new file will be created with owner root, group adm and permission 640

We haven’t discussed anything about ‘sharedscripts’ or ‘postrotate’ yet. Lets begin with prerotate/endscript and postrotate/endscript.

Prerotate and Postrotate

Both of these options control the scripts that will need to be run either before or after rotating a log file. Lets take the above example.

postrotate
if [ -f "`. /etc/apache2/envvars ; echo ${APACHE_PID_FILE:-/var/run/apache2.pid}`" ]; then
/etc/init.d/apache2 reload > /dev/null
fi
endscript

‘endscript’ is mandatory to be mentioned in both prerotate and postrotate to specify the ending of the script.

You would have also noticed the sharedscripts option in the apache2 logrotate config file. Now, ifsharedscripts options is not specified, prerotate and postrotate will run the scripts on each log file which is rotated. But if you look carefully at the file logs mentioned in the apache2 logrotate conf file, its a wild card /var/log/apache2/*.log i.e. not just a single file but a bunch of files matching a pattern. These files in our case will be /var/log/apache2/access.log and error.log. So, for every rotation we will have 2 log files.

But look at the script that we want to run after rotation. It reloads the apache server and there is probably no need to run this script twice. So, by using sharedscripts we are ensuring that the script doesn’t run on every rotated log file but only once.

Testing the log rotate

Logrotate.conf is executed as a cron job in linux, So u dont need to update any other file after changing /etc/logrotate.d You can run it forcefully by using the command

logrotate -f /etc/logrotate.conf

or else if you dont want to run it and just want to check for the errors or the possible outcome of your changes, you can run it with –d option

logrotate -d /etc/logrotate.conf

I hope this howto will be helpful for a lot of you. If you have any doubts or want something to be corrected here, feel free to comment.

About jeevan

Geek under construction, Tech Speaker, Tech Blogger @Geekospace.com, foodie, Cloud Evangelist, Moviez freak passionate learner, startup lover, business and tech savya, social media addict, biker, musician, extrovert, realistic, seminal, approachable.


Leave a Reply

Your email address will not be published. Required fields are marked *

*


*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>