For those of you who read my site much, you would have noticed that I recently added/created a couple of logging scripts; they do their thing, and most specifically, log the results and info to a log file. Fortunately, they are only text-based logs, so should not get very large very fast. But after a week or two, or even a month or two, those files will quite large. And worse yet, they will be very difficult to manually go through, since they will just be so big. So we need to rotate the log files every so often to keep things a little bit manageable. Sure, I could go ahead and add some kind of rotation to the scripts, but why go through all that hassle when there's a utility to do this for us: logrotate !
I really had no desire to manually create my own log rotation methods and include them in my scripts. Not when there's already something to do so for me, and can easily be configured to include my own log files. From logrotate's GitHub page:
"Logrotate allows for the automatic rotation compression, removal and mailing of log files. Logrotate can be set to handle a log file daily, weekly, monthly or when the log file gets to a certain size."
Fortunately for us all, logrotate is included in nearly all modern L:inux distros. There are SOME that will *NOT* have this utility built-in. For example, Checkpoint's SecurePlatform, though based off RedHat Enterprise Linux, does not use logrotate. To check if it installed, you can run the following command:
which logrotate
If you get output looking like this:
/usr/sbin/logrotate
Then it is installed. If you get an error or a message "no logrotate in ...", then it is not installed.
On Debian-based systems, you can install it with the command:
apt-get install logrotate
On RedHat-based systems, you can use:
yum install logrotate
CONFIGURATION
For the most part, we don't need to mess around too much with the logrotate configuration itself. Really, all we need to do is add a file to a folder, put in some specific configurations into the file, and let logrotate take care of things. First, let us create the file we will edit:
touch /etc/logrotate.d/conn
Now edit the file:
nano /etc/logrotate.d/conn
Let's take a look at my conn file for my script logs:
/var/log/conn/conn.log {
daily
compress
missingok
delaycompress
notifempty
dateext
copytruncate
}
/var/log/conn/https.log {
daily
compress
missingok
delaycompress
notifempty
dateext
copytruncate
}
Let's break this down a bit...
/var/log/conn/conn.log {
This is the log file you want to rotate. Everything that comes after this are the directives for the log rotation.
daily
Rotate the log daily.
compress
Compress the log files after they are rotated.
missingok
Do not error if the log file is actually missing.
delaycompress
Do not compress the recently rotated log file, only compress those that come afterwards. This wa, you will have file.log, file.log.1, file.log.2.gz, etc...
notifempty
Do not rotate if the current log file is empty.
dateext
Add the date to the extension. Example: output.log-20150907.gz (YYYYMMDD)
copytruncate
Truncate the original log file in place after creating a copy, instead of moving the old log file and optionally creating a new one.
These are just the directives that I personally use for these log files. There are many more directives you can add here. For a mre complete listing, you can either read the man page:
man logrotate
Or you can check out this website: http://linux.die.net/man/8/logrotate
This setup will keep the logs indefinitely, which is what I want. You can limit the number of old logs in a couple of way:
rotate <count>
Log files are rotated <count> times before being removed. If count is 0, old versions are removed rather than rotated.
maxage <count>
Remove rotated logs older than <count> days. The age is only checked if the logfile is to be rotated.
Last up, is the timing of the logrotate. Now, when I initially set the lgos to rotate daily, they were, but the timestamps were starting at like 6:53am, a weird time, and nowhere near midnight. It turns out that the log rotation timing is taken care of with a cron job. To edit this, you just need to edit the crontab file. Normally, you would want to use the command crontab -e to accomplish this edit. But this will only show you the cron jobs for that user. So we will need to actually edit the crontab file itself to accomplish this.
First, let us make sure to make a backup of the crontab, JUST IN CASE:
cp /etc/crontab /etc/crontab.ORIGINAL
Now edit it:
nano /etc/crontab
Look for the line that contains --report /etc/cron.daily, it should look somewhat similar to this:
53 6 * * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )
The first 2 colums are what we will be editing... The line starts with minutes and hours. Above, the script is set to run at 06:53 (note the 24hour format). To change this to midnight, just change the first 2 numbers to 0 (0000hours):
0 0 * * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )
To set it to 12pm noon, change it to:
0 12 * * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )
And that is it! You have now set the logrotate to rotate at your time (mine is midnight), and rotate your logs through for you. All you need to do now is let the logs run, and check up on the logs as needed.
Please feel free to post your comments, queries, or concerns below.
