Contents

How to Enable Logrotate and crontab on PhotonOS

Prevent Traefik logs from filling partition

I have moved away from using CentOS and/or Ubuntu for running Docker containers. My OS of choice for the last few years has been PhotonOS. I generally have my containers config and log files in the Host OS, but have not been dealing with the growing size in an automated way. That changes now! I don’t want my partitions to fill up with logs!

Install the necessary packages

PhotonOS does not install or configure logrotate and a crontab service by default so we need to do that now. For the crond service, PhotonOS uses “cronie”. If you’re unfamiliar with cronie, here is the package description:

cronie description:
Cronie contains the standard UNIX daemon crond that runs specified programs at scheduled times and related tools. It is based on the original cron and has security and configuration enhancements like the ability to use pam and SELinux.
1
sudo tdnf install -y logrotate cronie

After installation, a default config file will be created: /etc/logrotate.conf with the following contents: (valid as of PhotonOS 3.0)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
# see "man logrotate" for details
# rotate log files weekly
weekly

# keep 4 weeks worth of backlogs
rotate 4

# create new (empty) log files after rotating old ones
create

# use date as a suffix of the rotated file
dateext

# uncomment this if you want your log files compressed
#compress

# RPM packages drop log rotation information into this directory
include /etc/logrotate.d

# system-specific logs may be also be configured here.

The install also created a crontab file in /etc/cron.daily named logrotate

Since my PhotonOS systems are genarally used for running Traefik under Docker with config and log files residing in a subdirectory of my docker-compose.yml, I want to configure rotation of the log files associated with it, namely access.log and traefik.log.

Configure log rotation

Create a file named /etc/logrotate.d/traefik2.conf

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
/path/to/traefik/logs/* {
  weekly
  rotate 3
  size 50M
  compress
  delaycompress
  missingok
  notifempty
  dateext
  dateformat .%Y-%m-%d
  create 0644 root root
  postrotate
    docker kill --signal="USR1" $(docker ps | grep '\btraefik\b' | awk '{print $1}')
  endscript
}

Reference/Credit for the above config file - particularly the postrate script: Stackoverflow.com - How to enable logrotation for traefik?

Ref: https://doc.traefik.io/traefik/observability/logs/#log-rotation
Traefik will close and reopen its log files, assuming they’re configured, on receipt of a USR1 signal. This allows the logs to be rotated and processed by an external program, such as logrotate.
  • weekly means that the tool will attempt to rotate the logs on a weekly basis. Other possible values are daily and monthly.
  • rotate 3 indicates that only 3 rotated logs should be kept. Thus, the oldest file will be removed on the fourth subsequent run.
  • size=50M sets the minimum size for the rotation to take place to 50M. In other words, each log will not be rotated until it reaches 50MB.
  • compress and delaycompress are used to tell that all rotated logs, with the exception of the most recent one, should be compressed.
  • create sets the chmod and user group ownership of the rotated files
  • dateext and dateformat specify the suffix of the rotated files
  • postrotate / endscript - IMPORTANT: this kill signal allows traefik to reload the log file. Failing to include this will result in the newly rotated log file remaing zero bytes.

Now check the config file by performing a dry-run:

1
sudo logrotate -d /etc/logrotate.d/traefik2.conf

Results:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
WARNING: logrotate in debug mode does nothing except printing debug messages!  Consider using verbose mode (-v) instead if this is not what you want.

reading config file /etc/logrotate.d/traefik2.conf
Reading state from file: /var/lib/logrotate.status
Allocating hash table for state file, size 64 entries

Handling 1 logs

rotating pattern: /path/to/traefik/logs/*  52428800 bytes (3 rotations)
empty log files are rotated, old logs are removed
considering log /path/to/traefik/logs/access.log
Creating new state
  Now: 2021-04-07 07:39
  Last rotated at 2021-04-07 07:00
  log needs rotating
considering log /path/to/traefik/logs/traefik.log
Creating new state
  Now: 2021-04-07 07:39
  Last rotated at 2021-04-07 07:00
  log does not need rotating (log size is below the 'size' threshold)
rotating log /path/to/traefik/logs/access.log, log->rotateCount is 3
dateext suffix '-20210407'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
previous log /path/to/traefik/logs/access.log.1 does not exist
renaming /path/to/traefik/logs/access.log.3.gz to /path/to/traefik/logs/access.log.4.gz (rotatecount 3, logstart 1, i 3), 
renaming /path/to/traefik/logs/access.log.2.gz to /path/to/traefik/logs/access.log.3.gz (rotatecount 3, logstart 1, i 2), 
renaming /path/to/traefik/logs/access.log.1.gz to /path/to/traefik/logs/access.log.2.gz (rotatecount 3, logstart 1, i 1), 
renaming /path/to/traefik/logs/access.log.0.gz to /path/to/traefik/logs/access.log.1.gz (rotatecount 3, logstart 1, i 0), 
log /path/to/traefik/logs/access.log.4.gz doesn't exist -- won't try to dispose of it
renaming /path/to/traefik/logs/access.log to /path/to/traefik/logs/access.log.1

Enable crond

Now, in order for this rotation to take place and that /etc/cron.daily/logrotate file to be run, we need to enable our crontab tool. When cronie was installed, it created the necessary files for enabling a crond service, but did not actually enable or start the service so do that now:

1
2
sudo systemctl enable --now crond
sudo systemctl status crond

Output:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
● crond.service - Command Scheduler
   Loaded: loaded (/lib/systemd/system/crond.service; enabled; vendor preset: enabled)
   Active: active (running) since Wed 2021-04-07 08:03:27 PDT; 7s ago
 Main PID: 2844 (crond)
    Tasks: 1 (limit: 4731)
   Memory: 408.0K
   CGroup: /system.slice/crond.service
           └─2844 /usr/sbin/crond -n

Apr 07 08:03:27 lab-proxy-0.lab.livefire.dev systemd[1]: Started Command Scheduler.
Apr 07 08:03:27 lab-proxy-0.lab.livefire.dev crond[2844]: (CRON) STARTUP (1.5.5)
Apr 07 08:03:27 lab-proxy-0.lab.livefire.dev crond[2844]: (CRON) INFO (Syslog will be used instead of sendmail.)
Apr 07 08:03:27 lab-proxy-0.lab.livefire.dev crond[2844]: (CRON) INFO (RANDOM_DELAY will be scaled with factor 82% if used.)

There we go! Now my Traefik logs will be rotated regularly!

After a few days, check back on your logs folder (ls -ahl) and it should have a similar appearance to the following:

1
2
3
4
5
-rw-r--r-- 1 root     root   19M Apr 11 09:16 access.log
-rw-r----- 1 root     root  7.6M Apr  9 03:27 access.log.2021-04-09.gz
-rw-r--r-- 1 root     root  5.9M Apr 10 03:19 access.log.2021-04-10.gz
-rw-r--r-- 1 root     root  100M Apr 11 03:18 access.log.2021-04-11
-rw-r--r-- 1 root     root  105K Apr 11 09:09 traefik.log