Logrotate on GridPane Servers

9 min read

Introduction

Your GridPane servers keep 30 days of compressed access logs for every site, on both Nginx and OpenLiteSpeed (OLS), and they have built-in automatic safeguards so logs never fill your disk space. This article will walk you through how GridPane uses logrotate and how you can customize these settings if you ever need to.

You can also learn more about your different server and site logs in this article:
GridPane Logs – How to Find Them and When to Use Them

What is log rotation?

Every time someone visits a site, your web server writes a line to an access log. On a busy site, that file grows quickly, and left alone it would eventually eat up all of your disk space.

Log rotation solves this.

At regular intervals, the current log file is closed and renamed (it becomes an “archive”), a fresh empty log is started, and the archive is compressed with gzip. Old archives are then deleted once they pass a set age.

The result: you keep a useful history of your logs for troubleshooting and security investigations, without large files building up on the server. Access logs typically compress to around 1/15th of their original size.

How log rotation works on GridPane

Three layers work together:

  1. Rotation keeps 30 days of logs
  2. A per-log size cap stops any one site’s archives growing too large
  3. A disk guard steps in only if the server is close to running out of space.

All three apply to access logs only. Error logs and WAF logs keep their existing rotation and are not affected.

1. Rotation: 30 days of compressed logs

  • Nginx rotates each site’s access log once a day, or as soon as it passes 10 MB (checked hourly).
  • OpenLiteSpeed (OLS) rotates a log as soon as it reaches 10 MB.
  • On both, archives are compressed with gzip straight away and kept for 30 days.

On OLS, this includes the server-wide access log as well as each site’s log.

Because Nginx only checks sizes once an hour, a very busy site can briefly write well past 10 MB before its log rotates. Either way, the only uncompressed access log on the server is the live one.

2. Per-log cap: 3 GB or the last 24 hours

Each access log (there is one per domain) keeps up to 3 GB of compressed archives, or its last 24 hours, whichever is more. The cap is checked every hour and every night.

When a log goes over its cap, the oldest archives are deleted first. Nothing from the last 24 hours is ever deleted to meet the cap, so even a very busy site always keeps at least its last full day.

What this means in practice: a site writing up to about 62 MB of raw log per hour (roughly 1.5 GB a day) keeps the full 30 days within 3 GB. Most sites are nowhere near this, so the cap only affects very busy sites or sites under attack.

3. Disk guard: an emergency safety net

The disk guard does nothing on a healthy server. It only acts when free disk space gets low, and it works in three tiers. Each tier fires when free space drops below the smaller of a percentage of the disk and a fixed size:

Tier Fires when free space is below What it deletes
1 10% of the disk or 20 GiB Archives older than 7 days, oldest first across all sites, but only if that is enough to fix the problem; otherwise it leaves them alone
2 5% of the disk or 10 GiB Archives older than 24 hours, oldest first across all sites
3 2% of the disk or 2 GiB Last resort: recent archives too, biggest site first. Every log keeps its newest archive

Once triggered, a tier deletes archives until free space is back above its trigger, plus a small margin (2% of the disk or 4 GiB, whichever is smaller).

  • On small and mid-sized disks (up to about 160 GiB), tier 1 fires at about 90% used, tier 2 at about 95% and tier 3 at about 98%.
  • On larger disks the fixed sizes take over, so the guard waits longer.
  • On a 500 GiB disk, the tiers fire at 20 GiB, 10 GiB and 2 GiB free (about 96%, 98% and 99.6% used).

On Nginx, if a single live log has grown huge (over 1 GiB, or 1% of the disk), tiers 2 and 3 can also rotate that one log on its own so it can be compressed. Its archive ends in -gp, and Nginx is told to reopen its logs without a reload.

Important

The disk guard never deletes a live log, an error or WAF log, or anything other than access-log archives. It is a safety net, not a substitute for disk space: if a backup or import fills a disk quickly, the guard can't prevent that.

When it all runs

  • Hourly: the per-log cap, then the disk guard.
  • Nightly: the 30-day clean-up of old archives, then the cap, then the disk guard.
  • Every 2 minutes, but only once the disk is about 84% full: the disk guard checks free space and acts if needed.

All of this runs at the lowest CPU and disk priority, so it has no measurable effect on site performance.

Where your access logs are stored

In the paths below, replace <site> with the site’s domain, for example example.com.

Nginx

The live log is:

/var/log/nginx/<site>.access.log

Archives are named:

/var/log/nginx/<site>.access.log-YYYYMMDD-<epoch>.gz

OpenLiteSpeed (OLS)

The live log is:

/var/www/<site>/logs/<site>.access.log

Archives are named:

/var/www/<site>/logs/<site>.access.log.YYYY_MM_DD[.NN].gz

The server-wide access log is:

/usr/local/lsws/logs/access.log

Changing the defaults with GP-CLI

The size cap, the 24-hour protected window and the disk guard triggers can all be changed with gp conf commands, run as root over SSH. Changes take effect on the next run, which is at most an hour later. The 30-day retention itself is fixed.

Getting Started

If this is your first time connecting to you server over SSH, please see the following guides to get started:

All settings at a glance

Setting Default Accepted values
access-log-cap-default-mb 3072 Whole number of MB
access-log-cap-mb
(per site, or server ceiling)
Not set Whole number of MB
access-log-floor-hours 24 1–168
access-log-guard-t1-free-pct / -t1-free-gb 10 / 20 % 1–15, any whole GiB
access-log-guard-t2-free-pct / -t2-free-gb 5 / 10 % 1–15, any whole GiB
access-log-guard-t3-free-pct / -t3-free-gb 2 / 2 % 1–15, any whole GiB

An invalid value (not a whole number, or out of range) is ignored and the default is used instead.

Change the cap for every log on the server

The default is 3072 MB (3 GB). To set it to 1 GB:

gp conf write access-log-cap-default-mb 1024

Lower the cap for one site

A site can always be set lower than the server default with this single command.

gp conf write access-log-cap-mb 500 -site.env example.com

Raise the cap for one site above the server default

This takes two commands. First set a server ceiling (the highest value any site is allowed to use), then set the site’s own value:

gp conf write access-log-cap-mb 6000
gp conf write access-log-cap-mb 6000 -site.env example.com

The ceiling on its own raises nothing. It only allows a site value up to that size. If the site’s value is higher than the ceiling, the site falls back to the server default.

For example, with a server default of 1 GB, a site with no value of its own gets 1 GB, and a site set to 500 gets 500 MB.

A site set to 2000 gets 2 GB only if the server ceiling is 2000 or more. Without that ceiling, it stays at the 1 GB default.

Change the protected window (default 24 hours)

Archives inside this window are never deleted to meet the cap. Any whole number from 1 to 168 hours (one week) is accepted:

gp conf write access-log-floor-hours 48

Change the disk guard triggers

Each tier has a percentage (1–15) and a size in GiB. For example, to make tier 1 fire at 12% or 30 GiB free:

gp conf write access-log-guard-t1-free-pct 12
gp conf write access-log-guard-t1-free-gb 30

The same pattern works for tiers 2 and 3 (access-log-guard-t2-free-pct, access-log-guard-t3-free-gb and so on). The tiers must stay in order, with tier 3 lower than tier 2 and tier 2 lower than tier 1, for both the percentages and the sizes. If they don’t, all six values go back to their defaults.

Read or reset a setting

gp conf read access-log-cap-default-mb
gp conf delete access-log-cap-default-mb
gp conf delete access-log-cap-mb -site.env example.com

Deleting a setting returns it to the default.

Checking Your Current Settings, Log Sizes, & History

Check your current settings and log sizes

This command shows everything at once: disk usage, every setting and where it came from, each log’s size and cap, and whether the disk guard is active. It is read-only and changes nothing.

/usr/local/bin/monit-scripts/access_log_caps --status

See what the system has done

Every deletion, rotation and settings problem is recorded in /var/log/gridpane.log, tagged access-log-caps:
grep access-log-caps /var/log/gridpane.log
A line containing cannot reach target means the disk is filling up, but access logs are too small to be the cause. Look at backups, uploads or databases instead.

Questions and troubleshooting

My logs from three weeks ago are gone. Why?

Either the log reached its cap, or the disk guard ran. Run the following on your server:

 grep 'guard tier' /var/log/gridpane.log

If there are guard lines around that time, the disk was nearly full and the real space problem is elsewhere. If not, the log hit its cap, and you can raise it for that site using the commands above.

My disk is full and the guard did nothing

Either your access logs are too small to make a difference, or free space hasn’t dropped below the tier 1 trigger yet. On a 500 GiB disk, that means less than 20 GiB free.

There’s a large uncompressed file ending in -gp

That’s an emergency rotation of a very large Nginx log. It will be compressed automatically on a later run.

A site’s logs are crowding out my backups

Reduce the log volume at the source (WAF rules or rate limiting for bot traffic), or lower that site’s cap.