Troubleshooting High Disk Space Usage and Locating Large Files

4 min read

Introduction

Occasionally, we recieve tickets about puzzling disk space usage. There’s usually a pretty simple explanation for this issue, and it’s also straightforward to get to the bottom of when you know where and how to look. This article will help you get to the bottom of these issues.

Table of Contents

  1. Why Disk Space Warnings are Important
  2. Monit Stats
  3. Where and How to Check for Disk Space
  4. Freeing Up Disk Space

Why Disk Space Warnings are Important

Allowing your server’s disk space to fill up can have serious consequences. If you haven’t already, we recommend setting up Slack notifications for your account:

How to set up Slack Notifications

At best, MySQL won’t have enough memory to function and will go offline, taking all of your websites offline. At worst, you’ll suffer irreversible database corruption. In this case, you’ll need to restore a server snapshot or restore your websites from backups to another server.

These types of issues are beyond our scope of support. Make sure you’re monitoring your server notifications and either increase your server size accordingly or free up disk space by deleting unnecessary files or moving large websites to a different server.

Monit Stats

The first place to check is your server’s Monit stats. Here you can see your disk space size, MySQL size, backup size, and log size: 

In the beginning…

After the approximately 10GB of space taken up by the Ubuntu OS and our systems, the rest is your website files, their databases, website/server logs, and your local website backups.

Where and How to Check for Disk Space

Checking Monit like in the above screenshot above may make it obvious pretty quickly where you need to first begin checking. The following requires connecting to your server – please see the following guides to get started:

If the server has been live for an extended period then other logs not just inside /var/log can build up as well – especially if you have numerous sites on the server.

The three main directories are:

  • /var/
  • /opt/
  • /etc/

You’ll specifically want to check:

  1. Your backups
  2. Your websites /wp-content directory
  3. Your database sizes
  4. Server logs
  5. Site logs

You can narrow down large directories and files with the following two commands.

The first lists directory sizes (largest size at the bottom, along with the directory total size):

du -h --max-depth=1 | sort -h

The second lists files inside a directory in order of size (again with the largest at the bottom):

ls -laShr

Most Common Causes

  1. One or more of your websites (files or database) is much larger than expected.
  2. Lots of plugin-created backups are consuming a high amount of disk space.
  3. An old staging site is much larger than expected.

Check Your Website File Sizes

You can check your website sizes by first navigating to their directory:

cd /var/www

And running:

du -h --max-depth=1 | sort -h

Check Your Website Database Sizes

You can check individual database sizes with the following:

cd /var/lib/mysql
du -h --max-depth=1 | sort -h

Freeing Up Disk Space

Important: If you’re unsure if a file is important or not, don’t delete it. The best thing you can do is post in the community forum where our team or the wider community can provide an answer for the file in question, and everyone in the wider community can benefit.

Deleting Files

If you’ve identified files than are no longer needed, you can remove them with the rm command. For example, in the directory where the file exists, you can use the following:

rm bigfilename.here

Deleting Directories

Please exercise caution when deleting entire directories. This kind of action is irreversible and will require you to make use of a backup to rectify the issue. You can remove directories with the rm command with the -r flag. This will remove non-empty directories and all of the files inside them recursively. For example:

rm -r directorynamehere

Database Optimization

If any of your databases are far larger than you expected them to be, you may be able to reduce their size significantly by optimizing them.

Important

Be sure to backup your database (or ensure you have current website backups) before making significant database changes, such as running an optimization. Risks of things going wrong are extremely small, but it's important to ensure you have a backup just in case.

You can do this with a plugin like WP Sweep, the LS Cache plugin, Perfmatters, or other actively maintained and highly reviewed optimization plugins.

You can also use WP-CLI. On GridPane, you can use GP WP-CLI as follows (replacing site.url with your websites URL):

gp wp site.url db optimize