How to Restore a Moodle Site from Hosting Backups

Introduction

Our data center, the hosting provider, creates daily backups of the entire file system of a VPS (Virtual Private Server), for a window of fourteen days. This also includes the binary database files. But these can be hard to restore on another system. Therefore, we install a Bash script for each virtual host (i.e. Moodle website) that creates a database dump and compresses it in one go.

The advantage of this is that a database dump, together with the moodledata directory and Moodle’ s codebase, allows you to restore the complete Moodle installation on any system which meets the minimum requirements for the Moodle version, not just the original server.

1. Create A Restore Directory

hosting provider has the option to restore a specific directory of the file system in the backup (up to 13 days) to a directory of your choice. For instance, you can choose to restore /home/mensen/ to /restore. Either in full, or as a tar.gz file.

This means that performing a restore will take up additional disk space. But it’s also extremely easy to execute.

To create a restore directory, login to the VPS using ssh and issue a command like:

mkdir /restore

2. Select The Backup Files

First login to https://kis.hosting provider.de/.

Navigate to The Correct VPS

Now click on Product Admin:

Choose Virtual Server:

Now pick the system on which you want to perform the restore, by clicking on the Konfigurieren button (the KIS website is originally in German, which pops up in odd places in the English version).

Finding The Right VPS

If you don’t know which system (VPS) has the Moodle installation you’re looking for, use the command nslookup with the domain name as its argument. For instance, if I want to know the IP address of lms.example.com:

the lead admin@example-host:~$ nslookup lms.example.com
Server: 198.51.100.10
Address: 198.51.100.10#53
Non-authoritative answer:

Name: lms.example.com

Address: 198.51.100.11

Select The Source for The Backup

Next to Datum des Backups (which means backup date), click on the select box and pick the right date. Please keep in mind that the backup is usually created around midnight CET.

Next, click on the Browse button next to Quell-Verzeichnis (source directory).

In the following screen, you’ll get to see the file system as it was at the time the backup was made. On our VPS systems, we always store the files for the Moodle installations in their own home directories. So click on home first:

Find the home directory of the Moodle installation you want to restore, which typically has a part of its domain name as the directory name.

Now click on the auswählen (choose) link next to the home directory’s name, mensen in our example:

3. Perform The Restore

Once you have selected the source of the backup, you’ll be presented with the following screen, on the KIS website:

Here’s a translation and clarification for the fields in the form:

• Datum des Backups: backup date

• Quell-Verzeichnis: source directory – this is the directory you have created on the VPS in the first step.

• Ziel-Verzeichnis (muss existieren): target directory (must exist)

• Benachrichtigungen an E-mail: e-mail notification – the system will send you an email once the restore is done.

• Restore-Type: restore type – do you want to restore the target directory in its entirety (Direkt) or as a compressed file?

If you have filled out the form click Restore starten (Start restore).

The restore process will take some time (since it’s presumably copying the files from a slow backup medium).

Once the restore is complete, don’t forget to inform the customer of the results.

TO DO

Create more generic SOP about backup to external file storage system (such as lms.example.com/cloud-storage/?nedbksnst000adm0000kusebe). hosting provider does not do daily backups anymore.

Considerations

Don’t look at example-site’s backup solution – it’s too complicated.

Instead, rely on a rotation scheme (e.g. as provided by /etc/logrotate.d) to remove old backups out of the window. Although this probably can’t be done on the storage server if it’s really only a file server you can only access through sftp. In that case we need something that cycles through the numbers 1 – 14. E.g.:

$backupday = $weekday;
if ($weeknumber % 2 == 0) {
// even week number
$backupday = $backupday + 7;
}

(The script should probably also remove yesterday’s backup before creating a new one to be copied to the remote system).

Also consider encrypting the backup file on the fly:

$ tar -zcpf - folder | gpg -c -passphrase thesecrectpassphrase >
folder.tar.gz.gpg

And don’t forget to skip .git from the actual backup (when including public_html)

There are also existing scripts:

https://wiki.archlinux.org/title/duplicity

But these tend to be too complicated for our use case.

Incremental Backups

20230601

Large moodledata directories typically contain files that are added once but never changed afterward, such as video files. It doesn’t make much sense to store a complete copy of moodledata every day.

Instead, we need to create a full backup on day 1 (level 0, in tar terminology) and incrementally add to that until day 14 (or whatever the backup window is).

Notes from https://www.gnu.org/software/tar/manual/html_section/Incremental-Dumps.html and lms.example.com/incremental-backups-with-tar/:

• You can force ‘level 0’ backups either by removing the snapshot file before running tar, or by supplying the ‘–level=0’ option

• It is common practice to name meta files with the extension .snar, think of this as an amalgamation of snapshot and tar.

It is also possible to create differential backups: these take the full backup as the base and then subsequently store all the mutations since the full backup was made. The difference is this: incremental backups store only the mutations since the last backup, whereas differential backups may store the same file multiple times (a file that was newly added since the full backup).

This is explained quite well here: lms.example.com/how-to-create-incremental-and-differential-backups-with-tar

There seems to be a way to create incremental backups with mysqldump, but it looks cumbersome: lms.example.com/2020/01/30/mysql-incremental-backup-with-mysqldump-and-rdiff-backup/

Besides, most databases compress to a relatively small size so it doesn’t matter if they’re added 14 times in a row.

Differential Backups

20230605

We now have the following script in operation on the La the backup server server (198.51.100.12):

https://bitbucket.org/solindevs/serverbackupscripts/src/master/

This script is executed using a cron job started at different times for each vhost:

/usr/bin/php7.2 /home/backupvhost/scripts/backupvhost.php
-config="/home/inm/public_html/config.php" >/dev/null

This script creates differential backup and sends the encrypted tar ball to:

admin@example-host:/users/solin/{$hostname}_server_backups/{$CFG->dbname}/{$CFG->dbname}.backup_day{$backupday}.tar.gz.gpg

There is a user backupvhost on the La the backup server server who has an ssh key pair. The public key is stored on the Strato server to enable authentication without the need of specifying a password (there is no passphrase).

Please note: there will always be one additional full backup to make sure the incremental parts from the previous period can also be restored.

Troubleshooting

20230602

Not working:

• INM – destination folder on Hidrive was incorrectly named

• Leerhuis – was no longer present in backupvhost cron job

• Mensenrechten – first no cron job and then option –hostname=”example platform” was missing

20230619

Not working:

• Exampletenant2 – backup script must run under php 5.6 since it’s going to include config.php, which in turn reads some of the Moodle files. This is Moodle 2.6, so php 5.6 is needed, not php 7.2

• example-site – not clear why backup wasn’t created overnight. Test run works fine. I’ll have to keep an eye on it.

20230620

• example-site may still not be working: there was a fresh full backup made at 1:51 am, but today is day 2 (Tuesday, not Monday) so we would have expected a day2 backup… Let’s keep an eye on it.

20230621

• example-site is one day behind. Other than that, everything seems to be working fine. The cron job was set to work at 1:50 am, so maybe some date in the system is still at the previous day. I did another run, resulting in a ‘backup_day03’ file. I’ve also changed the cron job to run at 3:50 am.