A Practical Moodle Troubleshooting Workflow

List the Symptoms

Get access to Moodle and server

Ask the customer for:

• Moodle url

• Moodle admin username

• Moodle admin password

• Web server url or ip address

• Web server username

• Web server password

If the Moodle website is hosted on one of our own servers, we should already have this information – don’t bother the customer in that case, but check with your lead engineer instead.

Find the Root Cause

Checking and Enabling X-Sendfile for Large File Delivery (Moodle)

Overview

When Moodle serves large files (video, H5P assets, SCORM, large images), the default behaviour is for PHP-FPM to stream the file through pluginfile.php. This can block PHP workers for minutes and cause sitewide slowdown. Apache’s mod_xsendfile allows Apache to serve files directly from moodledata, bypassing PHP entirely. This section describes how to enable X-Sendfile and how to test whether it is working.

Symptoms

• Slow or hanging requests to pluginfile.php.

• PHP-FPM slow logs show stack traces in byteserving_send_file() or readfile_accel().

• Many long-running PHP-FPM workers during video playback.

• Users report Moodle becoming unresponsive while accessing large videos or files.

Step 1: Verify Apache has mod_xsendfile

On the server, run:

a2enmod xsendfile

If the module is missing, install it:

apt update

apt install libapache2-mod-xsendfile

a2enmod xsendfile
service apache2 restart

Step 2: Add required directives to the vhost

Inside the VirtualHost:

<IfModule mod_xsendfile.c>

XSendFile on

XSendFilePath /path/to/moodledata

XSendFilePath /path/to/moodle/code

</IfModule>

Example for a standard installation:

XSendFilePath /home/USERNAME/moodledata

XSendFilePath /home/USERNAME/public_html

Apache must be allowed to access both moodledata (filedir, temp, cache, etc) and dirroot.

Reload Apache:

service apache2 reload

Step 3: Enable X-Sendfile in config.php

Add these lines above the require_once(__DIR__ . ‘/lib/setup.php’); line:

$CFG->xsendfile = 'X-Sendfile';
$CFG->xsendfilealiases = array();

– Default for standard Apache Moodle sites: use X-Sendfile and leave $CFG->xsendfilealiases = [];

– If logs or an X-Sendfile probe show aliased paths such as /dataroot/…, add the required alias mapping for that site.

Step 4: Basic Apache test (without Moodle)

Create xsendtest.php in the webroot:

<?php
header("X-Sendfile: /home/USERNAME/moodledata/xsend-test.txt");
header("Content-Type: text/plain");

header(“Content-Disposition: inline; filename=\”xsend-test.txt\””);

exit;

Create the file:

echo "hello xsendfile" >
/home/USERNAME/moodledata/xsend-test.txt

Test:

curl -i https://yourdomain/xsendtest.php

Expected:

• Response shows Content-Disposition, Content-Type, but no X-Sendfile header (Apache strips it)

• File downloads quickly If you remove the file or path, Apache should return 404, confirming Apache is handling the file rather than PHP.

Step 5: Moodle side test (optional but recommended)

This test confirms Moodle’s send_stored_file() triggers X-Sendfile.

Create moodle_xsend_probe.php in public_html:

<?php
require(__DIR__ . '/config.php');
require_once($CFG->libdir . '/filelib.php');

// Create a temporary stored_file so Moodle goes through send_stored_file().

$fs = get_file_storage();
$syscontext = context_system::instance();
$content = "xsend probe test";
$file = $fs->create_file_from_string(
array(

‘contextid’ => $syscontext->id,

‘component’ => ‘local_test’,

‘filearea’ => ‘probe’,

‘itemid’ => 0,

‘filepath’ => ‘/’,

‘filename’ => ‘xsend_probe.txt’

),

$content

);

// Capture headers before Apache strips them.
register_shutdown_function(function() {
file_put_contents(
$GLOBALS['CFG']->dataroot . '/moodle_xsend_probe.log',

“headers_list() at shutdown:\n\n” . implode(“\n”, headers_list(

);

});

send_stored_file($file);

Test:

curl -I https://yourdomain/moodle_xsend_probe.php

Then check:

cat /home/USERNAME/moodledata/moodle_xsend_probe.log

Expected inside the log:

X-Sendfile: /home/USERNAME/moodledata/filedir/xx/yy/xxxxxxxx...

This is definitive proof that Moodle is using X-Sendfile.

Step 6: Final check if troubleshooting performance

If you still see slow requests:

• Inspect php-fpm slow logs (/var/log/php*-fpm.slow)

• Look for long traces involving:

byteserving_send_file

readfile_accel

pluginfile.php

If these disappear after enabling X-Sendfile, the configuration is correct.

Result

When X-Sendfile is working:

• PHP-FPM workers are not tied up streaming videos.

• pluginfile.php requests become extremely fast.

• Concurrency improves and site sluggishness disappears.

Tweaking php-fpm for Optimal Performance

Summary

Max Children Issue

Php-fpm may cause trouble if the number of pm.max_children is set too low, for a specific vhost (this is configured through a ‘pool’ file). You’ll see this mentioned in the log, /var/log/php7.4-fpm.log, e.g.:

[30-Sep-2021 10:08:11] WARNING: [pool www] server reached pm.max_children setting (5), consider raising it

Or:

26-Oct-2023 11:50:13] WARNING: [pool 1692438784256193] server reached pm.max_children setting (16), consider raising it

On Ubuntu (hosting provider), the config file for the web user (typically www-data) is here:

/etc/php/X.Y/fpm/pool.d/www.conf. The other pool files are in the
same directory. For instance:
root@augeo:/etc/php/8.1/fpm/pool.d# ls -lah

total 64K

drwxr-xr-x 2 root root 4,0K okt 26 15:06 .

drwxr-xr-x 4 root root 4,0K okt 26 14:49 ..

-rw-r–r– 1 root root 420 aug 17 12:06 169227396768674.conf

-rw-r–r– 1 root root 413 aug 17 13:09 169227776679898.conf

-rw-r–r– 1 root root 412 okt 26 14:54 1692438784256193.conf

-rw-r–r– 1 root root 438 aug 28 11:28 16932221191307961.conf

-rw-r–r– 1 root root 445 aug 28 11:32 16932223221310218.conf

-rw-r–r– 1 root root 438 aug 28 19:54 16932524711390798.conf

-rw-r–r– 1 root root 438 aug 28 19:57 16932526681392819.conf

-rw-r–r– 1 root root 473 sep 14 11:34 16946912911243061.conf

-rw-r–r– 1 root root 21K okt 26 13:37 www.conf

Here, 1692438784256193.conf is the php8.1-fpm configuration file for the user elo. How do we know? Well, virtualmin creates an entry in the vhost conf file, e.g. /etc/apache2/sites-available/lms.example.com.conf:

<FilesMatch \.php$>
SetHandler
proxy:unix:/var/php-fpm/1692438784256193.sock|fcgi://127.0.0.1
</FilesMatch>

As you can see, the socket number matches with the number used in the conf file name.

In addition, we find the user elo in the output of the php8.1-fpm configuration command:

php-fpm8.1 -tt

Which yields (for our example):

[1692438784256193]

prefix = undefined

user = elo

group = elo

listen = /var/php-fpm/1692438784256193.sock

Again, the numbers match and here the socket is also explicitly mentioned.

So, in the php-fpm conf file that you have found, change the setting for pm.max_children, e.g. from 5 to 25 (this is really just an example, see the spreadsheet below to compute the actual value), and do:

service php7.4-fpm restart

Or:

service php7.4-fpm restart

Getting The Correct php-fpm Settings

free -hl

command outputs free and available memory. Use the available memory in your calculations.

The difference between free memory vs. available memory in Linux is, free memory is not in use and sits there doing nothing. While available memory is used memory that includes but is not limited to caches and buffers, that can be freed without the performance penalty of using swap space.

lms.example.com/free-vs-available-memory-in-linux/

Use something like pstree -c -H 19741 -S 19741 to see the current number of php-fpm7.4 processes, where you find the pid by looking for php-fpm: master process (/etc/php/7.4/fpm/php-fpm.conf) in the output of ps -ef. This tells you how many ‘children’ are currently spun up.

Compute php-fpm process size:

python /usr/local/bin/ps_mem.py | grep php-fpm (grab the Python script here).

(Or: python3)

To convert Gi to Mi: lms.example.com/from/GiB/to/MiB

Multiple Sites or Vhosts

A final thing to keep in mind is ‘if these “tuned” values are calculated based on the maximum capacity of your server and you put the same values to every site, you’ll consume your resources multiple times. Instead, these values should be distributed between the pools so that the sum from all pools is equivalent with the “tuned” value.’

(..) ‘Whatever you do, keep your sites in separate pools with separate user accounts. Otherwise a compromise on a single site can spread across all your sites.’

https://serverfault.com/questions/952658/php-fpm-conf-per-site-vs-server-php-pool-performance-tunning

The latest versions of Virtualmin (writing this 20231026) seem to automatically create a pool file for each new vhost. These articles describes how to do it manually:

lms.example.com/how-to-run-sites-securely-with-apache-and-php-fpm-on-ubuntu-16-04-lts/
lms.example.com/how-to-install-php-fpm-with-apache-on-ubuntu-22-04-2/
lms.example.com/guide-to-combining-apache-virtual-hosts-and-php7-fpm/

From the latter article: “To complete the process, you should repeat the steps for each of your virtual hosts. When you are entirely sure mod_php is not being used anymore you can disable it through

$ sudo a2dismod php8.1

Until you’ve done this, Apache will still include a PHP process for every request, meaning the memory usage will stay the same and possibly be even higher.”

Checking php-fpm Configuration

Simply use this command to see the configuration parameters that are currently being used:

php-fpm8.1 -tt

And use the following command to test whether the configuration is correct:

php-fpm8.1 -t (or php-fpm8.1 --test)

As an aside, you can also check the configuration for Apache:

apachectl configtest

Checking the php-fpm status

Go to /etc/php/X.Y/fpm/pool.d/www.conf (e.g. /etc/php/7.4/fpm/pool.d/www.conf) and look for pm.status_path. Uncomment it and add /phpXY-fpm/status, e.g.:

pm.status_path = /php74-fpm/status

(Do NOT insert a dot here)

Then, in your /etc/apache2/apache2.conf file, add:

<LocationMatch “/php74-fpm/status”>

ProxyPass “unix:/var/run/php/php7.4-fpm.sock|fcgi://127.0.0.1/php74-fpm/status”

</LocationMatch>

Obviously, the paths must match exactly. Restart Apache and php-fpm, and go to your web server’s location, e.g.:

http://198.51.100.10/php81-fpm/status

This should give you something like:

pool: www
process manager: dynamic

start time: 26/Oct/2023:13:37:45 +0000

start since: 13

accepted conn: 1

listen queue: 0

max listen queue: 0

listen queue len: 0
idle processes: 31
active processes: 1
total processes: 32

max active processes: 1

max children reached: 0
slow requests: 0

Please note: this only shows the status for that specific pool. In the example above, that’s www.

Sluggish Moodle Or Totara Site

• Check the images that themes like Adaptle allow you to upload for the frontpage, header and background. We once had a Moodle site that was loading two essentially the same images of 7 MB each. Needless to say, this makes loading the site quite sluggish.

• The scheduled tasks may add up, especially if you have many users and many automatic cohort syncs (audiences, in Totara). Please note that Totara (and Moodle prior to version 3.7) does not have the task_scheduled table. No serious logging takes place for scheduled tasks.

MySQL Connection Timeout Or Lost Connection

From example platform server, /etc/mysql/mysql.conf.d/mysqld.cnf:

## TEMP 20230322 the lead engineer – successfully imported a 2.8 GB dump with these settings – main thing is innodb_buffer_pool_size which must be SMALLER

## See https://dba.stackexchange.com/questions/124964/error-2013-hy000-lost-connection-to-mysql-server-during-query-while-load-of-my

#innodb_lock_wait_timeout = 60

#net_read_timeout = 28800

#net_write_timeout = 28800

#connect_timeout = 28800

#wait_timeout = 28800

#delayed_insert_timeout = 28800

#innodb_buffer_pool_size = 4294967296

Timeout Issues

mod_fcgid: read data timeout in 40 seconds

This error may pop up if you’re using fcgi and you’re trying to upload a large scorm file (say 400M) that requires quite some processing time:

mod_fcgid: read data timeout in 40 seconds

(This error may be disguised as a HTTP 500 error, but the Apache log file should contain the actual error message.)

The solution is to increase the value of FcgidIOTimeout in your vhost configuration, e.g.:

FcgidIOTimeout 600

This directive can be put directly inside the VirtualHost part of the Apache configuration file for the website, e.g.:

<VirtualHost 198.51.100.11:443>

FcgidIOTimeout 600

Don’t forget to save the conf file and restart the webserver afterwards, with /etc/init.d/apache2 graceful.

Varnish Timeout

Usually it’s enough to increase max_execution_time in php.ini to solve any timeout issues. However, I recently (20250206) encountered a timeout issue in an AWS EC2 instance that turned out to be using Varnish, which is a caching tool. Varnish was set (in /etc/varnish/.default.vcl) to 30s. After resetting set bereq.first_byte_timeout to 300s (notice the ‘s’ by the way: Varnish completely crashes the site if you leave it out), I had to restart varnish:

sudo systemctl restart varnish

And that solved the issue (together with the max_execution_time increase, of course).

Common But Hard to Spot Issues

Spaces in PHP Files

If you inadvertently introduce a space before the <?php opening tag, especially in config.php, things will go wrong. This will pollute the output buffer in cases where output is created and sent to the browser through php, e.g. pluginfile.php or theme/yui_combo.php. Examples are images, css, and javascript. If any output, such as a space, is sent before the headers, the results are catastrophic, especially for all binary files such as files (often, css and javascript is zipped before being sent to the browser, making these resources binary too).

Symptoms: images do not load, no layout (css) is applied and menus (requiring javascript) do not work.

Configuration Issues

Unexpected Name Showing as Sender in Course Welcome Emails

We encountered a case where Moodle’s course welcome emails were being sent with the name of an unrelated user as the sender. This was confusing, since the person whose name appeared had nothing to do with the specific course.

Root cause
By default, the setting Send course welcome message – From in the Manual enrolments plugin (/admin/settings.php?section=enrolsettingsmanual) was configured as From the course contact. Moodle interprets “course contact” as the first user it finds with a role listed in the coursecontact configuration (commonly the editingteacher role). Importantly, this lookup also includes role assignments at the system or category level, not only at the course level.
Because a user had the editingteacher role at system level, their name was used as the sender in welcome emails across the site.

Resolution
We changed the configuration in:
Site administration → Plugins → Enrolments → Manual enrolments

Setting:
Send course welcome message – From → From the no-reply address

This ensures that all course welcome emails are now sent from the neutral noreply address (e.g. noreply@lms.example.com), and no personal user names appear unexpectedly as the sender.

Best practice
Always configure Send course welcome message – From to From the no-reply address unless there is a strong reason to display the course contact’s name. This avoids confusion and ensures consistency across all courses.

Case Study: Extremely Slow Course Creation (ICM 2025)

Symptoms

Creating a new course in Moodle 4.1 took minutes to complete, whereas on the previous host it completed in under 30 seconds. Logging in via SAML2 also appeared slower.

Root Cause

The issue was caused by inefficient MySQL write performance, specifically InnoDB’s default behavior of flushing transaction logs to disk after every single write operation. This configuration is safe but can be extremely slow on systems handling many small transactions (like Moodle’s course creation process).

In this case, the new hosting environment used SSD storage but was still using conservative, HDD-era MySQL defaults:

• innodb_flush_log_at_trx_commit = 1

• innodb_io_capacity = 200

• innodb_io_capacity_max = 2000

This caused excessive fsync operations (waiting for every commit to fully write to disk).

Resolution

The hosting partner (Kaliber) optimized MySQL for modern SSD storage and Moodle’s workload. The following key changes were applied:

innodb_io_capacity = 1000

innodb_io_capacity_max = 2000

innodb_flush_log_at_trx_commit = 2

innodb_read_io_threads = 4

innodb_write_io_threads = 4

Explanation:

• Increasing innodb_io_capacity allows MySQL to perform more I/O operations per second, aligning with SSD capabilities.

• Setting innodb_flush_log_at_trx_commit = 2 reduces log flushes to once per second instead of every transaction — a safe compromise that greatly reduces latency.

• Increasing I/O threads improves concurrency on SSDs with multiple queues.

Testing on the development server confirmed that even the most aggressive options (O_DIRECT_NO_FSYNC) only gave minimal additional gain; the 1 fsync per second configuration was the “golden mean.”

After implementing these settings, the “Create new course” operation returned to expected performance levels (seconds rather than minutes).

Lessons Learned / Best Practice

• Moodle’s “create course” is a write-heavy operation; database write latency dominates performance.

• Always review and tune MySQL’s InnoDB write path after migrations.

• On SSDs, conservative defaults such as innodb_flush_log_at_trx_commit=1 and low I/O capacity settings can cause severe slowdowns.

Start with:
innodb_io_capacity = 1000

innodb_io_capacity_max = 2000

innodb_flush_log_at_trx_commit = 2

• If performance issues persist, consider checking:

  • Actual disk type (ensure SSD/NVMe)

  • innodb_flush_method (O_DIRECT is optimal for SSD)

  • Background flushing and redo log sizes

Reference

This issue occurred on the ICM production system (2025) after migrating Moodle 4.1 to a new host. It was resolved by Bart (Kaliber) following performance analysis and parameter tuning.