The reason I wrote this script, is that last week I had to do some work for a NetSpot client who was having timeout issues. The Moodle plugin that they were using wasn’t completing a task. It was taking too long and was being killed by PHP. There were two issues that I identified, with the help of a colleague.
The first issue was that the plugin was calculating a grade and then writing that grade to the database. Basically the code did this:
- Retrieve all records related to grading of assignments, five records per assignment in this case
- Calculate the grades
- Use the records from step 1 as a list of assignment IDs to update
Can you see the problem?
Step 1 retrieves five records per assignment, and step 3 uses this as a list of assignment IDs to update. Therefore each assignment was updated 5 times with exactly the same grade. While this is certainly inefficient, this isn’t a big deal if have a small number of students in a course. But if you have 1,500 students in a course that means that there is going to be 7,500 database writes.
To fix this it was necessary to do two things:
- Store the assignment id, and a small number of other ids, only once
- Use this revised list when writing the grades to the database
Now instead of 7,500 database writes there were only 1,500 writes, one for each student. Additionally, rather than retaining the entire database record for later, only the small number of IDs per record were retained. This second change also reduced memory consumption of the script.
The second issue was tricker to resolve. As part of the investigation I needed to use the profiler of the Xdebug extension for PHP to profile the script and understand where the performance issues were. I also used the Xdebug helper extension for Google Chrome to help only profile the scripts that I needed to.
Profiling a script has a significant performance impact. Writing the output file to a RAM disk, which is significantly faster than even a SSD, helps speed up the process. Hence the development of the MakeRamDisk script which made it easier to make RAM disks of various sizes. The trick with RAM disks is to create one that is big enough for the task, which doesn’t deprive the rest of your system of the RAM that it needs to work.
Lastly I used the qcachegrind application, part of the KCachegrind suite, to analyse the file and identified a few areas that could be optimised. As always with applications like this it was easy to install it using Homebrew.
Now I have some new skills to add to my forensic programming toolkit.