The Rotunda in Elder Park, Adelaide – Australia

Below is a gallery of photos that I took the other day on my way to work. I ended up running a little late, and even tweeted about it.

I hope to take more photos in the future and post them here.

New Utility Script: MdlDeprecatedCheck

'Extravaganza Deprecated' by Jan Kus

‘Extravaganza Deprecated’ by Jan Kus

For the past few weeks the focus of my work at NetSpot has been on Moodle upgrades. One of the many aims of an upgrade project is to ensure that there are minimal debugging messages being displayed during the testing process. One of the most common reasons for debugging messages being displayed is the use of functions that have been deprecated in the latest version of Moodle.

To help in fixing these issues before the client sees the message, I’ve developed a new utility script. The script is called MdlDeprecatedCheck and it is available in my GitHub repository. The script builds a list of functions, based on the deprecatedlib.php file, and compares it to the list of functions used in the target file or directory of files. If a match is found it is reported.

In this way I can identify the parts of the code that need updating to use the latest functions, and stop debugging messages from being displayed. As always, if you find any of my utility scripts useful, please let me know.

Using WordPress with HTTPS: Lessons Learned

'Learning' by Anne Davis 773

‘Learning’ by Anne Davis 773

A few days ago I wrote a post about why I switched my little part of the Interwebs to HTTPS. When I restarted my blog a few months ago I chose WordPress, mainly because I have been using it off an on for a number of years.

For example in the past I used WordPress to set up a centrally administrated blogging platform for the university that I was working for at the time. I even wrote a chapter on a small number of WordPress plugins that I wrote in a book on library mashups. The book was published in 2009.

I have learnt two new things from using WordPress with HTTPS.

Redirecting non HTTPS requests

The first lesson to learn was how to automatically redirect requests made using HTTP, to use the HTTPS version of the same URL. For example to automatically redirect from ‘http://techxplorer.com’ to ‘https://techxplorer.com’.

To do this I’ve added the following rewrite rules to my .htaccess file.

The important thing is to add these lines before the rules to make WordPress permalinks work. As an example, the relevant lines of my .htaccess file, including the rules for using permalinks, is as follows:

With this in place, any request for content using HTTP is automatically redirected to use HTTPS.

Fixing Absolute URLs to Post Content

The second lesson was that WordPress stores links to content within a post, such as the image above, as an absolute URL. That is, a URL that specifies the protocol, hostname and full path to the content. This is to ensure the link to the content works with the permalink format when the post was originally written, and in the future if the permalink format changes.

Including content loaded via HTTP when the page is loaded via HTTPS causes a mixed content warning to be displayed by the browser. This is because a page that contains mixed content is considered a security risk. More information on mixed content is available here.

WordPress uses a MySQL database to store post content. Therefore fixing the issue is a simple matter of using a SQL query like this:

Obviously if you’re intending to this with your own site you will need to:

  1. Take a backup of your database first, just in case
  2. Replace my domain name with your own

I remain pleasantly surprised by how easy this whole process has been. It has been so easy, that the cynical IT professional in me is just waiting for something bad to happen.

If you notice anything wrong with the site, please let me know.

Forensic Programming: Just Because You Can, Doesn’t Mean You Should

'Risk' by carnagenyc

‘Risk’ by carnagenyc

I’ve written a few posts detailing my thoughts on what I’ve called Forensic Programming. Today I’d like to add another one, which is part reflection and part cautionary tale.

As you know I work at NetSpot as a Senior Software Engineer, supporting our partners in their use of Moodle. The past week has seen us enter our upgrade season. During the next few months we work with all of our clients in undertaking a major upgrade of to a newer version of Moodle. This work was inspiration for two utility scripts, which I released the other day.

These projects are driven by a lead software engineer, as well as a lead consultant who provides project management oversight. Typically a working group is also formed at the partner organisation. As you can imagine, a major upgrade is a significant project.

As the lead software engineer on an upgrade project, you get a very good look at all of the changes made to Moodle to help our partners make the most of their investment. You also get to see the plugins that they have installed, as well as customisations that have made to them. Some of the customisations I’ve seen recently have been the impetus for this post.

Moodle, the third party plugins we have installed, and the customisations we write are all open-source software. As such we have the ability to change the source code to make Moodle meet the needs of our partner organisations. The ability to undertake these customisations is one of the reasons why Moodle, and our services, is attractive to our partners. We can customise their Moodle instances to work the way that the need.

Such customisations can be dangerous sometimes, and carry significant risk. Just because we can edit the source code, and therefore the behaviour of Moodle, doesn’t mean that we should. This is particularly true of changes to third party plugins which integrate with a third party website or service.

Recently I upgraded a third party plugin for one of our clients, and this caused some very odd behaviour in the third party system. For example records were being created in the third party system that didn’t have the unique identifier that our partner expected.

This error was caused by a customisation to a very early version of the plugin. The customisation had survived at least one major Moodle upgrade, and has become an integral part of the way in which the plugin is used.

Unfortunately the change was made so long ago that any documentation on the customisation had been lost. There was no indication that the customisation needed to be carried over, until the functionality provided by the plugin broke.

Changes like this to third party plugins, which integrate with services over which we have no control, carry a high degree of risk. For as we saw, our partner lost functionality that they required, and there was little help from the third party plugin provider. This is because we’re using their plugin in a way that they had not envisaged.

This is not to say that I believe the change should not have been made. Far from it. If a change is required by a partner organisation, so that they can achieve the most from the system, it should be made. However the risk of making the change to future upgrades must be assessed and steps taken to mitigate that risk.

At a minimum documentation needs be made to ensure that the customisation is tracked. This includes documentation in a system such as JIRA, as well as by comments in the code. In this way when the code change conflicts during an upgrade, or the code is being evaluated as part of an upgrade, the required change is more easily identified.

In summary, just because you can change the code, doesn’t mean that you should. All changes carry risk, and changes to code that integrates with third party services carry a higher degree of risk. Additionally if a change is made, it needs to be appropriately documented.

Switching my part of the Interwebs to HTTPS

'Padlock' by rjp

‘Padlock’ by rjp

Earlier this week I saw this post, by one of my favourite authors Charles Stross. In it he linked to this post by Google that indicates that the use of HTTPS for a website will start to be a ranking single for ranking search results.

Over the past few days I’ve taken the plunge and switched my little part of the Interwebs over to the HTTPS.

I was pleasantly surprised by how easy my hosting provider, Dreamhost, made the whole process. It was so easy that the cynical IT professional in me is just waiting for something bad to happen.

If you notice anything wrong with the site, please let me know.

Communication is critical to your success

'Type & Write' by ®DS

‘Type & Write’ by ®DS

Earlier this year I undertook a technical writing course. Specifically the Technical Writing course at TAFE SA. I’d like to take a moment to reflect on why I undertook the course.

Over my career I’ve received positive feedback on my written and oral communication skills. Including the documentation that I’ve written for a variety of software projects. This documentation varied from procedure manuals and technical notes, through to longer pieces such as a my honours thesis and various reports.

I feel that in a technical role, such as a software engineer, the technology is only one part of the solution that you’re working on. Communication with stakeholders, clients, team members, and management is just as important as the software design and implementation.

For example:

  1. Communication with project stakeholders: In my experience there are a significant number of stakeholders in any sufficiently large project. If you can’t communicate effectively with the stakeholders, who all have varying levels of technical expertise, you put the project at risk
  2. Communication with the client: If you can’t reach consensus with the client on what their needs are, the specification for your development will be flawed. This in turn, leads to a flawed implementation and a project failure
  3. Communication with your team: If your team doesn’t understand what it is that you’re doing, or have done. They can not effectively back you up when you’re not available or have moved on to a different project. As such, your team isn’t performing a well as it should. The same can be said for any new process or system that you are implementing internally for your team.
  4. Communicating with management: In my experience, effective communication with your manager is critical to your success as a software engineer or other technical role. Your manager communicates what you’re doing with the wider organisation. They also have a wider view of the organisation than you do, and can share this with you

Ultimately it comes down to this:

In any technical role, the technology is only one factor in your success. The people that you work with, in my opinion, are a more significant factor. In fact, I would argue that the people are critical to the success of your current task, both past and present projects you have worked on, and your career overall.

Even though I have received consistently positive feedback on my technical writing, I still felt it was important to have my skills evaluated by people who work in the field of technical writing. I learnt new skills, and ways of looking at my writing, and I’m using them in my day-to-day work.

In summary, if I could offer one piece of advice it is this: Don’t neglect your communication skills, as they are critical to your career success.

Two new Git related utility scripts

It is upgrade season at NetSpot, not to be confused with rabbit season or duck season. I’m the lead software engineer for at least two major Moodle upgrade projects this year. As such I’m always looking for things that will help me get my job done. Today I’m releasing two new utility scripts which do just that.

The first script is called GitFindCommit. This script can search the log of a a Git repository, looking for commits which have a description which matches a pattern. This is particularly helpful in identifying the commit, or commits, which relate to a particular JIRA item. This is because it is NetSpot policy to preface the description of a commit with the JIRA code.

The second script is called GitLogExport. The purpose of this script is to identify commits which match the specified pattern, and export the list of those commits into a CSV file. This is helpful in identifying a list of commits that represent code changes that need to be migrated as part of the upgrade project. Additionally the script can filter the list of commits to reverse the order of commits (making the most recent last) and remove duplicate descriptions.

It is going to be a busy few months while we work through all of the major upgrade projects that we have. These two scripts will make life a little easier. As always, if you find any of my utility scripts useful, please let me know.

New Utility Script: FixLineEndings

'End of the line - Game over' by bengt-re

‘End of the line – Game over’ by bengt-re

At NetSpot, where I currently work as a Senior Software Engineer, we’re rather particular about the code that we add to our repositories. In particular, we pay careful attention to unnecessary whitespace and non unix line endings.

To help in integrating third party code in our source code repositories I’ve written a small script today called FixLineEndings. It’s one of the scripts in the Techxplorer’s Utility Scripts repository on GitHub.

The script is relatively simple in that it:

  1. Recursively searches for files, starting at a specified parent directory
  2. Filters this list of files to only those matching a set of file extensions
  3. Executes the dos2unix command on each file

I’ve used the dos2unix command to fix the line endings on the files for a very specific reason. The more I thought about the types of files I need to work with, the more I became concerned about edge cases that I’d need to be careful of.

Therefore in the spirit of, ‘Don’t Repeat Yourself‘ and leveraging the ‘Make each program do one thing well‘ philosophy I used a well known tool to do the heavy lifting for my script.

I look forward to using this script to make my job easier. As always, if you find any of my utility scripts useful, please let me know.

Two new Moodle language related utility scripts

Bounty Hunting School - Language Course: by Stéfan

Bounty Hunting School – Language Course: by Stéfan

In what is possibly the largest single contribution to my Techxplorer’s Utility Scripts, I added two new Moodle language related utility scripts earlier this week.

The first script is called MdlLangCompare. The purpose of the script is to compare one or more Moodle language customisation files to the corresponding default language file.

For example, comparing a list of files that override the language strings of various Moodle components with institution specific customisations. A classic example is where the language strings have been modified to change all references of ‘course’ or ‘courses’ to ‘site’ and ‘sites’.

The script will identify:

  1. Any customisations for strings that are not used any more;
  2. A colourised diff of the two strings for the same identifier; and
  3. Statistics on the differences, which helps identify common replacements.

The second script is called MdlLangConvert. The purpose of this script is to compare a language customisation file with the default Moodle language file. The list of customisations is analysed in a similar way to the MdlLangCompare script. The user can then select a list of customisations and, have this list saved to an XML file. We use this XML file at NetSpot to help us manage our clients language customisations.

The impetus for these scripts was twofold:

The first was that I’m currently the lead software engineer for a clients upgrade. As such we’re migrating their language customisations into the XML format used by us internally to manage these customisations. I didn’t just simply want to copy all of the existing customisations over. I wanted to identify those customisations that were no longer necessary, as well as those that could be consolidated down into global find and replace rules.

The second was that I wanted to learn more about how Moodle manages its language strings, and explore options for producing a diff of two strings at the word level. I also wanted to continue my explorations of technologies such as PHPUnit, and continuing to exercise my PHP development skills.

As always, if you find any of my utility scripts useful, please let me know.

Eating Your Own Dog Food, Why?

"How Others See Me" by carterse

“How Others See Me” by carterse

I heard the phrase “Eating your own dog food” earlier last week while listening to a podcast. It reminded me of a conversation we had on our internal social network at NetSpot earlier this month. The discussion was in relation to one of our teams exploring options for a replacement to our XMPP server for internal team chat, communication and collaboration.

NetSpot is part of Blackboard, and as such we have access to Blackboard products and services. The discussion centred around the possible use of a Blackboard product to replace the XMPP server.

It was during this discussion that the phrase “eating our own dog food” was used. In a suggestion that we should be using our own products and services internally. Because it would be better than using one of the other options under consideration, as it was an internal product.

Wikipedia defines the term as:

Eating your own dog food, also called dogfooding, is a slang term used to reference a scenario in which a company uses its own product to validate the quality and capabilities of the product.

In this case it wasn’t to validate the quality of capability of the product, rather it was because they were readily available, and as such may be a better candidate due to reduced costs etc. Additionally using the product, or service, would help familiarise some of our teams in their use to help them in primarily sales discussions with our clients.

My concern with this approach, as I outlined in the post to our internal social network, is that our own products and services may not meet our specific needs. Let me explain.

The products and services that we support, in what I like to think of as collaborations with our clients, are designed to meet their needs and allow them to achieve their goals. One of their main goals is to provide an environment that facilitates teaching and learning for their students and staff.

One of our main goals is to work collaboratively with our clients, so that they can meet their goals. These goals are different, and as such we have different users with different use cases and requirements.

A technology that supports teaching a learning, may not be the best fit to help us to work together as a teams, across teams, and with our clients to help them achieve their goals.

In short, our use cases, and our user requirements may not be met the products and services that we support. This isn’t because they’re bad. On the contrary, I think some of them are very goo. It is because they’re aimed at different users, with different requirements and expectations.

With this in mind, when you hear the phrase “eat our own dog food”, take a moment to critically and consider if the “food” is the best fit for your needs and goals.  You may be surprised. It may be a good fit, and then again it may not.

The most important thing is to critically evaluate it, against the other candidates, with your own needs and goals at the top of the list. Additionally, don’t add any more importance to the product or service, simply because it is internal, evaluate all candidates on a level playing field.