Shy Birds Breakfasting in North Adelaide

On my way into work earlier this month, I cam across these two shy birds having breakfast in the parklands.

Unfortunately they were too shy for any close up photos.

Hercules is a Supporter of the Irish Cricket Team

I was walking through the parklands on my way to work at Blackboard, formerly NetSpot, in North Adelaide when I came across Hercules. I’m assuming he is a supporter of the Irish Cricket Team. My friend Mark Drechsler tells me that they were in town on the weekend for a cricket match.

A Green Torrens Lake

On many mornings, my walk to work at Blackboard, formerly NetSpot, in North Adelaide is very pleasant. It is a nice way to start the day, and is around a 3km walk from the bus stop in the CBD where I get off the bus. The only mornings where it isn’t pleasant is if it is really hot, or really cold, or it is raining and I’ve forgotten my umbrella.

On this particular morning in late February the Torrens Lake, part of the river Torrens, was quite green. There were a significant numbers of algal blooms on this particular morning. It was so striking that I had to stop and take a few quick photos.

Algal Bloom in the Torrens Lake

Algal Bloom in the Torrens Lake. By: techxplorer

To get a sense of the scale of the bloom that I could see, I used the panorama function of my venerable iPhone 4S.

Panorama of the green Torrens Lake. By: techxplorer

Panorama of the green Torrens Lake. By: techxplorer

No, I will not upload my address book. So stop asking!

"Privacy" by: _Bunn_

“Privacy” by: _Bunn_

Those people who follow me on Twitter would have seen the following conversation last night that I had with a member of the LinkedIn Customer Service team.

Now that I’ve succumbed to the lure of LinkedIn, I’ve been exploring the service in order to make the most of it. I think I should work to extract the most benefit out of sharing my personal and private information with a faceless corporation.

Screen capture of the twitter conversation.

Screen capture of the Twitter conversation.

One thing that I have noticed is that LinkedIn is very persistent in asking me to upload my address book. Either directly, or by giving it access to my Google Account. This is something that I am not prepared to do, and being asked to do it repeatedly is getting very tiresome.

There are two main reasons why I am not willing to upload my address book.

The first, I alluded to in my earlier post. My address book contains contact information on people who I have some form of relationship with. I consider the information that I have about others to be private, and I won’t share it with outside services. I simply don’t think it is ethical or moral to do so.

The second is that my address book as information in it about my doctor, and other personal information. I don’t want a faceless corporation mining that data for some sort of nefarious purpose. Yes, I know I’m being a little hyperbolic, but that’s simply how I feel.

What surprised me, as a software engineer always thinking about ways to make the software I write easier to use, is that it isn’t possible to disable this “feature”. As you’ll note in the above conversation, Kat says that “There’s no way to permanently opt-out of this feature”.

Upon reflection it isn’t that surprising. After all, the data about me is what is most important to social media services, not necessarily the experience I have as a user of their service. Especially if I’m not using their product, and I am the product that they’re selling.

I’ve succumbed to the lure of LinkedIn

"Linkedin Chocolates" by Nan Palmero

“Linkedin Chocolates” by Nan Palmero

Late last week I succumbed to the lure of LinkedIn and created an account. You can see my profile here.

In the past, I have been adamant that I wouldn’t have a LinkedIn account, and I’d limit my “social networking” to Twitter.  My Twitter profile is available here. So what changed.

Firstly, I don’t think any social network is inherently bad, nor are they inherently good. They’re tools to be used, just like the Internet is. The benefits of using them need to be weighed against the costs. For while having a LinkedIn account is free, that’s only an example of a monetary cost.

The main cost to me is privacy. By uploading my employment history into a third party service, over which I have no control, I’ve lost a bit of my privacy. Not only does LinkedIn share information with people in my network, I also makes select portions of it available to the public. As such it’s a piece of my private that  I’m never going to get back.

In the past, this cost was greater than any perceived benefit that I had. What has changed is that the benefit now outweighs the cost. It became apparent in the past week that I’ve reached a point in my career where a LinkedIn profile has become expected. If I want to continue to progress in my career, I need to have a LinkedIn profile.

That is to say, the lack of a profile to me was a greater cost than the cost to my privacy. So at this point creating a profile was a logical decision.

If I am asked wether someone should have a profile or not, my response has not changed. It depends on the individuals career goals, objectives, and the costs that they’re willing to incur. I still don’t think that having a LinkedIn profile, or any other social media profile for that matter, should be “required” and something that should entered into with critical thought.

Now that I have a profile, if I’ve worked with you in the past, feel free to connect with me. For no matter how many times LinkedIn prompts me, I am not willing to upload my email address book so that they can potentially find other people for me to connect with. This is because my address book contains personal information about other people, and I respect their privacy and won’t be sharing their details without their consent.

Saving the Page of Failed Behat Tests in Moodle

"Agapostemon splendens, f, head, anne arundel county, md_2014-07-09-13.37.57 ZS PMax" by  USGS Bee Inventory and Monitoring Lab

“Agapostemon splendens, f, head, anne arundel county, md_2014-07-09-13.37.57 ZS PMax” by USGS Bee Inventory and Monitoring Lab

I will be the first one to admit that I was skeptical when we introduced automated Behat tests for our Moodle codebase at NetSpot, now Blackboard. For those that may not be familiar with Behat, it is a tool that allows for behaviour driven development (BDD).

Conceptually BDD is all about testing the behaviours of software. It emulates the actions that a user undertakes when using an application, and ensures that the results of undertaking that action are expected.

In our case, when using Behat, it does this by interacting with a special Moodle instance, making requests for web pages, as if the Behat script was a real user. It then analyses the pages returned by Moodle to ensure they contain specified strings. More information about how Moodle is integrated with Behat is available on the Behat integration page in the Moodle Developer Documentation wiki.

One aspect of the integration that used to cause significant frustration for me, was that it was difficult to work out why a test failed. This is because it wasn’t possible to see the contents of the page that had resulted in the error.

Or at least, that is what I thought.

When configuring the Behat integration I used options like this:

The other day I stumbled across a fourth option which has proven really useful. Now I configure the Behat integration like this:

This fourth line configures some code that ships with the Moodle Behat integration that is triggered when a Behat test fails. What it does is save the content of the HTML page that was returned when the test failed. This is a vital piece of information to work out why a test failed.

I am now an advocate for BDD and have used the Behat integration extensively for a new activity that I’m developing for one of our clients. It’s funny how one small configuration option can have such a huge impact on the way in which you use, and think about, a tool.

Debugging using a magical exploding class

'marvin' by photoguyinmo

‘marvin’ by photoguyinmo

Last week I was debugging an issue that I had in Moodle related to the use of a global variable in PHP. If you’re not familiar with global variables in PHP, conceptually they are a variable that can be accessed from any part of your script. Typically through the use of the ‘global’ keyword.

The issue I had was that a global variable was having a string property set to false somewhere during the execution of the script. This was causing errors in a completely unrelated part of the codebase.

I discussed this with a colleague of mine. They showed me a debugging technique which uses what they called a ‘magical exploding class’.

The first part of the technique is to define a class like this:

Once the class is defined, it is possible to use it like this:

The two key parts to this technique is that:

  1. The instance of the magical class automatically returns its value as a string automatically. This ensures that that other parts of the code don’t realise that the field in the global class is set to our class, and not a simple string
  2. If the content of the field is replaced with something else, the magical class is destructed which causes an exception to be thrown. In this way the code that is changing the field can be identified

Using this technique I was able to identify the troublesome code, and write a defect resolution. More importantly the issue wouldn’t have been possible if the global objects that Moodle uses were immutable objects. But that’s a discussion for another time.

Techxplorer’s Utility Scripts – Version 2.0

'number two' by mark dyer

‘number two’ by mark dyer

If there are two constants in the life of a software engineer they would be:

  1. You’re always learning. For example, learning new paradigms, programming languages, frameworks or just a different way of doing something you do regularly
  2. Software is never finished, there is always something to do.

Both of these constants came together this weekend when I decided to work on rewriting the scripts available in the Techxplorer’s Utility Scripts.

I’ve started rewriting the scripts that I use the most, taking advantage of the things I’ve learnt over since August 2013 when I started the project.

The three scripts that I’ve re-written so far are:

  1. DbAssist.php – Automate various database related tasks
  2. GerritPush.php – Make it easier to push changes to a Gerrit code review system
  3. GitFetchReset.php – Automate fetching changes from Git and reseting the current branch to the latest upstream version

Other scripts that I use regularly will be updated as time allows. All of the pre-existing scripts are still available in their own branch in the repository.

jQuery and Moodle. Just don’t do it.

'stop+think' by Caffeinatrix

‘stop+think’ by Caffeinatrix

Fair warning, this post is going to be a little bit of a rant.

My role at NetSpot / Blackboard is working as a Senior Software Engineer as the client development team for Enterprise Moodle. As part of my role I need to review plugins that our clients want to use as part of their Moodle instance.

The things we’re most in the lookout are potential issues that will have an adverse impact on our ability to provide our service to the high standard expected by our clients. An example is having very long timeouts on cURL requests, doing things in Moodle without appropriate considerations for security, or not using the standard Moodle libraries.

One thing I have noticed is that a number of plugins, and themes, use JavaScript that is built using the jQuery library. For reasons that I suspect are lost to history, and are not relevant to this discussion, Moodle HQ has settled on the YUI library.

Wether you agree with this decision or not, the fact remains that core Moodle uses YUI, and uses it extensively. Incorporating jQuery into your plugin or theme causes a number of issues for us and our clients.

One of the most severe are rendering issues of pages because the JavaScript code breaks in some way. Most often because loading both jQuery and YUI cause conflicts. This breaks the rendering of pages and takes us time to rectify.

The second issue is that by including jQuery the developer is, in my opinion, being disrespectful to the users of the plugin. This is because you’re forcing users to spend time and bandwidth downloading two JavaScript libraries.

In the event that, as a developer, you must use jQuery to achieve your goals, please adhere to the guidelines developed by Moodle HQ, which are available here. In particular don’t include your own copy of the jQuery library. It causes way too many issues for those of us who need to support users of your code.

If you feel strongly about the use of YUI, and want to contribute to the discussions of Moodle HQ about the future of YUI and Moodle, I encourage you to checkout out this Moodle tracker item, and this associated Moodle forum thread.

Here ends the rant.

Updated: January 26, 2015

A trusted colleague of mine has pointed out that discussions are underway at Moodle HQ on the possible switch to jQuery, and Require.js with possibly using Grunt as a build tool. More information is available in this Moodle forum thread, which I also linked to above. The most relevant forum post, so far, is available here.

This is particularly relevant as in August, 2014 it was announced that the YUI library is no longer actively maintained. You can read more about the decision here.

I believe that my original point still stands. As a developer, if you are writing JavaScript for Moodle, you should be using YUI and not jQuery unless there is a very serious and compelling reason to do so. As it causes complications for those of us that need to support users of your code, and is disrespectful to users of your code by forcing them to spend time and bandwidth on loading two JavaScript libraries.

This is because the decision on which JavaScript library Moodle will use in the future is yet to be decided. Until that decision has been made, and the version of Moodle that has made the switch reaches critical mass, the use of YUI is still going to be preferred.

Sending a password in a second email is not secure

'Broken Rusty Lock: Security (grunge)' by  Nick Carter

‘Broken Rusty Lock: Security (grunge)’ by Nick Carter

There are times when decisions made for reasons of ‘enterprise security’ perplex me. Take as an example this story related to me by a friend of mine the other day.

Every month the payroll department of the organisation my friend works for send them their pay slip via email. The payslip is a password protected PDF file. All good so far, right? Password protecting a file containing private and confidential information is a good thing.

Following the first email a second is sent by the payroll department.This second email contains the password required to open the PDF file. Both of these emails are sent in plain text. Additionally the password used to ‘secure’ the PDF file never changes.

I can see the chain of decisions that culminates in this process:

  1. The decision is made to send the payslip as a PDF file
  2. A payslip contains personal and confidential information, so it should be password protected
  3. To ensure the recipient can open the PDF file, the password needs to be shared with them
  4. Sending the PDF file and the password in the same email is not ‘secure’
  5. The conclusion is to send the password in a separate email
  6. To minimise the inconvenience to all parties the password should stay the same

The issue that I have with this process is that it isn’t very secure. In fact I would argue that it is not secure at all, and that it only provides the illusion of security to those who fundamentally don’t understand how email works.

The intention is to ensure that if the email with the PDF file is intercepted by a ‘bad actor’, they don’t get the password at the same time. However if one of the mail servers or networks that the email goes through is compromised, to the extent that bad actors can access an email, what is stopping them from intercepting the second email? The answer is nothing.

Additionally even if the second email is not intercepted, the password is always the same. Meaning that the bad actor only needs to intercept one of the emails with the password once, in order to gain access to the private and confidential data contained in the payslip.

This is an example of security theatre, designed to provide the illusion of security where none actually exists. As the emails are sent in plain text, there is no improved security here at all. Merely a small increase in annoyance for the recipients who need to use a password to access their payslip.

In short, this is not even a dubious example of security through obscurity. It is an example of the decision making process that occurs in enterprises. As it is a process which provides an illusion of increased security, with minimum impact on those following the process.

I wonder what it would take for this organisation to change this process?