A Green Torrens Lake

On many mornings, my walk to work at Blackboard, formerly NetSpot, in North Adelaide is very pleasant. It is a nice way to start the day, and is around a 3km walk from the bus stop in the CBD where I get off the bus. The only mornings where it isn’t pleasant is if it is really hot, or really cold, or it is raining and I’ve forgotten my umbrella.

On this particular morning in late February the Torrens Lake, part of the river Torrens, was quite green. There were a significant numbers of algal blooms on this particular morning. It was so striking that I had to stop and take a few quick photos.

Algal Bloom in the Torrens Lake

Algal Bloom in the Torrens Lake. By: techxplorer

To get a sense of the scale of the bloom that I could see, I used the panorama function of my venerable iPhone 4S.

Panorama of the green Torrens Lake. By: techxplorer

Panorama of the green Torrens Lake. By: techxplorer

No, I will not upload my address book. So stop asking!

"Privacy" by: _Bunn_

“Privacy” by: _Bunn_

Those people who follow me on Twitter would have seen the following conversation last night that I had with a member of the LinkedIn Customer Service team.

Now that I’ve succumbed to the lure of LinkedIn, I’ve been exploring the service in order to make the most of it. I think I should work to extract the most benefit out of sharing my personal and private information with a faceless corporation.

Screen capture of the twitter conversation.

Screen capture of the Twitter conversation.

One thing that I have noticed is that LinkedIn is very persistent in asking me to upload my address book. Either directly, or by giving it access to my Google Account. This is something that I am not prepared to do, and being asked to do it repeatedly is getting very tiresome.

There are two main reasons why I am not willing to upload my address book.

The first, I alluded to in my earlier post. My address book contains contact information on people who I have some form of relationship with. I consider the information that I have about others to be private, and I won’t share it with outside services. I simply don’t think it is ethical or moral to do so.

The second is that my address book as information in it about my doctor, and other personal information. I don’t want a faceless corporation mining that data for some sort of nefarious purpose. Yes, I know I’m being a little hyperbolic, but that’s simply how I feel.

What surprised me, as a software engineer always thinking about ways to make the software I write easier to use, is that it isn’t possible to disable this “feature”. As you’ll note in the above conversation, Kat says that “There’s no way to permanently opt-out of this feature”.

Upon reflection it isn’t that surprising. After all, the data about me is what is most important to social media services, not necessarily the experience I have as a user of their service. Especially if I’m not using their product, and I am the product that they’re selling.

I’ve succumbed to the lure of LinkedIn

"Linkedin Chocolates" by Nan Palmero

“Linkedin Chocolates” by Nan Palmero

Late last week I succumbed to the lure of LinkedIn and created an account. You can see my profile here.

In the past, I have been adamant that I wouldn’t have a LinkedIn account, and I’d limit my “social networking” to Twitter.  My Twitter profile is available here. So what changed.

Firstly, I don’t think any social network is inherently bad, nor are they inherently good. They’re tools to be used, just like the Internet is. The benefits of using them need to be weighed against the costs. For while having a LinkedIn account is free, that’s only an example of a monetary cost.

The main cost to me is privacy. By uploading my employment history into a third party service, over which I have no control, I’ve lost a bit of my privacy. Not only does LinkedIn share information with people in my network, I also makes select portions of it available to the public. As such it’s a piece of my private that  I’m never going to get back.

In the past, this cost was greater than any perceived benefit that I had. What has changed is that the benefit now outweighs the cost. It became apparent in the past week that I’ve reached a point in my career where a LinkedIn profile has become expected. If I want to continue to progress in my career, I need to have a LinkedIn profile.

That is to say, the lack of a profile to me was a greater cost than the cost to my privacy. So at this point creating a profile was a logical decision.

If I am asked wether someone should have a profile or not, my response has not changed. It depends on the individuals career goals, objectives, and the costs that they’re willing to incur. I still don’t think that having a LinkedIn profile, or any other social media profile for that matter, should be “required” and something that should entered into with critical thought.

Now that I have a profile, if I’ve worked with you in the past, feel free to connect with me. For no matter how many times LinkedIn prompts me, I am not willing to upload my email address book so that they can potentially find other people for me to connect with. This is because my address book contains personal information about other people, and I respect their privacy and won’t be sharing their details without their consent.

Saving the Page of Failed Behat Tests in Moodle

"Agapostemon splendens, f, head, anne arundel county, md_2014-07-09-13.37.57 ZS PMax" by  USGS Bee Inventory and Monitoring Lab

“Agapostemon splendens, f, head, anne arundel county, md_2014-07-09-13.37.57 ZS PMax” by USGS Bee Inventory and Monitoring Lab

I will be the first one to admit that I was skeptical when we introduced automated Behat tests for our Moodle codebase at NetSpot, now Blackboard. For those that may not be familiar with Behat, it is a tool that allows for behaviour driven development (BDD).

Conceptually BDD is all about testing the behaviours of software. It emulates the actions that a user undertakes when using an application, and ensures that the results of undertaking that action are expected.

In our case, when using Behat, it does this by interacting with a special Moodle instance, making requests for web pages, as if the Behat script was a real user. It then analyses the pages returned by Moodle to ensure they contain specified strings. More information about how Moodle is integrated with Behat is available on the Behat integration page in the Moodle Developer Documentation wiki.

One aspect of the integration that used to cause significant frustration for me, was that it was difficult to work out why a test failed. This is because it wasn’t possible to see the contents of the page that had resulted in the error.

Or at least, that is what I thought.

When configuring the Behat integration I used options like this:

The other day I stumbled across a fourth option which has proven really useful. Now I configure the Behat integration like this:

This fourth line configures some code that ships with the Moodle Behat integration that is triggered when a Behat test fails. What it does is save the content of the HTML page that was returned when the test failed. This is a vital piece of information to work out why a test failed.

I am now an advocate for BDD and have used the Behat integration extensively for a new activity that I’m developing for one of our clients. It’s funny how one small configuration option can have such a huge impact on the way in which you use, and think about, a tool.

Debugging using a magical exploding class

'marvin' by photoguyinmo

‘marvin’ by photoguyinmo

Last week I was debugging an issue that I had in Moodle related to the use of a global variable in PHP. If you’re not familiar with global variables in PHP, conceptually they are a variable that can be accessed from any part of your script. Typically through the use of the ‘global’ keyword.

The issue I had was that a global variable was having a string property set to false somewhere during the execution of the script. This was causing errors in a completely unrelated part of the codebase.

I discussed this with a colleague of mine. They showed me a debugging technique which uses what they called a ‘magical exploding class’.

The first part of the technique is to define a class like this:

Once the class is defined, it is possible to use it like this:

The two key parts to this technique is that:

  1. The instance of the magical class automatically returns its value as a string automatically. This ensures that that other parts of the code don’t realise that the field in the global class is set to our class, and not a simple string
  2. If the content of the field is replaced with something else, the magical class is destructed which causes an exception to be thrown. In this way the code that is changing the field can be identified

Using this technique I was able to identify the troublesome code, and write a defect resolution. More importantly the issue wouldn’t have been possible if the global objects that Moodle uses were immutable objects. But that’s a discussion for another time.

Techxplorer’s Utility Scripts – Version 2.0

'number two' by mark dyer

‘number two’ by mark dyer

If there are two constants in the life of a software engineer they would be:

  1. You’re always learning. For example, learning new paradigms, programming languages, frameworks or just a different way of doing something you do regularly
  2. Software is never finished, there is always something to do.

Both of these constants came together this weekend when I decided to work on rewriting the scripts available in the Techxplorer’s Utility Scripts.

I’ve started rewriting the scripts that I use the most, taking advantage of the things I’ve learnt over since August 2013 when I started the project.

The three scripts that I’ve re-written so far are:

  1. DbAssist.php – Automate various database related tasks
  2. GerritPush.php – Make it easier to push changes to a Gerrit code review system
  3. GitFetchReset.php – Automate fetching changes from Git and reseting the current branch to the latest upstream version

Other scripts that I use regularly will be updated as time allows. All of the pre-existing scripts are still available in their own branch in the repository.

jQuery and Moodle. Just don’t do it.

'stop+think' by Caffeinatrix

‘stop+think’ by Caffeinatrix

Fair warning, this post is going to be a little bit of a rant.

My role at NetSpot / Blackboard is working as a Senior Software Engineer as the client development team for Enterprise Moodle. As part of my role I need to review plugins that our clients want to use as part of their Moodle instance.

The things we’re most in the lookout are potential issues that will have an adverse impact on our ability to provide our service to the high standard expected by our clients. An example is having very long timeouts on cURL requests, doing things in Moodle without appropriate considerations for security, or not using the standard Moodle libraries.

One thing I have noticed is that a number of plugins, and themes, use JavaScript that is built using the jQuery library. For reasons that I suspect are lost to history, and are not relevant to this discussion, Moodle HQ has settled on the YUI library.

Wether you agree with this decision or not, the fact remains that core Moodle uses YUI, and uses it extensively. Incorporating jQuery into your plugin or theme causes a number of issues for us and our clients.

One of the most severe are rendering issues of pages because the JavaScript code breaks in some way. Most often because loading both jQuery and YUI cause conflicts. This breaks the rendering of pages and takes us time to rectify.

The second issue is that by including jQuery the developer is, in my opinion, being disrespectful to the users of the plugin. This is because you’re forcing users to spend time and bandwidth downloading two JavaScript libraries.

In the event that, as a developer, you must use jQuery to achieve your goals, please adhere to the guidelines developed by Moodle HQ, which are available here. In particular don’t include your own copy of the jQuery library. It causes way too many issues for those of us who need to support users of your code.

If you feel strongly about the use of YUI, and want to contribute to the discussions of Moodle HQ about the future of YUI and Moodle, I encourage you to checkout out this Moodle tracker item, and this associated Moodle forum thread.

Here ends the rant.

Updated: January 26, 2015

A trusted colleague of mine has pointed out that discussions are underway at Moodle HQ on the possible switch to jQuery, and Require.js with possibly using Grunt as a build tool. More information is available in this Moodle forum thread, which I also linked to above. The most relevant forum post, so far, is available here.

This is particularly relevant as in August, 2014 it was announced that the YUI library is no longer actively maintained. You can read more about the decision here.

I believe that my original point still stands. As a developer, if you are writing JavaScript for Moodle, you should be using YUI and not jQuery unless there is a very serious and compelling reason to do so. As it causes complications for those of us that need to support users of your code, and is disrespectful to users of your code by forcing them to spend time and bandwidth on loading two JavaScript libraries.

This is because the decision on which JavaScript library Moodle will use in the future is yet to be decided. Until that decision has been made, and the version of Moodle that has made the switch reaches critical mass, the use of YUI is still going to be preferred.

Sending a password in a second email is not secure

'Broken Rusty Lock: Security (grunge)' by  Nick Carter

‘Broken Rusty Lock: Security (grunge)’ by Nick Carter

There are times when decisions made for reasons of ‘enterprise security’ perplex me. Take as an example this story related to me by a friend of mine the other day.

Every month the payroll department of the organisation my friend works for send them their pay slip via email. The payslip is a password protected PDF file. All good so far, right? Password protecting a file containing private and confidential information is a good thing.

Following the first email a second is sent by the payroll department.This second email contains the password required to open the PDF file. Both of these emails are sent in plain text. Additionally the password used to ‘secure’ the PDF file never changes.

I can see the chain of decisions that culminates in this process:

  1. The decision is made to send the payslip as a PDF file
  2. A payslip contains personal and confidential information, so it should be password protected
  3. To ensure the recipient can open the PDF file, the password needs to be shared with them
  4. Sending the PDF file and the password in the same email is not ‘secure’
  5. The conclusion is to send the password in a separate email
  6. To minimise the inconvenience to all parties the password should stay the same

The issue that I have with this process is that it isn’t very secure. In fact I would argue that it is not secure at all, and that it only provides the illusion of security to those who fundamentally don’t understand how email works.

The intention is to ensure that if the email with the PDF file is intercepted by a ‘bad actor’, they don’t get the password at the same time. However if one of the mail servers or networks that the email goes through is compromised, to the extent that bad actors can access an email, what is stopping them from intercepting the second email? The answer is nothing.

Additionally even if the second email is not intercepted, the password is always the same. Meaning that the bad actor only needs to intercept one of the emails with the password once, in order to gain access to the private and confidential data contained in the payslip.

This is an example of security theatre, designed to provide the illusion of security where none actually exists. As the emails are sent in plain text, there is no improved security here at all. Merely a small increase in annoyance for the recipients who need to use a password to access their payslip.

In short, this is not even a dubious example of security through obscurity. It is an example of the decision making process that occurs in enterprises. As it is a process which provides an illusion of increased security, with minimum impact on those following the process.

I wonder what it would take for this organisation to change this process?

Project Ara and the Paradox of Choice

’29/52 choice paralysis’ by Lauren Macdonald

’29/52 choice paralysis’ by Lauren Macdonald

Last week I was listening to one of my favourite podcasts, Daily Tech News Show (DTNS), when the topic of Project Ara from Google came up. The archived episode is available here. It was an interesting discussion, as they always are on DTNS, and I had a thought. I wondered what, if any, impact the paradox of choice would have on Project Ara.

For those not familiar with the paradox of choice, this is my understanding. It is ‘conventional wisdom’ that having more choice is good. Indeed, the more choices that you have the better. However this is not always the case. In particular:

  1. More choices increases anxiety about making the ‘best’ choice, by increasing the number of options and comparisons that you have to make. For example two choices mean that you have two comparisons to make in order to determine which one is ‘best’. If you have four choices you now have six comparisons to make and five choices means nine comparisons.
  2. More choices increases feelings of regret that you may not have made the ‘right’ or ‘best’ choice. The more choices you have, the greater the anxiety that the ‘best’ choice wasn’t made because one of the other options may have been better.

In short, the more choices that you have, the more anxious that you are during the decision making process, and the more regret you feel and after.

As a software engineer I try to keep this in mind when developing software that contains configuration screens. Particularly if there are a significant number of choices. Because the anxiety becomes so great that the user makes the easiest choice available to them. That is, leave the options at their defaults. A classic example related to Microsoft Word is available here. As such ensuring suitable defaults is paramount, but I digress.

I wondered if this type of thinking would apply to Project Ara. At its core, Project Ara is about giving consumers the ability to build their own mobile phone by selecting from a wide range of different modules and integrating them into a single frame. I can see that this type of paradigm means that a consumer will have many choices to make. Perhaps even too many choices.

If this is the case, I can’t help wondering if consumer satisfaction with Project Ara phones will suffer. Because of anxiety during the process of deciding on which modules to have, and anxiety afterward the purchasing decision has been made due to concern that a different combination may have been ‘better’.

I’m interested in mobile computing and will keep an eye on news of the project and see what the world makes of it.

Users are critical to an Enterprise Architecture

Late last year I gave a presentation to the Blackboard Service Delivery Manager (SDM) team in Adelaide about the Enterprise Moodle architecture. It was an adaptation of a session I’ve run twice now for final year computer science students at Flinders University for their Enterprise Systems topic. This was exciting for two reasons:

  1. I enjoy sharing information with colleagues and clients;
  2. It was the first opportunity that I’d had to share information in a cross team workshop.

I can’t share the full details of the presentation here. Below is a de-identified version of the final architecture slide to give you an idea of the complexity of the system that we discussed.

De-identified Enterprise Architecture overview  (Click for full size version)

De-identified Enterprise Architecture overview
(Click for full size version)

During the session I asked a question that I’ve also asked of the students, which is:

What is the most critical consideration when it comes to an n-tier architecture?

I always get a variety of answers including:

  • Money
  • Hardware
  • Software / programming language
  • Location of the datacenter

The answer that I’m looking for is that the users are the most critical consideration. This is because their requirements inform all of the considerations. Additionally users, and their requirements and needs, are the main reason for the system existing. Without users we may as well just go home and surf the interwebz.

It is easy to loose sight of this. In particular by those in technical roles who have little interaction with users. However I believe that users are critical to the overall success of the system. I’ve been involved in a number of enterprise system development and deployment projects. Those that struggled were the ones that users did not engage with, mainly because they didn’t see the system as meeting one of their needs.

Similarly their needs influence the architecture decisions. For example some users of an enterprise system, have a requirement that their data be stored in their country. It doesn’t matter if the system is housed in a state of the art datacenter, if it isn’t in their country they won’t use it.

A simpler example was a government department that made a significant investment in a Microsoft Excel based spreadsheet to collect data from the researchers at an institution where I worked. The spreadsheet was very complex. It included data validation and filtering, as well as restricting the information collected to that which they government department was interested in.

Unfortunately there were a significant number of researchers, who for a variety of reasons, didn’t have access to Microsoft Excel. Most notably those in computer science who preferred to use Mac OS X or a flavour of Linux. These users didn’t want, or simply couldn’t, use the spreadsheet. So they developed their own template and provided the information that they thought we needed. In some cases it was clear they had spent significant amounts of time collecting this data.

Unfortunately we then had to throw away most of that data as it didn’t fit within the specifications of what was required by the spreadsheet. In essence:

  • Key stakeholders didn’t engage in the process because the system, in this case the spreadsheet, didn’t meet their requirements
  • Users spent significant amounts of time collecting data that we couldn’t pass on
  • The government department missed out on data that could have been relevant
  • Users became disenfranchised with our efforts and didn’t engage in future activities

This is a long way of saying that in my personal opinion users are critical to the success of a system. With their needs and requirements informing decisions made at the strategic and architecture level, right through to bug fixing and communication.

Losing sight of this has adverse affects, from users losing trust in the system and those people working hard to support them, through to users becoming so disengaged with the system that they move elsewhere.

As someone working in a technical role, it can be useful to take a step back and try to see the problem from the users perspective.