Managing markers with the Same Coordinates in a Google Map

2010 February 5
by techxplorer

By: Sam UL

[update]

Updated: Monday, 8 Feb 2010.

Further testing over the weekend has revealed that 100 metres is too much inaccuracy and so the random changes to the coordinates has been decreased to 10 metres, simply by adding another 0 to the denominator. For example 1500 becomes 15000

[/update]

Yesterday I posted about the work I’m doing for the AusStage project and the development of map based interface to event and venue data. One of the issues with this data is that there can be multiple venue records in the data with the same geographic coordinates. Most often this is caused by a venue changing its name over time.

Google Earth has a way of dealing with placemarks with the same coordinates by grouping them together under one placemark which, when clicked, will expand the placemarks out around a central point. This type of functionality isn’t available in Google Maps.

To workaround the issue of markers with the same coordinate in the Google Map I introduced a small algorithm in my code. This algorithm detects when markers with the same coordinates and adds a randomly generated number to the coordinates. The number that is generated equates to at most 100 meters latitude and longitude. This is relatively easy to implement as coordinates stored in the AusStage database use the Decimal Degrees method of expressing coordinates which is also what Google Maps uses.

In the initial testing this has proven to be enough distance to separate the markers without introducing too much inaccuracy. For example in one test case one marker was placed in the middle of the building while the duplicate was placed on the map within the outer walls of the building.

The code that I use currently looks like this:

// get the coordinates
var hash = points[i].getAttribute("lat") + points[i].getAttribute("lng");
hash = hash.replace(".","").replace(",", "").replace("-","");

// check to see if we've seen this hash before
if(coords[hash] == null) {
  // get coordinate object
  var latlng = new GLatLng(parseFloat(points[i].getAttribute("lat")), parseFloat(points[i].getAttribute("lng")));

  // store an indicator that we've seen this point before
  coords[hash] = 1;
} else {

  // add some randomness to this point
  var lat = parseFloat(points[i].getAttribute("lat")) + (Math.random() -.5) / 1500;
  var lng = parseFloat(points[i].getAttribute("lng")) + (Math.random() -.5) / 1500;

  // get the coordinate object
  var latlng = new GLatLng(lat.toFixed(6), lng.toFixed(6));
}

This code snippet is replaces line 29 in the first code snippet that I posted yesterday.

The key lines are:

  • Lines 02 – 03 which get the coordinates and strip them down to a series of digits to use a key
  • Line 06 checks to see if the key has been used before
  • If the key hasn’t been used before lines 08 creates a GLatLng object using the coordinates as they are and line 11 uses the key to store a simple value in the coords object
  • If the key has been been used before lines 15 and 16 add the random amount to the coordinates separating this coordinate from the first coordinate by up to 100m on the map

This is the easiest and most efficient way of working around the issue.

The photo “Do Not Duplicate” was uploaded to Flickr by Sam UL and used under the terms of a Creative Commons License.

Creating dynamic InfoWindows on a Google Map with Clustered Markers

2010 February 4
by techxplorer

My next major task for the AusStage project is the development of a browse based interface for venue and event data. The basic idea is that on a single map all of the venues that we have coordinate information for would be listed. This allows a user the freedom to explore the map and visualise venue and event data in a new way.

Using the test database there are over 900 venues with coordinate information and this means that I needed to find some way to cluster markers to help speed up the performance of the map. Also a map with over 900 placemarks on it would be imposing to first time users. After some searching of the Interweb I decided on the use of the MarkerClusterer JavaScript library which is part of the GMaps Utility Library.

To help speed up the initial generation of the map I also decided to separate the venue information from the event information. This means that the InfoWindows, the little popups that are displayed when a user clicks on a placemark on the map, need to be generated dynamically using an AJAX based technique.

I also wanted to use the MapIconMaker library to style the placemarks to keep thelook and feel of the maps on the site consistent.

The code needed to achieve this is divided into two sections.

The first section builds the map and adds all of the placemarks to it. The MarkerClusterer library takes care of clustering the placemarks and the MapIconMaker library build the icon. The code looks like this:


// declare helper variables
var markerUrl = "url-to-venue-data";
var markerCluster;

// create a new map and centre it on australia
map = new GMap2(document.getElementById("map"));
map.setCenter(new GLatLng(-25.947028, 133.209639), 3);
map.setUIToDefault();

// get the marker data
GDownloadUrl(markerUrl, function(data) {

  // get an icon - in keeping with style
  var newIcon = MapIconMaker.createMarkerIcon({width: 32, height: 32, primaryColor: "#FFBC3F"});

  // store the created placemarks
  var markers = [];

  // get the xml data
  var xml = GXml.parse(data);

  // extract the markers from the xml
  var points = xml.documentElement.getElementsByTagName("marker");

  // build a group of markers
  for (var i = 0; i < points.length; i++) {

    // get the coordinates
    var latlng = new GLatLng(parseFloat(points[i].getAttribute("lat")), parseFloat(points[i].getAttribute("lng")));

    // create the marker
    var marker = new GMarker(latlng, {icon: newIcon});

    // build the url for the info
    var url = "url-to-event-data" + points[i].getAttribute("id");

    // get a function to respond to the click
    var fn = markerClick(url, latlng);

    // add an event listener to listen for the click on a marker
    GEvent.addListener(marker, "click", fn);

    // add the marker to the list
    markers.push(marker);
  }

  // add the markers to the map
  markerCluster = new MarkerClusterer(map, markers);
});

The key lines are:

  • Line 11: Where the venue data is downloaded using the GDownloadUrl function
  • Line 14: Where the icon style for the placemarks is defined
  • Lines 19 – 23: Where the XML data is processed
  • Lines 26 – 45: Where the XML is used to create a group of markers
  • Line 48: Where the MarkerClusterer library is used to manage the custering of the placemarks on the map

The key to achieving the dynamically loading InfoWindows is line 38. This line uses the markerClick function, the second part of the strategy, to construct another function which will respond to the event when a user clicks on a placemark.

The markerClick function looks like this:

// function to build a function to respond to the click on a marker
function markerClick(url, latlng) {
  return function() {
    // download the event information
    GDownloadUrl(url, function(html) {
      // open an info window with the information
      map.openInfoWindowHtml(latlng, html, {maxWidth:450, maxHeight:400, autoScroll:true});
    });
  }
}

This function again uses the GDownloadUrl function to download data, this time a HTML snippet that lists all events associated with a venue. The data that is retrieved is then used to populate an InfoWindow that is opened using the openInfoWindowHtml function associated with the main map object. By using the same GLatLng object as that used to construct the placemark the InfoWindow appears in the right spot, over the placemark that was clicked.

Dynamically retrieving the content that is used to populate the InfoWindow introduces a slight delay to the display of the InfoWindow as the HTML content must be displayed. A planned refinement of this technique is to display some sort of loading message to the user to keep them informed of what is happening.

A pre-alpha version of the map, when it is first generated looks like this:

Map with clustered markers.
(Click for larger version)

Clicking one of the clustered markers zoom in the map and expands the cluster of markers until such time as single makers are displayed. For example in the example below the map has been zoomed into Adelaide by clicking the clustered marker at this area.

Map with clustered markers - zoomed into Adelaide. (Click for larger version)

Map with clustered markers, zoomed in on Adelaide.
(Click for larger version)

One of the benefits of the markers created by the MarkerClusterer library is that the clusters are automatically numbered to show how many individual markers are contained in the cluster. Clicking an individual marker, there are four on this map, brings up an InfoWindow displaying information in a similar way to that which I’ve outlined before on this blog.

Creating a log file in Apache Tomcat 6

2010 February 3
by techxplorer

By: Tattooed JJ

The main issue I have with developing web based application with Java and Apache Tomcat is that there always seems to be a bazillion ways of doing something and none of them documented very well. Or at least not with documentation I can find on the Interweb.

Recently with my work for AusStage I needed to create a log that included some additional information that isn’t included in the standard access logs. From what I have been able to learn from the Interweb there are a number of different ways to do logging in a Tomcat based environment. One of the main sources of information I found was the Logging in Tomcat page in the Tomcat Documentation.

After a great deal of searching this is the technique that I’ve developed for logging in my application to date.

The first step is to have a file called logging.properties in the WEB-INF/classes directory of your web application. For me this meant putting it into the root of my source tree. The text inside the file is as follows:


handlers = org.apache.juli.FileHandler

############################################################
# Handler specific properties.
# Describes specific configuration info for Handlers.
############################################################

org.apache.juli.FileHandler.level = FINE
org.apache.juli.FileHandler.directory = ${catalina.base}/logs
org.apache.juli.FileHandler.prefix = file-name-prefix

This file is based on the second example on the Logging in Tomcat page in the Tomcat Documentation. Essentially it means to create a log file in the standard logs directory, as specified by ${catalina.base}/logs, and with the file prefix file-name-prefix. All of the remaining settings are left at their defaults.

To add an entry to the line I use a call to the log() method of the ServletContext object that is available to my servlet via the getServletContext() method of the ServletConfig object. The ServeltConfig object is passed into the init() method of my Servlet, an object which is a child of the HttpServlet class which is in turn a child of the GenericServlet and I store a reference to it for when I need it as a private variable.

The “Logs” photo was uploaded to Flickr by Tattooed JJ and used under the terms of a Creative Commons license.

AusStage and Maps – Development Update

2010 January 29
by techxplorer

A number of changes have been made to the AusStage mapping service in the two since my last update post. The service has been expanded to include the following features:

  • Trajectory information is now displayed in the map
  • A slider is included on the page to allow the user to choose a date range for the display of placemarks
  • Various refinements to the underlying system

The generation of data in the Keyhole Markup Language (KML) format has also undergone some tweaking to include both trajectory information and improved timespan elements. Further development work to include style information, such as coloured trajectory lines, is underway.

When the user generates a map it now looks like this when it first appears.

Map with Advanced Options.
(Click for larger version)

The advanced options section now gives the user two options. The first is to select a checkbox that will include trajectory information into the map. The second is a slider that is built using the jQuery UI Slider from a Select Element plugin. The slider allows the user to choose a date range for events that are of interest. When the user makes changes to these options they click the Reload Map button and the map is updated to reflect the changes that they’ve made.

Currently this requires a call back to the server for updated information, an item on my list of development tasks is to undertake this all through JavaScript so the additional call to the server for more data isn’t required.

When a map include trajectory information it looks something like the map below, which has also been zoomed in on the Adelaide region.

A sample map showing trajectory information.
(Click for larger version)

Feedback on these enhancements has been positive and we hope to start alpha testing soon with a larger group of people.

Continued use of VUE for Documentation

2010 January 29
tags:
by techxplorer

Earlier this month I posted about my explorations of the Visual Understanding Environment (VUE) from Tufts University to generate documentation for the services I’m developing for AusStage. Initial feedback on the diagrams has been very positive and the software is now used on two types of diagrams. The first outlines all of the components that are used in the development of a service. As an example, below is the most recent component diagram for the mapping service.

AuStage Mapping Service Beta - Component Overview
(Click for larger version)

As you can see the system has grown in complexity since the last diagram. It has gotten so complex that I broke the data flow information into a separate diagram. The most recent version of this diagram can be seen below:

AusStage Mapping Service Beta - Data Flow Diagram (Click for larger version)

AusStage Mapping Service Beta - Data Flow Diagram
(Click for larger version)

With the increased capabilities of the system the complexity of the flow of data through the system has also increased. A diagram such as this provides a clear overview of where data flows through the system.

Building a Colour Gradient in JavaScript

2010 January 29
by techxplorer

One of the aspects of the AusStage mapping project that I’m working on is the inclusion of what we are calling trajectory information. Essentially these are lines that link placemarks and provide an indicator of where an organisation has performed over time. In the current implementation the earliest line is pure red, and the latest line is pure yellow. To achieve this I needed to develop two functions that build a colour gradient. The first was for a total number of lines less than 255 and the other was for a total number of lines more than 255.

The first function is called getScaledColour and is outlined below:


function getScaledColour(index, maximum) {

  // determine starting colour components
  var redHex   = "FF";
  var greenHex = null;
  var blueHex  = "00";

  // define values for formula
  var startGreen = 0;
  var endGreen   = 255;

  // calculate the green value
  var greenVal = startGreen + ((endGreen - startGreen) * (index / (maximum -1)));

  // round the green value to an integer
  greenVal = Math.round(greenVal);

  // convert from decimal to hexadecimal
  greenHex = greenVal.toString(16);

  // pad the hexadecimal number if required
  if(greenHex.length < 2) {
    greenHex = "0" + greenHex;
  }

  // return the final colour
  return "#" + redHex + greenHex + blueHex;
}

There are probably a few too many brackets on line 13 but I prefer to be explicit about precedence in a formula like this. The function assumes that the colours are to be used as part of an array of values. Therefore index is a zero based integer and maximum is the number of steps between pure red and pure yellow. This works because the transition from red to yellow is possible by only incrementing the green component of the colour when using the RGB colour model.

The second method is useful when there are more than 255 lines to colour. Although I’ll be the first to admit it is a bit of a hack. The first part of the code is a function that generates an array of the 255 possible colours and looks like this:


function getColourGradient() {

  var redHex = "FF";
  var greenHex;
  var blueHex = "00";
  var colourGradient = new Array(256);

  for(i = 1; i <= 255; i++) {
    // get the yellow colour
    greenHex = i.toString(16);

    if(greenHex.length < 2) {
      greenHex = "0" + greenHex;
    }

    // add the new colour to the gradient
    colourGradient[i] = "#" + redHex + greenHex + blueHex;
  }

  return colourGradient;
}

To then use the array the following JavaScript is used


// use array if over 255 colours
index = Math.round((255 / trajectoryPoints.length) * i);

if(index == 0) {
  index = 1;
}

lineColour = colourGradient[index];

In the limited testing that we’ve done so far these functions work well. An example of map with trajectory lines is outlined below:

A sample map showing trajectory information.
(Click for larger version)

Implementing the AusStage Data Exchange API

2010 January 25
by techxplorer

By: Toban Black

One of my tasks last week was the implementation of an API that could be used to retrieve data from the AusStage service in real time in a variety of different formats. The API allows users retrieve event data related to organisations, contributors and events in the following formats:

  1. HTML – as an unordered list
  2. JavaScript Object Notation (JSON) – as an array of objects
  3. XML – as a group of event elements
  4. RSS – as a group of 10 item elements

The API is simply a URL with a number of attributes that are required and some that are optional. The attributes are:

  • type (required) – the type of search to conduct
  • id (required) – a comma seperated list of ids to use as search parameters
  • output (optional) – one of the four output types listed above defaults to HTML
  • limit (optional) – the number of records to return, defaults to 10
  • script (optional) – enclose the output in a JavaScript function definition

We wanted to make the API as easy to use as possible for those that wanted to include AusStage data in web pages and other systems such as blogs and wikis. To help with including data in a web page I had initially developed a small script that used the XMLHttpRequest object. Unfortunately this caused some difficulty due to the security model employed by Internet browsers.

The security model is called the Same Origin Policy and essentially means that a XMLHttpRequest object can only request data from the same server that the HTML document came from. I had hoped that by having a small JavaScript library hosted on our site, that a HTML page could include, would work under this model but this is not the case.

As I understand it this security model is being reviewed and there is a Cross-Origin Resource Sharing recommendation available from the W3C that outlines a way to implement the type of functionality that I had originally envision. Unfortunately the only browser that we tested with that supported this was Mozilla Firefox 3.5.x as outlined in the HTTP access control page on the Mozilla Developer Centre.

To provide the functionality that we required within the current security model I implemented a variant of the JSON with Padding technique. This technique involves the use of a <script> tag that calls the API and defines the script option. When the API is called in this way the request data is wrapped inside a JavaScript function definition named ausstage_data(). A website developer can then write some custom JavaScript that will call the function and then work with the data that is returned.

Initial testing has shown that the API works successfully for importing data into a wiki powered by MediaWiki using the External Data extension. It is anticipated that the RSS format will be most useful for those who are using a hosted service for a blog and can use a widget or similar technique to include an RSS feed into their sidebar.

The photo “Sharing” was uploaded to Flickr by Toban Black and used under the terms of a Creative Commons License.

Generating JSON in Java

2010 January 20
tags: ,
by techxplorer

By: qthomasbower

As part of my work on the AusStage website I’ve been asked to explore ways in which website authors could readily access data housed in the AusStage database for inclusion on their own websites. The easiest way to accomplish this, I believe, is to provide website authors with an AJAX based library that would allow them to access certain defined subsets of data. For example an organisation may wish to list all of the events that they’ve been involved in on their website and they could use this service to achieve that.

In a way the service can be seen as the first step in supporting the creation of mashups of AusStage data with potentially other data as well.

One of the data formats that the service outputs is the JavaScript Object Notation (JSON) format. The other formats are HTML, in the form of an unordered list, or in XML. With these three formats the majority of use cases can be supported.

The issue I faced was how to generate the JSON code for an array of items that held event details. A search of the Interweb showed that there are a number of libraries and the easiest one, in my opinion, to use was the JSON.simple library. To achieve my desired outcome I create a JSON array, and then populate it with a series of JSON objects.

The only potential problem with the library is that it, at least on my system, generates unchecked exceptions when adding items to its internal data structures. I believe this has something to do with the use classes within the Collections Framework that don’t explicitly declare that they are going to be used for storing string objects.

Regardless they seem to be harmless errors as the library is so far working brilliantly and providing me with exactly the functionality that I required.

The image “Compositing The Creative Commons” was uploaded to Flickr by qthomasbower and used under the terms of a Creative Commons License.

Documenting the AusStage Mapping Service

2010 January 15
tags:
by techxplorer

By: Kasaa

Documentation is a critical component of any development project and it is one which is often overlooked. This past week I’ve spent some time on exploring documentation options for the AusStage Mapping Service which I’m developing. I’ve focused primarily on the development aspects of the system so far.

The first aspect is to include appropriate comments in the Java code using the Javadoc syntax. The ant build script that I’m using, based on this sample one from the Apache Software Foundation, has provision for generating the documentation as one of its build targets. This documentation should be useful for other developers who would like to contribute to the project as it progresses.

The second aspect is to provide an overview of how the system is constructed and how it works. For this type of diagram I have used many different tools in the past but this time I’ve settled on the Visual Understanding Environment (VUE) from Tufts University. This software meets my three primary criteria:

  1. Cross platform and preferably open source.
    VUE is written in Java and has distribution packages for Windows, MacOS, and Linux. The source code is also available and it is open source.
  2. Be easy to use and support the development of complex diagrams.
    So far VUE has managed worked well with the diagrams I’ve created.
  3. Develop diagrams that don’t require specialised knowledge to understand.
    This third criteria meant that I couldn’t use some of the more specialised packages such as those that created diagrams in the Unified Modeling Language (UML) standard.

The main diagram that I’ve created with the package so far is one that provides an overview of the components of the system, as well as an overview of the way data flows through the system. I currently exploring the idea of breaking this up into two separate diagrams. Particularly as the system gets more complex.

The most recent diagram can be seen below:

AusStage Mapping Service - System Diagram. (Click image for larger version)

It will be interesting to see how the diagram evolves over time as the system is developed. I also plan on exploring some of the more advanced features of the VUE application to see how these types of diagrams can be enhanced.

As always I’ll post here as my explorations continue.

The photo “Another pile of accordion file folders” was uploaded to Flickr by Kasaa and used under the terms of a Creative Commons License.

AusStage and Maps – Development Update

2010 January 15
by techxplorer

This week I’ve continued working on integrating AusStage data with Google Maps and Google Earth.

In response to user feedback the information displayed in the information windows in a Google Map has been refined so that it takes up less space and follows a format similar to that of the AusStage website with which users are familiar. Also, events are now listed in reverse chronological order. The name of the event is now a persistent link into the AusStage website for that event.

The new format can be seen in the screen capture below:

Revised InfoWindow display.
(Click image for larger version)

Work has also started on an page that allows users to export data in the Keyhole Markup Language (KML) format with advanced options. The first of these options is to include TimeSpan elements in the KML. Using this element an application such as Google Earth can automatically generate a slider that allows the user to view events that occurred during a particular time frame.

This can be seen in the screen capture below where Google Earth is displaying events that occurred between 2002 and 2008.

Google Earth displaying a map with a timeline.
(Click image for larger version)

Experiments are underway to expand on this advanced export functionality to include LineString elements and elevation information. When LineString elements are included it is possible to link all events together which gives a sense of where an organisation has been over time. When combined with TimeSpan elements it is possible to view the places an organisation has visited over time by following the lines drawn between events.

We’re also exploring the use of elevation information to map time as a third dimension on the map. Which when combined with LineString elements can give another way of analysing where an organisation has toured over time.

Other changes to the system have been implemented which aren’t user facing. For example refactoring JavaScript includes to make it easier to reuse code, as well as changes to the underlying Java classes and servlets to clean up the code.

Different approaches to documenting the system have also been explored, including the development of system overview diagrams. This will be the topic of a future post.