Spirit Machine

Follow @SpiritMachine

Ask Me
Archive / RSS

25-01-2013
A Simple OpenLayers App with Yeoman, Sinatra, MongoDB and Backbone - Part 2

Introduction

In the previous part we set up the solution structure. In this installment we’ll get our API using MongoDB to persist/retrieve data and begin visualising this data with OpenLayers.

Setting up MongoDB

There are several options available to us for using MongoDB. The easiest way to run locally is just to install via a package manager. The default Ubuntu repositories have MongoDB, but it’s a little behind the latest version. If you must have the latest, you can add the 10gen repository and install it from there. Just follow these instructions.

The easiest way, and the way we will use, is to use a cloud provider. A couple of well known options are MongoHQ and MongoLab. This example uses the latter.

MongoLab also have an great open source tool for managing servers and replica sets called Mongoctl. I have this on all my Linux installations. I won’t go into it here, but if you’re inclined, check it out on GitHub here.

Getting the API to Talk to MongoDB

In a previous post, I explored a method of creating dynamic Ruby models suitable for use with MongoDB, but this time we’re going to use one of the Ruby object document mappers (ODMs) - Mongoid. We already added this gem in part one.

First we need for Mongoid to be able to connect to a database. Just create a file call mongoid.yml in the solution root. The simplest configuration for connecting to a local database looks like the example below. If you go with a hosted service, you just have to replace the host, user and password as appropriate.

Naturally we need a model that represents a point of interest (POI) on the map. To keep it simple, I just created poi.rb in the solution root folder. Here’s what it looks like.

Next we need some routes for our API, so let’s set up a typical suite for handling CRUD operations:
- A GET method for retrieving a list of POIs.
- A GET method for retrieving a single POI by its ID.
- A POST method for creating a POI.
- A PUT method for updating a POI.
- A DELETE method for (surprise) deleting a POI.
As can be seen in our new API code below, Mongoid allows us to make short work of this.

Let’s give it a quick test using cURL.
```
  > curl -X POST -d 'poi={"name":"test","desc":"test feature","pos":[5,5]}' 
  -v http://localhost:4567/poi
  
  About to connect() to localhost port 4567 (#0)
    Trying 127.0.0.1... connected
  POST /poi HTTP/1.1
  User-Agent: curl/7.22.0 (i686-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 
    zlib/1.2.3.4 libidn/1.23 librtmp/2.3
  Host: localhost:4567
  Accept: */*
  Content-Length: 53
  Content-Type: application/x-www-form-urlencoded
  
  upload completely sent off: 53out of 53 bytes
  HTTP/1.1 201 Created
  Content-Type: application/json;charset=utf-8
  Location: /poi/50f719e18cbaec991c000001
  Content-Length: 0
  Connection: keep-alive
  Server: thin 1.5.0 codename Knife
  
  Connection #0 to host localhost left intact
  Closing connection #0
```
Sweet. We’ve got a 201 and location header telling us where the newly created resource can be found. Let’s retrieve it. Don’t worry about the -v (verbose) flag this time.
```
  > curl http://localhost:4567/poi/50f719e18cbaec991c000001
  
  {"_id":"50f719e18cbaec991c000001","desc":"test feature",
    "name":"test","pos":[5,5]}
```
And just because we’re thorough testers…
```
  > curl -X PUT -d 'poi={"_id":"50f719e18cbaec991c000001","desc":"updated",
    "name":"test","pos":[5,5]}' -i http://localhost:4567/poi

  HTTP/1.1 200 OK
  Content-Type: application/json;charset=utf-8
  Content-Length: 0
  Connection: keep-alive
  Server: thin 1.5.0 codename Knife

  > curl http://localhost:4567/poi/50f719e18cbaec991c000001
  
  {"_id":"50f719e18cbaec991c000001","desc":"updated","name":"test","pos":[5,5]}

  > curl -X DELETE http://localhost:4567/poi/50f719e18cbaec991c000001

  > curl http://localhost:4567/poi
  
  []
```
That’s it. Our API is functional and MongoDB is holding our data.

Talking to the API with Backbone.

First we need a Backbone model for our POIs. Let’s kick things off with Yeoman’s Backbone generator:
```
  > yeoman init backbone:model poi
```
This creates a model skeleton in app/scripts/models/poi-model.js. For brevity I’m just going to keep the default names that Yeoman creates. It’s important to set the idAttribute to the name of the MongoDB generated ID field so that Backbone can uniquely identify the model instances. Adding some defaults and a URL where instances of the model can be persisted make it look like this.

Next we need a collection. Collections are ordered sets of model instances. Via a collection we can fetch all of our POIs from the API, listen for certain events and (my favourite) use all the Underscore iterator methods on the set. We’ll see one of these in action shortly.
```
  > yeoman init backbone:collection poi
```
Yeoman creates this collection in app/scripts/collections/poi-collection.js. We then need to indicate the type of models in the collection and specify the URL where the collection can access data. This is the result.

This is all we need right now. In the third and final part, we’ll be asking more of Backbone.

Showing Data on the Map

We need to include these new scripts along with their dependencies - Backbone and Underscore, in index.html. It now looks like this:

The final piece of the puzzle for this part is main.js. What we need is:
- A module to encapsulate our mapping functionality including rendering POIs.
- Code to fetch all the POIs from the API via our Backbone collection.
In the code below, we first set up some namespacing (used in our Yeoman generated Backbone files). Then we declare a mapping module with an init method that creates a base layer and a vector layer for POIs with some simple default styling. This module also includes the addPois method that takes our collection and uses Underscore’s map method to transform the set into an array of OpenLayers features, which are added to the vector layer.

After the mapping module is instantiated, all we do is then fetch our collection and pass it to the mapping module for display.

Now we haven’t yet added any front end UI for creating/editing/deleting POIs - this will come in the next part. To test that our JavaScript is working let’s add a POI via cURL.
```
> curl -X POST -d 'poi={"name":"test","desc":"test feature","pos":[145, -37.8]}' 
    http://localhost:4567/poi
```
Now if we view index.html via the base route of our application we should see this POI rendered in the center of our map - something like this.

That wraps up this part. In the third and final part, we’ll add interactivity to our map and use Backbone views to edit our POI data.
28-12-2012
A Simple OpenLayers App with Yeoman, Sinatra, MongoDB and Backbone - Part 1

Introduction

This post is essentially a learners journey into a few new (at least for me) technologies. I’ve been wanting to try out Backbone and Yeoman for a while and I recently had to develop some OpenLayers functionality at my day job. The thought came to me combine these with Sinatra and MongoDB, to see if I could hack out some-real time display and peristence of locations on a map.

It became apparent that this would become a lengthy affair for a single post, so I decided to break it into parts. Part one deals with getting Sinatra and Yeoman to play nicely together, then getting Backbone and OpenLayers ready to use.

Setting up the Environment

I put this all together on my netbook, which runs Lubuntu 12.04.

Following these instructions, I was able to set up Yeoman. The instructions also include setting up Ruby via RVM, which we need for Sinatra; so it’s two birds with one stone if you don’t already have Ruby installed. Just make sure to read the comments - there are some corrections to instructions in the post.

Setting up the Sinatra Project

First let’s get the Sinatra side of things up and running.
```
  > mkdir ol-blog-post
  > cd ol-blog-post
  > touch app.rb config.ru Gemfile Procfile
```
Next we add some dependencies to Gemfile. For a web server, I normally use Unicorn, but to keep it simple I’m going with Thin this time. The mongoid gem will be utilised when we start talking with MongoDB in the future parts.

Now we create our Sinatra application in app.rb. We’ll test the base route in a moment to check that it works.

Our rackup file is config.ru, which should look like this. It just points at app.rb as our Sinatra application.

I like to use Foreman by default, because it allows us to start our application according to the processes defined in Procfile. Heroku utilises procfiles too, so it’s just a way of making deployment to the cloud one less step away. If you don’t already have it, install the gem via the following command.
```
  > gem install foreman
```
Procfile just has a single web process that starts serving the application via the rackup file. It will run on port 4567.

Now we install the dependencies from our gemfile and start the application using foreman, which detects and uses our procfile automatically.
```
  > bundle
  > foreman start
```
Now we can either navigate to localhost:4567 via a browser or cURL. If we see the greeting, we’re good to go.

Creating a Project with Yeoman

I haven’t seen much in the way of how one is supposed to use Yeoman alongside something like Sinatra, but the method below seems to work well enough. It could probably be more elegant though, so go ahead and comment if you’ve got ideas.

Yeoman provides inbuilt generators that can scaffold a backbone application automatically. However, we don’t need all of the assets that it creates by default; so let’s set up a vanilla application to start with. In the root folder of the Sinatra application, run this command and answer no to the prompts.
```
  > yeoman init
```
Now we can see that Yeoman has generated some files for us, notably the application content within a sub-folder called app. Note: I used version 0.9.5, which left out the bootstrap CSS link from index.html, so I added it manually.

Yeoman includes a build command that does things like image optimisation, CoffeeScript and SASS compilation, JavaScript and CSS minification etc. Check the help docs for more information. Let’s test the default behaviour by using this command.
```
  > yeoman build
```
This creates a folder called dist where the optimised, deployable files for the application go. This folder is configurable in Gruntfile.js, but we’re going to bend Sinatra to Yeoman instead of vice versa.

The Sinatra Public Folder

By default, Sinatra serves static files from a folder named public if it exists. This folder is configurable, but we don’t just want to point it at app where Yeoman generated our files - we wan’t to use the dist folder with our optimised content when we run in production.

So let’s change app.rb so that the default route shows index.html from the public folder, which is set based on the environment in which we’re running. It turns out to be quite simple.

If we restart the application and navigate to the base URL, the content of index.html appears as we expect. To test that the public folder is set to dist when running in production, change the environment from development to production in the procfile. Now if we navigate to the base URL and view the source, we should see that our JavaScript and CSS files are prefixed with what looks like randomly generated strings. These are the optimised files generated by Yeoman - it’s working like a charm.

Setting up OpenLayers

To use OpenLayers, all we do is add a script tag linking to this file and create a div for our map container. The file below has had some of the default yeoman-generated content removed for brevity.

This is what main.css looks like to begin with. Bootstrap has a style for img that has a max-width: 100% attribute. This causes the map tiles to display in distorted vertical strips. Our style below rectifies this.

The last thing to get us started is some code to initialise the map. I’m using Open Street Map for the base layer. I’ll tidy this up in the next part, but for now it’s enough to ensure that we have OpenLayers working correctly.

Installing Backbone

As stated above, we haven’t used the Backbone generator to scaffold our application, but Yeoman integrates with Bower, so we’ve got a package manager that can install and manage our dependencies. Let’s use it to install Backbone.
```
  > yeoman install backbone
```
packages installed via this method are placed in app/components. The components folder is deployed to the dist folder when we invoke the Yeoman build command.

In the next part we’ll set up our API to get data in and out of MongoDB, as well as writing the JavaScript that uses OpenLayers and Backbone to show spatial features and communicate with the API.
07-10-2012
Dynamic Models with Ruby and MongoDB

Introduction

Over the past twelve years, most of my development has involved statically typed languages and relational databases. Over the last year or so, among other things, I’ve delved into Ruby and MongoDB and as with many others, enjoyed the ease and speed with which these can be used to create applications. Furthermore, I’ve found the new perspectives invaluable to my development skills as a whole.

What I’m going to show here is a specific flexibility that Ruby (or any other dynamic language) and MongoDB can provide compared to the statically typed/relational DB environment.

The functionality I’ll demonstrate is already provided in Ruby ODMs for Mongo such as MongoMapper and Mongoid, so it’s not a revelation. This exercise is more a diary entry of my own “Aha!” moment.

The main disclaimer I’ll make is that I’m neither a Ruby or MongoDB expert, so go ahead and feel free to comment.

A Simple Ruby Model

So let’s say you’ve come straight over from cutting C# or Java and you decide you’re going to write some class that persists it’s data into MongoDB. Something like this might result.

You can see here, we’ve got some instance variables, typical CRUD operations and a factory method for locating Person instances. Some validations and safety condition checking would be required of course, but I’ll leave this out at the moment for clarity.

Basic usage examples might look something like this.

As a start, this is OK. It won’t give us too much trouble in a simple scenario. However, we’ve really just transplanted a statically typed paradigm and have foregone the great power available to us in this new dynamic environment. We are:
- Thinking tables (fixed list of columns) instead of documents.
- Thinking static types (fixed list of members) instead of dynamic types.
What if we wanted to store any old attributes with our person?

Well, we know straight away that as a document database, Mongo will let us store any key/value combinations in a record - it’s schema-less. We can also see from the example above that with the Ruby driver, we just pass and receive hashes (think C# dictionaries) to represent this data. What we need is flexibility in our Ruby model.

Hello Dynamic Language

The first thing that we need to be able to do is hydrate a class instance by creating instance variables from a hash. We wave goodbye to our statically typed mindset and rewrite our constructor like this.

Yes; that easy. This is one of the things I love about Ruby. I find that when I write (or copy) code that seems like it should work, most of the time it just does.

So now we can create a person by using a hash with any values, but of course our save method still only persists a fixed list of attributes. We need to be able to turn our instance variables back into a hash. This turns out to be a simple matter of introducing the following method.

Then our save method changes to use it thusly.

This is a (limited as we’ll see) working dynamic class. We can create a Person instance with any extra attributes we want. We can save it as a Mongo document and we can retrieve it from the database having all of the attributes as instance variables, whatever they might be.

Now let’s say we want to use this model in an API that exchanges JSON. We’ve made it (almost too) easy for ourselves by handling our data with hashes. We just need to add the two methods below for returning JSON and for creating a Person instance from JSON.

That’s it; we’re in business. If we knock up a Sinatra or Grape API, we can use this model to read from and persist to a MongoDB database.

Not So Fast

Although we can create, save and retrieve a Person instance with our cleverness, there’s (at least) one grave limitation to our model. If our usage requires direct access to new dynamic members that we’ve introduced, we’re out of luck.

The last line will throw a NoMethodError because we have no public accessor for the new instance variable that we created on-the-fly.

There are Ruby methods that enable us to dynamically add attribute accessors to a class, but we really just want to add them to individual instances. We can do this, but hold on to your hat.

Metaprogramming Magic

The technique below uses what’s known as a Metaclass. It comes courtesy of the enigmatic _why. To read more about the concept specifically, check out this post.

First the Metaclass. Don’t worry. It hurt my brain too at first.

And now our new (and final for this post) constructor.

For each of the hash members, in addition to dynamically creating and setting the value of an instance variable, we’re creating a get/set instance method. Note: they’re actually class methods on the class that is the Metaclass for our instance, but let’s not split hairs.

To prove this does what I claim, try it out.

And finally, our finished product. There’s more that we could do with this, such as drawing out the now-generic methods into a base class. Another thing might be validation that enforces a list of mandatory fields; but I’ll leave that as an exercise for the reader.

I think you’ll agree. It really does tie the room together.
08-09-2012
Customising MyLocationOverlay in Mono for Android

Introduction

Android has available, a very easy mechanism for visualising the user’s current location in Google Maps - MyLocationOverlay. With some extension, this Activity implementation can become a very versatile addition to your toolkit.

I’m not going to go over the process getting Google Maps running in your project - the online tutorials cover this perfectly well. What I will demonstrate is:
- Broadcasting the current location.
- Implementing a distance threshold.
- Customising the location icon.
Broadcasting the Current Location

Android has mechanisms for achieving this kind of functionality - Broadcasts and Broadcast Receivers, but there’s a bit of ceremony in setting those up. For the sake of simplicity in this example I’m going to demonstrate a nifty library called TinyMessenger, along with my favourite little IoC container, TinyIoC. They’re available via NuGet and are each added to your project via a single .cs file.

The first thing we need to do after including TinyIoC and TinyMessenger in our project is comment out line 19 in TinyIoC. This will include a preprocessor directive that causes TinyIoC to register an instance of TinyMessengerHub with the default container. After this we can just include a parameter implementing ITinyMessengerHub in our constructors and have the container resolve it for us. We’ll see this further down.

Next we need to define the message type that will be sent with location updates. TinyMessenger facilitates this nicely with generics - we’ll use Android’s Location type.

To finish our basic implementation, we inherit from MyLocationOverlay, ensuring that our constructor takes a parameter implementing ITinyMessengerHub. Then we override OnLocationChanged() to publish the location asynchronously via the hub.

Now we’re cooking. All we have to do in some other part of our application is subscribe to messages of the type we’ve defined and do something with them. A contrived listener might look something like this.

Distance Threshold

This is quite trivial, but I’ll include it for completeness. All we’re doing is setting a minimum value in meters that the location must change by in order to cause a location broadcast. Each time our threshold is exceeded, we store the location to compare any newly reported locations against. This way, we avoid sending and processing messages for trivial movements. This is how our overlay class should look now.

Using a Custom Location Icon

Using your own icon with MyLocationOverlay isn’t hard, but needs to be done a partcular way if you want to keep functionality like the compass and the icon shadow. The example here behaves gracefully if no custom image has been set for the overlay.

First of all, we need to add a property to our overlay for the image. This is going to be of type Drawable. Note that we have to set bounds on the object, so the map knows how to draw it relative to a map point - in this case, the centre/bottom of the image.

Next we have to override the Draw() method. This knocks out some functionality that would otherwise be handled for us in the default implementation, so we’ll have to take up the slack. The new method below draws using the default technique if no custom image has been provided by calling through to DrawMyLocation(). It also explicitly draws the compass if it is enabled.

So here we have the final version of our customised implementation of MyLocationOverlay.

Now we have an overlay that’s highly re-usable. It’s easily themed based on the particular application we want to use it with; and we can utilise the location being reported to our overlay in a myriad of ways without tight coupling or further dependencies.

Happy Hacking.

Spirit Machine

25-01-2013

A Simple OpenLayers App with Yeoman, Sinatra, MongoDB and Backbone - Part 2

Introduction

Setting up MongoDB

Getting the API to Talk to MongoDB

Talking to the API with Backbone.

Showing Data on the Map

28-12-2012

A Simple OpenLayers App with Yeoman, Sinatra, MongoDB and Backbone - Part 1

Introduction

Setting up the Environment

Setting up the Sinatra Project

Creating a Project with Yeoman

The Sinatra Public Folder

Setting up OpenLayers

Installing Backbone

07-10-2012

Dynamic Models with Ruby and MongoDB

Introduction

A Simple Ruby Model

Hello Dynamic Language

Not So Fast

Metaprogramming Magic

08-09-2012

Customising MyLocationOverlay in Mono for Android

Introduction

Broadcasting the Current Location

Distance Threshold

Using a Custom Location Icon