Analytics Expert Blog

This post has been sitting in our draft folder since April but with Apple and Google preparing for a mobile advertising battle we figured we would update it a bit and publish.

All watched over by machines of loving grace.

Several years ago Adam Greenfield wrote an article titled, “All watched over by machines of loving grace: Some ethical guidelines for user experience in ubiquitous-computing settings”. Adam provided some general principles for us to observe as designers and developers. We always keep these guidelines nearby. Here they are.

• Principle 0, Is, of course, first, do no harm.
• Principle 1. Default to harmlessness. Ubiquitous systems must default to a mode that ensures their users’ (physical, psychic and financial) safety.
• Principle 2. Be self-disclosing. Ubiquitous systems must contain provisions for immediate and transparent querying of their ownership, use, capabilities, etc., such that human beings encountering them are empowered to make informed decisions regarding exposure to same.
• Principle 3. Be conservative of face. Ubiquitous systems are always already social systems, and must contain provisions such that wherever possible they not unnecessarily embarrass, humiliate, or shame their users.
• Principle 4. Be conservative of time. Ubiquitous systems must not introduce undue complications into ordinary operations.
• Principle 5. Be deniable. Ubiquitous systems must offer users the ability to opt out, always and at any point.

What do these ethical guidelines have to do with Steve Jobs and Apple?

Along with Apple’s announcement of iPhone OS 4 in March the iPhone Developer Agreement was updated to restrict how device data is shared between applications and 3rd parties so to improve consumer privacy. Some have argued that Apple really doesn’t care about consumer privacy and that the intention of this update was to cripple competition to iAds, specifically Google. While that may be an outcome, the agreement changes do in our opinion represent an understandable step by Apple to watch over their carefully crafted user experience and coveted, insanely loyal consumer base.

Our mapping of the ethical guidelines above and the updated iPhone Developer Agreement.

Updated Section 3.3.9 of the iPhone Developer Agreement.

3.3.9 The following requirements apply to You and Your Application’s use, collection, processing, maintenance, uploading, syncing, storage, transmission, sharing and disclosure of User Data:

• All use of User Data collected or obtained through an Application must be limited to the same purpose as necessary to provide services or functionality for such Application. For example, the use of User Data collected on and used in a social networking Application could be used for the same purpose on the website version of that Application; however, the use of location-based User Data for enabling targeted advertising in an Application is prohibited unless targeted advertising is the purpose of such Application (e.g., a geo-location coupon application). - See Principle 4 above.
  
• You may only provide or disclose User Data to third parties as necessary for providing services or functionality for the Application that collected the User Data, and then only if You receive express user consent. For example, if Your Application would like to post a message from a user to a third party social networking site, then You may only share the message if the user has explicitly indicated an intention to share it by clicking or selecting a button or checking a box that clearly explains how the message will be shared. - See Principle 3 above.
  
• Notwithstanding anything else in this Agreement, Device Data may not be provided or disclosed to a third party without Apple’s prior written consent. Accordingly, the use of third party software in Your Application to collect and send Device Data to a third party for processing or analysis is expressly prohibited. - See Principle 1 above.
  
• You must provide information to users regarding Your use and/or Transmission of User Data and explain how Your Application will use User Data, e.g., by providing information in the App Store marketing text that accompanies Your Application on the App Store, by adding an About box within Your Application, or by adding a link to Your privacy policy on Your website. - See Principle 2 above.
  
• You and the Application must take appropriate steps to protect any User Data from unauthorized disclosure or access. If a user ceases to consent to Your use and/or Transmission of User Data, You must promptly cease all such use and/or Transmission and destroy any such information from Your records (except to the limited extent necessary for Your Application back- ups and record-keeping or as otherwise prohibited by law). - See Principle 5 above.

Note: PercentMobile provides mobile analytics for Desktop/Mobile Sites and Web Apps delivered via a phone’s Web Browser and have choosen not to be distracted by Mobile Application Tracking. The availability of hundreds of thousands of apps is great, but our interest is currently focused on helping people understand the mobile ecosystem and  behavior/actions of the mobile audience to billions of desktop and mobile Web pages.

The recent Economist contained an article on the expected rise in the mobile Internet usage in the so called BRICI countries (Brazil, Russia, India, China, Indonesia). 

http://www.economist.com/node/16944020

What is their current mobile Internet ecosystem?

Nokia holds the lion’s share of close to 50%, followed by Samsung, SonyEricsson and Blackberry with 13%, 8% and 7%. The rest splinters off into a lot of small segments. Apple surprisingly reaches already a 2.8% share. Android OS is virtually non existent with a mere 0.3%. What is interesting is that while close to 60% of the devices were purchased in the last 2 years only 15% are Smartphones or Experience Phones. This is en par with Europe, but lower then North America.

Unsurprisingly, WiFi usage is only 15% (North America and Europe reach 30%), this makes sense according to the Economist article  which states that broadband access is either not readily available everywhere or expensive. It is however very interesting to see that the rate of WiFi usage for devices that support WiFi is over 60%, in Europe and North America this ratio is 30%.

A few weeks ago, the PercentMobile team came to me to see if I could help write some new libraries for them. Contemporary web and mobile web sites are written on a vast array of different platforms… and obviously the more that PercentMobile supports, the better.

Something I love about programming is that there are so many languages to choose from – why restrict yourself to learning or becoming an expert at one when the same problems are also being solved in other, sometimes better, ways? Each language or platform has strengths and weaknesses of course. But I believe that understanding the differences – and more often, the similarities – makes one a better programmer.

So, generous polyglot that I am, I took the challenge, and plunged in.

Python & Django

I love Python – its readability, its libraries, and its overall philosophies. In a web server environment, Python can be used in a fairly raw way, behind a generic Web Service Gateway Interface (WSGI), but there are also a number of powerful web application frameworks using the language – most notably Django. PercentMobile want to provide painless Django support, but also to make sure that other Python server environments were not precluded.

percentmobile.py is the single file that provides everything you need.

If you are running code in a WSGI environment, you will have a handle to the WSGI environment. This is normally called environ by convention, and is passed in to your application via your top-level WSGI callable. To make the PercentMobile tracking code work, you need only make one call to the percentmobile.tracker_cookie_insert function. It returns two values: the cookie that you’ll need to set in the HTTP response headers, and the HTML that you should insert into your page.

Say, for example, you had a very simple WSGI app. This responds to requests with an HTTP 200 status code, a single header and some simple HTML:

def my_app(environ, start_response):
    status = '200 OK'
    response_headers = [('Content-type','text/html')]
    start_response(status, response_headers)
    return ["Hello world"]

To add PercentMobile tracking to this code, you need to firstly import the tracker library of course, and then call the tracker_cookie_insert function:

import percentmobile
cookie, insert = percentmobile.tracker_cookie_insert(environ, '1234555')

(Of course you should replace the final string with your own site ID!)

The cookie return value is actually a dictionary of the different parts needed to construct its string serialization. This will allow you to splice together multiple cookies, or alter the expiry time, path scope and so on. To convert the dictionary to a string, and to get the Set-Cookie header sent back in the request headers, the following code will suffice:

cookie = "%s=%s; expires=%s; path=%s" % (
    cookie['name'],
    cookie['value'],
    cookie['expires'],
    cookie['path']
)
response_headers = [('Content-type','text/html'), ('Set-Cookie', cookie)]

Finally, you need to make sure the HTML fragment, in the insert variable, gets placed in the HTML response:

return ["Hello world %s" % insert]

And that’s a wrap. The whole, PercentMobile-tracked WSGI application looks like this:

def my_app(environ, start_response):
    cookie, insert = percentmobile.tracker_cookie_insert(environ, '1234555')
    cookie = "%s=%s; expires=%s; path=%s" % (
        cookie['name'],
        cookie['value'],
        cookie['expires'],
        cookie['path']
    )
    status = '200 OK'
    response_headers = [('Content-type','text/html'), ('Set-Cookie', cookie)]
    start_response(status, response_headers)
    return ["Hello world %s" % insert]

Pretty easy, huh? Well, not as easy as tracking an app if you’re using the amazing Django framework! Contained in the same percentmobile.py file is a class that can be used as Django middleware. It uses the same underlying function as the WSGI implementation, but also takes care of the headers and insertion for you.

Assuming you’ve placed the library file in your Python or Django path, you simply need to add two lines of code in the settings file. Firstly add the percentmobile.PercentMobileDjangoMiddleware class to your list of middleware:

MIDDLEWARE_CLASSES = (
    ...
    'percentmobile.PercentMobileDjangoMiddleware'
)

And secondly, add your PercentMobile site ID to the settings file too:

PERCENTMOBILE_SITE_ID = '1234555'

Um… that’s it. The middleware will intercept the request, figure out the cookie that will need to be sent in the response, create the insertion code, and place it just before in the response. You’re golden. I love Django.

ASP.NET: C# & VB.NET

Switching over to another world altogether, let’s take a quick look at the ASP.NET implementation of the tracking code. One of the great things about .NET is that you can choose between all sorts of different languages to write your applications and pages in. I decided to write the tracking library in C#, but you can use it declaratively in your page code, or programmatically from C#, VB.NET, or any other supported language.

The library is implemented as a User Control – that is, as a .ascx file. You need to download and add PercentMobile.ascx to your web application project.

To embed the tracking logic into a page (or probably preferably, a master page), you simply register the user control as residing in that file:

And then embed the control straight into the .aspx file contents, wherever you want it to be:

So a simple, tracked .aspx file might look something like this:

Hello World

To be honest, this is so simple that I would expect most people to use the control declaratively. But if, for some reason, the control needs to be inserted programmatically, that’s pretty easy too. You need to use the Reference directive, instead of Register, at the top of the file, but otherwise it’s not much harder. Here, we’re using VB.NET to programmatically achieve exactly the same result as above:

Hello World

(It should be fairly straightforward to see how to do this in other .NET languages – PercentMobile includes some examples in their install instructions.)

Java

Onwards. Let’s take a look at the tracking code for Java. There are numerous ways in which Java can be used for web or application server environments. To keep things simple, I decided that supporting JSP was more or less the most familiar and reusable approach.

JSP doesn’t enjoy the rich page event model that I was able to use in ASP.NET to intercept headers, write cookies, and insert HTML, all with one include. is OK for adding the HTML snippets, but wouldn’t let me access the HTTP headers. does, but would only work if you weren’t planning to emit any HTML of your own after the tracking code – something of a radical assumption.

So I settled for an approach where an include directive creates an instance of an inner PercentMobile class defined within the JSP class. To cut a long story short, this means you merely add a reference to the library file at the top of the JSP file:

And then call the track method on that instance, somewhere within the page:

The included JSP file takes care of instantiating the percentMobile object and giving it references to the request and response stream. This means it can read and write cookies, and later emit HTML. Your tracked Hello World in Java then? It’s as simple as this:

    Hello World

node.js

OK, OK. Python, ASP.NET, Java. No big deal, right? That’s all so 2003 or so, right?

Well, there’s an alternative future for web server technologies. It’s one that’s blazingly fast, lightweight, event-driven, and… where you write your server logic in Javascript.

node.js is one of the most exciting things I’ve seen for a long time, and I know I’m not alone. Fresh, fashionable, and somewhat unproven, admittedly, it’s had a lot of gushing coverage. But since it turns the web server model (almost literally) inside out, I do believe there’s something important going on.

Would it be possible to write a web app with node.js and still have it tracked by PercentMobile? My challenge.

I decided to rely on Connect, a middleware framework for node.js, which, if you are writing web applications, provides a whole host of other helpful web logic. Using Connect to create a simple node.js app is extremely easy:

var Connect = require('connect');
Connect.createServer(
  function (req, res, next) {
    res.simpleBody(200,
      "Hello World" +
      "",
      {
        "Content-Type": "text/html"
      }
    );
  }
).listen(88);

To add PercentMobile tracking to this application firstly requires you to pull in the PercentMobile module:

var PercentMobile = require('./percentmobile');

(Where the percentmobile.js file has been placed in your node.js environment or in the common modules location.)

The module needs to be initialized as a piece of Connect middleware in the createServer function:

PercentMobile.init('1234555')

There are then two module functions, cookie and html, which both take a reference to the response object, and which return the cookie string and HTML to insert, respectively. These can be used in Connect’s simpleBody function, for example, meaning that our tracked application is as simple as this:

var Connect = require('connect');
var PercentMobile = require('./percentmobile');
Connect.createServer(
  PercentMobile.init('1234555'),
  function (req, res, next) {
    res.simpleBody(200,
      "Hello World" +
        PercentMobile.html(res) +
      "",
      {
        "Content-Type": "text/html",
        "Set-Cookie": PercentMobile.cookie(res)
      }
    );
  }
).listen(88);

That’s a wrap

So that’s it. A whistle-stop tour of the new languages and frameworks supported by PercentMobile. I’d love to hear your feedback on how easy (or hard!) these new APIs are to use.

And, oh… who will be the first to build a mobile web app on node.js?

James Pearce is the former CTO of dotMobi and Argogroup, and has evangelized, coded, written and spoken about the mobile web and mobile development for over a decade. Find him online at http://tripleodeon.com/about