Exploiting activity data in the academic environment

Which events are significant?

This guide is about compiling (in tabular form) a compendium of activity data to assist in planning what recommenders may be possible in a given institution or group of institutions that share activity data.

E H Carr famously explored the difference between ‘facts of the past’ and significant ‘historical facts’ (What is History? 1961), comparing cake burnings in general with royalty-related cake burnings. We are faced with a comparable challenge of judgement in determining what to select from the mass of data arising from the use of computer systems. Given an area of interest (such as student success or utilisation of subscribed resources), the problem of selecting significant facts or ‘events’ can be split in to two along the lines suggested by Carr:

• What exists? What event data are we collecting or what could we collect?
• What is significant? What event data is worth preserving?

Before we can address those questions, it may help to have a sense of the possibilities. Put simply, event data records any user action (online or in the physical world) that can be logged on a computer, ideally containing a reliable form of user identification. We can usefully think of it in three categories:

• Access – logs of user access to systems indicating where users have travelled (e.g. log in / log out, passing through routers and other network devices, premises access turnstiles)
• Attention – navigation of applications indicating where users have been are paying attention (e.g. page impressions, menu choices, searches)
• Activity – ‘real activity’, records of transactions which indicate strong interest and intent (e.g. purchases, event bookings, lecture attendance, book loans, downloads, ratings)

Given this definition, you may already be concluding that you do in fact have a lot of data in your systems, not all of which is likely to tell useful stories. The first step is therefore to determine the types of event data that may be of relevance. Perhaps start with a checklist along these lines:

Problem space – Analysis of learning resource usage by undergraduate students

It may be useful to undertake a more general exercise with your ‘team’. Use a similar table (e.g. the first six columns). Don't start with the problem to be addressed but rather work together to compile the list of every example of event data that exists, could or should exist. One you have the list, you can add extra ‘value’ columns for different analytical purposes – for example a VLE team might add columns for such as student success, local content utilisation, usage patterns by faculty, VLE performance optimisation. You can then do a second pass assessing the significance of each event for each purpose – perhaps initially in a scale of yes / possibly / no.

The LIDP project considered a range of library ‘events’ - http://library.hud.ac.uk/blogs/projects/lidp/about/ and http://www.slideshare.net/gregynog/bw-dave-pattern-lidp

The EVAD project investigated to range of event logging in the Sakai VLE - http://vledata.blogspot.com/

The JISC Business Intelligence projects have taken a wider sweep of what events might be useful to derive business intelligence - http://www.jisc.ac.uk/whatwedo/programmes/businessintelligence/

Educause reports a survey of the data being used for ‘academic analytics’ - http://www.educause.edu/ers0508