Description

Co-interpretation of usage data: A mixed methods approach to evaluation of online environments

RICHARD PROCTER

Centre for Applied Research into Educational Technologies, University of Cambridge, Cambridge, UK

PATRICK CARMICHAEL

Centre for Applied Research into Educational Technologies, University of Cambridge, Cambridge, UK

VITO LATERZA

Centre for Applied Research into Educational Technologies, University of Cambridge, Cambridge, UK

ABSTRACT

This paper describes a novel research approach in which server usage data collected from online environments provides a ‘focus’ for semi-structured interviews in the course of which research participants are involved in a process of co-interpretation and ‘sensemaking’. Online environments generate large volumes of data which are useful in determining trends in behaviour, broad patterns and sequences of events but tell us little about the working or learning environments in which online activities are located. We use our experience of researching the use of an online environment with groups of education researchers to demonstrate that it is possible to use these data in the construction of rich and illuminating qualitative accounts which help to ‘make sense’ of single events repeated sequences, activity-flows and long-term trends in online activities. Our findings suggest that using suitably reduced quantitative data as the focus of qualitative enquiry in this way allows greater understanding of working practices in complex environments and can contribute to evaluation, generation of user requirements and provision of improved support to users of online environments and infrastructures.

Keywords: sensemaking; co-interpretation; web usage data; virtual research environments; collaborative work; focused interviews