Usability Interviews for Makewav.es

This set of interviews was intended as a health check for a school social networking site. We were aware that users found certain aspects of the site difficult to learn/remember and that there were certain bugs needing attention due to the age of the site. The interviews set out to identify the primary uses and priorities of the current users in order to help establish a roadmap for upgrading the software. The results of the interviews were surprising and suggested that the business might be taken in new directions.

Image 1: Before the Upgrade

The Users

Four users that were well known to community management staff were selected. Two were teachers (teacher 1 and teacher 2), one was a teacher trainer working for a consultancy business and the fourth was an informal educator and director of a third sector organization. These users covered the range of different types of educators using the site.

As committed users of the software, these interviewees could be expected to show a certain degree of favoritism for the site but their responses would reflect their familiarity with the site. Also, keeping the sample small allowed us to test what sort of responses we would get before committing to using resources for either additional interviews or other forms of testing.

The Questions

The questions were divided into four groups with different aims: to establish the context and extent of use by the interviewee and their wider organization, to document the interviewees’ specific uses of the software and its limitations, to determine which features were most useful and if any new features would help, and, finally, to get a sense of the user’s overall perception of the software and its contribution to their work.

The majority of questions were open ended and the interviewer only directed answers in order to ensure understanding. The term ‘features,’ for example means something quite specific to those who work in the tech industry but this term wasn’t always immediately clear to interviewees.

Results

Extent and Context of Use

Teacher 1 was personally using the software with about 20 students and teacher 2 reported that 10 members of staff were using the software with 240 students, which attests to the potentially drastic difference in use/take up amongst users. Both teachers reported that they were using the software to share work between students and with parents. The trainer, whose company worked with 40 schools, reported that the teachers she worked with were using the social media and sharing aspects of the software but that they were not using the Mozilla Open Badges activities on the site (one of the primary features of the software). Teacher 1 said she was using Open Badges and teacher 2 was not. The informal educator used the site to determine what aspects of science children were interested in learning and as a tool for his informal education delivery via Open Badges.

Specific Uses of the Software by Interviewees

Teacher 1 was using a set of Open Badges titled Digital Leaders with all of her students. The primary limitation she found was that there were not very many secondary schools using the software and as a result her students did not have a community to interact with in terms of sharing work or using the social media features. In addition, she also felt that more badges aimed at secondary school students would help, especially if they were of a better quality and more challenging for the students.

Teacher 2 used the software almost entirely for its sharing features, both within the school and with parents. She hoped to see a further reach in terms of communication amongst schools worldwide.

The trainer introduced the software to schools in training sessions and, as noted above, reported that schools were using the sharing and social media aspects of the software but not the Open Badges. The trainer specifically asked that the business not focus on Open Badges at the expense of other well loved features.

The informal educator, as also noted above, was using the software for research into children’s interests in science as well as for activities to use in his informal teaching. However, he also noted that he had hoped to find communities of teachers interacting with each other and discussing the Open Badges or other items they might be using on the software. In general, he felt badge uptake should be better and that the community should be more active.

Features

According to the interviews, the most commonly used features were reviewing and commenting on work, sharing work, pushing badges to take to students and awarding badges. This is the perception of the interviewees, however. More accurate data will be available through analytics or further testing.

In terms of new features that might be added, the informal educator requested additional options for creating quizzes or questionnaires. At the moment there are only three options for assessment: uploading text, images or video. The trainer suggested that students could have a theme builder like the avatar builder feature, which she said was very much enjoyed by students.

Overall Perception

Each of the interviewees expressed a great deal of enthusiasm for and appreciation of the software. These interviewees were chosen because they are regular users and therefore may be somewhat biased. However, we can still assume that users feel an emotional attachment to the site or are at least very invested in using it and continuing to use it.

Each of the interviewees also was able to name a list of different usability problems, including confusing terminology, confusing path to awarding badges, slowness of the system and a lack of provision for students forgetting passwords. The overall perception of ease of use of the software was not positive.

Recommendations

This site clearly needed a lot of usability work and there was more to understand in terms of the needs of users. The following sets of recommendations would take the site and the business through a step by step process of improvement.

Design and Development

Use analytics to establish which features are most likely to be used (eg, uploading and sharing, Open Badges, social features like commenting). On the basis of these results, choose which features to prioritize because they are most likely to be used and which features could be prioritized in line with plans for future business development.

Create a new information architecture for the site.

Create new user journeys for the site with a specific focus on currently problematic features such as the awarding of badges.

Use a card sort to test the new IA.

Use wireframe prototypes to test the new user journeys.

Full redesign of the site.

Marketing and Business Development

Preliminary research into whether expansion or promotion of newer features (eg, Open Badges, quiz functionality) would take the business in a profitable direction, including a survey and competitor analysis.

Use analytics data to establish primary user groups and create personas.

Use personas to develop a marketing and content plan to promote current and new features.

Image 2: After the Upgrade