Planning and delivering an Intern-led UX Library Project (Embedding ethnography Part 3)

This post originally appeared on the Lib-Innovation blog.

Last time out, as part of the Embedding Ethnography series, Emma Gray wrote about what it was like to be a UX Intern here at York, and the techniques she employed while she worked with us. Next time I'll write about what the study discovered.

If it's in any way possible to get an intern to help out with your ethnographic project I'd highly recommend it, so this post is about our process for setting the project up and working with Emma.

Here's a summary of the project:

Recruiting an intern

This was the part of the whole project we struggled most with, and were fortunate in how it worked out.

Five of our staff went to the first UXLibs Conference in 2015, and came back wanting to immediately implement some of the things we'd learned. But we all had not nearly enough day-to-day time in our roles to do any serious amount of ethnographic observation and interaction. So I submitted a proposal to a University-wide Intern scheme - but despite making it attractive as a I could, all the applicants chose to go for other Internships on offer from the University. If anyone has any tips on writing a great UX Intern job spec and advert, I'd love to hear them in a comment below...

We then got an email from the Head of HR in the Library saying a student at Durham University who lived locally wanted to work for the library over the summer, and did anyone have any suitable work? Naturally I jumped at this and sent Emma the existing job spec, she agreed it looked interesting, and she came in for an interview.

Emma Gray talks to a colleague

It was a very informal interview, just me and my manager and Emma without a huge list of pre-prepared questions. Emma didn't have any UX knowledge prior to coming in, but that didn't matter. As it happened she did have experience of working in a public library but that wasn't essential either. For us, the essential qualities were to show some initiative (Emma ticked this box, having found my website and read my reviews of the UXLibs conference...) and above all to be a good communicator. UX work involves a LOT of dialogue with users, so if that isn't something you enjoy it's going to be a slog... Emma was naturally communicatory so we had no doubts about offering her the role. As it turned out she was much more brilliant than we could have anticipated.

Pre-arrival set up

As Emma's manager I set about doing several things before she started at the Library:

1) Putting together a resource list on UX in Libraries to get her up to speed with an area she was unfamiliar with - I made that publicly available here

2) Putting together a document that outlined the aims of the internship so Emma would know exactly what she was working towards - I've put this on Google Drive here for anyone interested. I've not edited this from the original so there's some York-centric language - also I said 'emoji' when I meant 'emoticons' so you'll have to forgive me. Here's a preview:

Part of the Aims of the Internship document I put together for Emma

(It's worth noting that we didn't achieve some of the aims - for example visiting Cambridge and Sheffield Hallam, or trying out group interviews.) 

Essentially the thing that made this project different to future UX projects we'd undertake is this one was at least partly about understanding UX processes as well as our actual users - so Emma was tasked with setting up a UX Toolkit for our future use 

3) Sort out all the admin stuff associated with a new member of staff - entry card, username and password, where Emma would sit, PC she'd use, access to Google Drive folders etc etc 

4) Put together a timetable for the first week or so of her employment, after which she would become more self-directed. This included inviting Emma to a number of meetings and a couple of teaching sessions, so she could go away with a more rounded impression of what life in an academic library, and particular in the Academic Liaison Team, was like. We wanted it to be as rewarding and CV-enhancing as possible for her, as well as focusing on our project.

All of this took AGES. Any work you can put in beforehand is worth it though, otherwise it quickly takes up most of your job generating work and things to do for the intern. (This is something I've heard echoed across other sectors too.) 

[Feel free to re-purpose the reading list or the aims document if they help at your own organisation.]

Planning the project

As mentioned part of the aim was to build a UX toolkit - a suite of information and resources to call upon for future projects. As such as we decided Emma would use, as far as possible, all four of the interactive ethnographic techniques we'd learned (cognitive mapping, unstructured interviews, touchtone tours, love/break-up letters) with each participant, as well as doing behavioural mapping. My explanations of how to do these are in the 'Aims of the Internship' document, or see Emma's own post her description of each of these. 

This meant that a) Emma could start on the behavioural mapping and general observation while we recruited participants, and b) we'd need at least an hour of each participant's time. This would in turn mean a large amount of time spend interpreting and analysing the results; as a rule of thumb UX work takes 4 hours of analysis and reporting for every 1 hour of ethnographic fieldwork - a 4:1 ratio. 

The UX Team (the five conference attendees) met to discuss what sort of thing we should focus on in the project - I found this tricky because you want to provide a framework and guidance for an intern, but also part of the spirit of UX is to let the data tell you the story and not to go in with preconceptions to, or even seeking specific answers to questions. In the end we settled on using the project to better understand Postgraduate students simply because, during the summer holidays as this was, there were many more of them around. There were various things we hoped to learn - or various aspects we hoped to learn about - but we didn't put these into the project documentation or ask Emma to focus on them (or even tell her about them); we wanted the process to be as neutral as possible. 

We agreed that the five of us would meet during Emma's 6 weeks with us to discuss progress, look at the results, steer the further direction and so on.

During the project

Once Emma arrived and worked her way through the reading list, we started with observation and behavioural mapping. Observation is a great way for an intern to settle in because it's a relatively low pressure environment - it's a break from ingesting huge chunks of written information and a chance to be in your own head-space, and actually DOING ethnography where the stakes are much lower if you're not familiar with it yet. Not being sure about how to label a map of someone's path through the lobby is less intimidating than not being sure how to ask someone to write love-letter to a library service! 

The biggest problem we had was recruitment. We put requests for participants on social media, e.g.

If you're a postgrad, join us for cake & help us to design better services! Pls RT @GSAYork @WentworthColl @UniOfYork pic.twitter.com/W1AdP6XH4L

— Uni of York Library (@UoYLibrary) August 5, 2015

.. and we put similar info on a giant whiteboard in the Postgraduate Lounge area. We also approached people face to face and left them with info about the project and Emma's contact details. All in all these approaches yielded just three participants.

So all the Academic Liaison Librarians emailed their PostGrad cohorts via Departmental administrators: this was much more successful and yielded lots of emails to Emma, most of whom went on to book appointments with her. 23 people in total were recruited this way. The students were a mixture of PGTs and PGRs, from a variety of Departments and a variety of nationalities.

As it happened this project would conform to the 4:1 analysis to field work ratio almost EXACTLY: Emma was with us for 125 hours in total, and engaged with 26 participants in that time for around an hour each, spending the other 99 hours doing everything else: analysing, interpreting, transcribing, and writing up (and getting to grips with UX in the first place). It must be said that Emma was an incredibly proficient transcriber, having done this kind of work before: for mere mortals (me, for instance) the 4:1 ratio would not be remotely possible with transcription included, and in fact transcription itself often comes with a 4:1 ratio of its own, before you even get as far as analysis.

In general we consider ourselves incredibly lucky to have got Emma as our first ever UX intern: she was extremely bright and showed a great deal of initiative and confidence, as well as working extremely hard. She produced a brilliant report detailing her experiences across the 26 participants, with the findings clustered around the areas of: study space, noise levels, the catalogue, the library building, and facilities. We learned more about those 26 students than we'd ever learned about any students before.

Working with an intern is a brilliant way to free up enough time to actually start the process of UX and ethnography, although it still takes existing staff time to manage the project.

Michelle Blake is going to blog about the results of this and the next UX project we did; I'll add a link here when this is online.

The next post on here in this series will be another guest slot from an Intern, Oliver Ramirez, so undertook our second UX project at York.

A UX Intern writes... Emma Gray on ethnography

This is the second post in a series about Embedding Ethnography at York. The introductory post is here; the next post will be on the same project as this one, but written by me and focusing on the logistics of organising the whole thing, rather than undertaking the ethnography.

This is Part 2, written by Emma Gray, who was our first UX Intern. At the time of her work with us she was a 2nd year undergraduate at another institution. She did an absolutely brilliant job here, and we learned a lot through her work - about our students but also about ethnography itself. Here's her take on the project. It's a great intro to what we think of at York as the Big Five ethnographic techniques in libraries. There are plenty more, but so far we've focused on these ones, in various combinations, across all our UX work.

---

UX TECHNIQUES IN A LIBRARY SETTING – INTERNSHIP REFLECTION

In August 2015, I began an intern project looking into how several ethnographic UX techniques can be applied to a library setting and used to investigate possible improvements of the library user’s experience in the University of York library. As the project took place outside of term time, there was a focus on the experience of postgraduate students, who were still using the library over the summer break. The techniques covered over the internship include cognitive mapping, love/break-up letters, interviews, behavioural observation and touchstone tours.

BEHAVIOURAL OBSERVATION

Behavioural mapping is the first technique that I tried out at the beginning of the project. Being unfamiliar with the library before the project began, it was a valuable experience to observe how students behaviour in different spaces in the library. Firstly, I made a grid to record observations over the six week period in different locations within the library, increasingly concentrating on the more busy areas. Using the AEIOU framework, I recorded action (how the students are working in the space), environments (noting atmosphere, for example noise levels), objects (which services are used, e.g. technology and printed resources) and finally users (it is useful to note who is using the space, be it students, staff or external users). Secondly, it was also useful to record pathway maps using coloured diagrams of how individual students move in the space. In the York University library, it was particularly relevant to record pathways in the entrance foyer because the result show which building of the library students head when they enter.

An example of one of Emma's pathway maps

COGNITIVE MAPS

Participants were given six minutes to draw a map of the library, and I asked participants to change the colour of their pen every two minutes so that it could be easily remembered in which order things were drawn. It was a good starting activity because it allowed students to think about how they use the library and its services as whole. This visual representation of the spaces seems to also get the students thinking about how they use the library because I asked them more detailed questions during the interview.

The idea is that students would instinctively begin by drawing areas of the library they area most familiar with. However, I found that most participants started drawing the entrance in detail, maybe because the layout of the library as a whole is quite complex. I therefore encouraged participants to begin by drawing the area of the library they are most familiar with and use the most. This gave stronger results because it showed which objects and areas are most used.

A Cognitive Map drawn by one of the participants of the study.

In order to process the results of these maps, I divided the library into each floor of each building, and for each noted number of occurrences on the maps, its identification index which reflects the percentage of time is occurs (number of occurrences /number of participants), its representative index (number of occurrences / times category is drawn), and its temporal index (3 points for first pen colour, 2 points for the second and 1 point for the third). This process was then repeated for each individual object that is drawn on the maps, which for example included desks, computer areas and other library services. The advantage of this technique is that is gives a visual representation of how the students view the library, and it can also yield quantitative data. However, it is also a very time consuming technique, so the practicality of the data very much depends on the size of the library.

A screengrab of part of Emma's coding of Cognitive Maps

INTERVIEWS

I began the interview by asking the participants to describe and elaborate on their cognitive maps. Many participants elaborated on how they use certain spaces, which provided useful context for the cognitive map. I also asked participants to explain the process of how they use library services when working through an assignment. I asked general questions about the students’ experience with various services that the library provides, and encouraged to give their opinion on how their experience could be improved.

I found that being an intern and a student myself was invaluable to the interview process because the students did not shy away from being honest about their experience and more comfortable being critical than they might have been with a member of staff. Following the advice from Bernard’s ‘Research Methods in Anthropology: Qualitative and Quantitative Approaches’, I occasionally paused after a short answer was given, and often the participant would continue talking in more detail after a few moments, which kept the conversation flowing.

The major advantage of interviews is the breadth of information they yield, especially because things that individuals bring up can be elaborated upon and questions can also be adapted for future participants (e.g. finding out whether students had similar experiences with aspects of the library mentioned by other participants). Having a few standard open-ended questions is key, but participants seem to much more at ease if the interview flows like a conversation, rather than reading off a list of questions.

The only downside is that transcribing long interviews (they usually lasted between ten and fifteen minutes) takes a long time, but is definitely more valuable than making notes at the time because you are free to focus on what the participant is saying.

LOVE / BREAK-UP LETTER

Participants were asked to write either a love or break-up letter to a specific service that the library offers, so that the students can more emotively express how they feel about this service. I encouraged students to write about they feel (either negatively or positively) passionate about.

The participants were given a maximum of ten minutes to complete their letter. I definitely found this technique the most interesting because many participants wrote very imaginative, emotive letters that gave real insight into how library services and staff are perceived, and how this affects their daily university life. I did not find that this technique had any real disadvantages, only that some students preferred to write a wish list on what could be changed in the library. This also gave insightful information, but often did not show the reasoning behind these requests.  

An example of a usefully insightful Break-up Letter... Participants write these by hand but Emma wrote them up for ease of searching / reading. 

And for balance, here's a Love Letter...

TOUCHSTONE TOURS

For touchstone tours, I asked participants to give me a guide of the library. I voice recorded the tours to be transcribed later, so that every detail of the tour could be recorded.

Before the tour began, I usually encouraged participants to give as much detail as they could about how they use the services and different spaces, and also to point out their favourite space in the library. During the tour, I tried not to interrupt the participants or ask any more questions so that they could speak freely. Occasionally, I asked questions after the tour was over if any clarification was needed, or to enquire why the tour hadn’t included a certain area of the library.

The touchstone tours were interesting because the participants were very free to take me wherever they want, and point out anything they want, meaning that the participants brought some things up which I hadn’t thought to mention in the interview. It is notable that the quality of the tour heavily depended on how comfortable the participants were speaking out loud and narrative as we went along.

CONCLUSION

All of the techniques describe above have their own advantages and disadvantages, but they also each contributed to a more multi-faceted insight into how each participant engages with library services. I found the cognitive mapping the most interesting technique because I think it was the most unfamiliar to the participants and therefore got them thinking about how they use the library in a new way. As was the case in this particular library, it also highlights any areas that are being underused. Conclusively, having an intern collect UX data for a university library is a great idea, because the students are definitely more comfortable making their feelings known than they would be with a staff member.