Usability testing was recently conducted with 6 library users in MD Anderson Library.
A “user cafe” was set up in the MD Anderson Library lobby. Passers-by were enticed to complete the test with promises of free coffee and snacks after completion.
The test asked participants to complete 3 tasks using the new version of the main Libraries website.
In this post:
- Methods
- Tasks given
- Results
- Task results
- User mistakes
- Major observations
- Participants
- Why did they come to the library?
- Lessons learned
- Next steps
1. Methods
Usability testing means observing users completing tasks with a website (or anything else) to see how it is used.
By observing how users actually use the website (instead of how they say they use it), design changes can be made to enhance and improve usability.
Usability testing was recently conducted on a new Libraries website that is currently in development.
A “user cafe” was set up in the MD Anderson Library lobby. Passers-by were enticed to complete the test with promises of free coffee and snacks after completion.
Credit: This “cafe” protocol of user testing has been adapted from a presentation by Zoe Chao, UX Librarian at Penn State University Libraries.
This new method of collecting user research data is currently being piloted over the summer. More will be written about this “user cafe” method in the coming months.
Users were recruited by Mario Gonzalez, student employee in Library Technology Services. Mario also controlled passers-by to avoid interruptions to participant interviews.
Users filled out a brief survey before starting, seen above.
A recording was taken of the participant’s voice and screen during testing.
Tasks given
Users were asked to complete each task using the website without searching.
The following tasks were given to participants. Task wording is shortened for this blog post.
Task #1
Show me where on this website you’d find the times that the Architecture and Art Library is open on Saturday.
This is referred to as Hours task in the rest of this post.
Task #2
Show me where on this website you’d find the email address for the chemistry subject librarian.
This is referred to as Email task in the rest of this post.
Task #3
Show me where on this website you’d find the room your Photoshop course is in on Monday, June 26 at 11:30am
This is referred to as Training task in the rest of this post.
2. Results
A total of 6 participants completed all 3 tasks within a 1.5 hour period. Tests lasted from 7 to 13 minutes.
Task results
Participants were either successful, mistaken, or failed.
Success: Participant found the information asked.
Mistaken: Participant incorrectly thought they found the correct information.
Failure: Participant gave up looking for information.
The graph above shows the tasks results for all 3 tasks.
One third of participants incorrectly completed the Hours task, while only one participant incorrectly completed the Email task.
One third of participants gave up looking for the subject librarian email address and for information about training classes.
User mistakes
Users made specific mistakes on two different tasks:
Hours task
Two participants mistakenly thought the hours found the homepage of the site were the same hours for Saturday. The hours for the Architecture & Art Library are different on Saturday.
Users had to select the “Today’s Hours” link above the Hours table to get to the full Hours page and find the Saturday hours. One participant, who ultimately found the correct hours, noted that the “Today’s Hours” link is not obviously a link.
Email task
One participant thought the Accessibility Help form would be enough to get in contact with a subject librarian. Rather than continue to look for an email address — as the task asked for — they satisficed on the form used for accessibility requests.
Major observations
Low navigation use
Similar to previous usability tests, no participants used or noticed the “Contact Us” button at the very top of the navigation.
In general, use of website navigation was low. Most users went to homepage content for completing tasks.
Participants relied on the homepage links
Similar to previous usability tests, all participants started with homepage links, especially during Email and Training tasks.
Unfamiliarity with library terms
Several times, participants were confused on library terminology.
Examples include instruction, call numbers, and course reserves.
Two participants assumed that the Instruction page was the needed resource to contact a subject librarian. Those participants did eventually find the email for the correct subject librarian from the Subject Librarian Directory link found on the page.
One participant thought the Course Reserves page was related to training classes. This is likely due to confusion on the definition of course reserves.
One participant thought “call numbers” meant phone numbers. They then thought the Call Numbers & Location Guide page was a guide to library departments in the building, not items in the stacks.
Comments on visual design
Most participants did not comment on the visual design of the new website on their own.
The one participant who did thought the homepage, particularly the search area, was unappealing, explaining that they “don’t know how to describe this… it just doesn’t look nice.”
Spaces & Rooms confusion
Several participants were confused by the Spaces & Rooms category during the Training task.
Because the task asks to locate a room number, participants naturally assumed that Spaces & Rooms would help with the task.
Participants
Of the 6 participants, 4 (67%) were undergraduates, 1 (17%) was retired faculty, and 1 (17%) was unaffiliated.
Two participants were senior-aged.
Majors/departments represented include:
- Computer engineering
- Chemical engineering
- Graphic design
- Communications
- Physics
Why did they come to the library?
When asked, participants gave the following reasons for coming to the library:
- Meet a friend
- Printing (2 participants)
- Studying/classwork
- Set up accounts
- Attend a meeting
Lessons learned
The homepage of the website is very important to users. The links/sections on the homepage should be reviewed on a regular basis and re-designed if necessary.
The navigation did not get much attention during this test. More testing specific to the navigation should be done to determine just how noticeable and usable it is.
The Libraries should be aware of the terminology it uses considering that many UH users simply are not familiar with some library terms. A balance between simplifying language and teaching users new terms is the goal.
Next steps
Immediate design changes might be made in response to these results.
More testing will be done in the near future to confirm these results and/or evaluate more specific parts of the site.