Usability testing was recently conducted with 12 library users in MD Anderson Library.
A “user cafe” was set up in the MD Anderson Library lobby. Passers-by were enticed to complete the test with promises of free coffee and snacks after completion.
The test asked participants to complete 3 tasks using the Libraries main website, info.lib.uh.edu.
In this post:
- Methods
- Tasks given
- Results
- Task results
- Mistakes explained
- Major observations
- Participants
- Why did they come to the library?
- Lessons learned
- Next steps
1. Methods
Usability testing means observing users completing tasks with a website (or anything else) to see how it is used.
By observing how users actually use the website (instead of how they say they use it), design changes can be made to enhance and improve usability.
A “user cafe” was set up in the MD Anderson Library lobby. Passers-by were enticed to complete the test with promises of free coffee and snacks after completion.
Credit: This “cafe” protocol of user testing has been adapted from a presentation by Zoe Chao, UX Librarian at Penn State University Libraries.
This new method of collecting user research data is currently being piloted over the summer. More will be written about this “user cafe” method in the coming months.
I recruited and tested participants myself. Users filled out a brief survey before starting, seen above.
A recording was taken of the participant’s voice and screen during testing.
Tasks given
Users were asked to complete each task using info.lib.uh.edu without searching.
The following tasks were given to participants. Task wording is shortened for this blog post.
Task #1
Show me where on this website you’d find the times that the Architecture and Art Library is open on Saturday.
This is referred to as Hours task in the rest of this post.
Task #2
Show me where on this website you’d find the email address for the chemistry subject librarian.
This is referred to as Email task in the rest of this post.
Task #3
Show me where on this website you’d find the room your Photoshop course is in on Monday, June 26 at 11:30am
This is referred to as Training task in the rest of this post.
2. Results
A total of 12 participants completed all 3 tasks within a 2.5 hour period.
The recording for the first participant was lost.
Task results
Participants were either successful, mistaken, or failed.
Success: Participant found the information asked.
Mistaken: Participant incorrectly thought they found the correct information.
Failure: Participant gave up looking for information.
The graph above shows the tasks results for all 3 tasks.
Mistakes explained
There were 3 instances of participants being mistaken.
One participant incorrectly thought the day’s hours listed on the top of the Architecture & Art Library page applied to every day. The task asked for Saturday’s hours, which are different than usual.
One participant did not realize they were looking at the wrong library’s hours on the Hours page.
One participant incorrectly thought they could only use the Ask Us a Question form on the Contact Us page to email a subject librarian.
Major observations
Limited navigation use
Few used navigation options to find information. No participants used the “Contact Us” link in the nav. Only 1 used the link to the Hours page found on the nav.
Heavy use of homepage links
The link farm on the website homepage was where most participants started to complete most tasks.
“Contact Us” doesn’t mean staff directory to some
While most participants eventually found the staff directory and correct email address, some expressed confusion in arriving there via Contact Us. They noted that they didn’t expect “Contact Us” to have a link to a directory of all staff. Note that some users got to the staff directory through other paths, e.g. homepage links, footer links, and not through Contact Us at all.
Links in sidebar sometimes ignored
Several participants did not notice links located in sidebar content, especially on the Contact Us page.
Rare use of staff directory filters
Only 1 participant actually used the “Subject Librarian” filter on the Staff Directory to find a certain subject librarian. The rest scanned the list of all staff members.
control+F for searching pages
For those participants that scanned the list of all staff members, several used the control+F function to search the page for the chemistry librarian to save time. If those participants had noticed the filters mentioned above, this function would be unneeded.
“Course Reserves” is unfamiliar to some
A few users went to the Course Reserves page hoping to find information about the Technology Training courses due to unfamiliarity with what “course reserves” are.
Participants
Of the 12 participants, 6 (50%) were undergraduates, 4 (33%) were graduate students, and 2 (17%) were UH Language & Culture Center students.
Majors for the remaining 10 participants include:
- Industrial engineering
- Mechanical engineering
- Business
- Communications
- Education, natural sciences, and math
- Industrial engineering
- Political science, law
- Electrical engineering
- Computer science
- Electrical engineering
Why did they come to the library?
When asked, participants gave the following reasons for coming to the library*:
- Use a library computer (3 people)
- Studying/reading (3 people)
- Work/homework (3 people)
- Eat lunch (2 people)
- Relax (1 person)
- Return books (1 person)
- Checking schedule for technology courses (1 person)
- Getting familiar with campus (1 person)
*Note: Some participants listed more than one reason
3. Lessons learned
- The User Cafe concept is a workable solution for routine user testing, but changes to the protocol are needed. Many passers-by did not understand the sign and tried to get coffee/snacks before completing a test. Passers-by also often tried to talk to me during a usability testing, which was distracting and likely affected results in some way.
- It is likely viable to attract users to take part in a short test if coffee and snacks are offered. Usability tests, user interviews, card sorts, and other testing methods can be applied to the Cafe concept.
4. Next steps
- Another Cafe session is planned for the next few weeks. As it is currently being piloted during the summer, the protocol will be adjusted to make things clearer to passers-by and to prevent interruptions.
- The next Cafe session may use different user testing or research methods than usability testing, e.g. semi-structured 10 minute user interviews.
- Data and insights from this session will be used on future designs of the Libraries website.