How Tree testing will help you to build a better Information Architecture
While working at MediaMonks I had to deal with a big fish. It was a huge project that was asking us for a complete redesign of its website.
I had the honor to participate in the early stages of the project when the groundwork for the production phase is laid down. And it was where we had to propose a new Information Architecture (IA).
IIn order to test our proposal of the new Sitemap, I used a tool from Optimal Workshop called Tree Jack. This tool is perfect when it comes to testing a Sitemap remotely.
Tree testing is an exercise that will help us to understand if users felt comfortable and if they were successful in navigating through the new Information Architecture we proposed (without the influence of navigation aids and visual design).
Tree testing helps us to prove the site structure will work before we get into interface design.
To prepare the testing we proposed a number of tasks. We will ask our users to find something in the Sitemap, so they have to navigate through and select where they would find the piece of content we are asking for.
How to set up a Tree testing
Participants
We didn't have a magic number on how many people needed. But I can tell that we needed at least 30 people in order to collect useful insights and see trends.
Tasks
Write as many tasks as you need but not too many since there will be a high risk of abandons. The exercise shouldn’t take longer than 15 minutes.
Your tasks should be written in the users' own language, with words easily recognizable for them. Don’t give any clue to your users. This means that you should use different words from your Sitemap, therefore, you are taking the risk of giving away the answer.
Tip: introduce a hypothetical scenario for people to bring to mind. I like to use “imagine that you need…”
Test the test
Before launching the test, grab 3-5 colleagues and ask them to try the test. Arrange individual time slots that allow you to sit next to each colleague and observe them in person. In this way, you could see if they understand what they have to do and the tasks are clear enough for them to complete the test.
Iterate and improve it and when you feel like it’s ready: send the link to your users!
Analyzing and interpreting the results
Before diving into results and starting to make conclusions, there are three main factors to take into consideration when it comes to reading the results:
Success rate: Users who found the right category for that task.
Directness: Users who went to the right category immediately, without backtracking or trying any other categories.
Time spent: The average amount of time elapsed from the beginning to the end of the task.
Ok, I think we are ready to start. The steps that I took to review the results and get some insights from are the following:
Step 1. Review the table of participants.
Take some time to review the participants' table to make sure they are all worth including.
You can sort the table according to Time Taken, and exclude participants that you think finished the tree test too quickly (because they were selecting the answers randomly), or took too long (because they were doing other stuff and they were not focused).
Step 2. Task by task breakdown.
Then have a look at the overall view of each task.
You will find out:
How many participants found the correct answer
How directly they arrived at their chosen answer
How long they took to do it
Observe the percentage of participants who selected the correct answer. A success score from 80% is a good score for a task.
Step 3. Follow the paths.
Look closely at the paths to see where the users were expecting to find the information and which labels led people in the wrong direction. You can filter this table to have a better understanding of what happened.
Conclusion
Tree testing is a nice tool to consider when building Information Architecture. It will help you out to find out if the proposed labels are working for your users but also if the categories make sense for them. I completely recommend it to effectively evaluate a site’s navigation hierarchy and to avoid common design mistakes.
