Our usability test lab has a software environment allowing for capture, recording, analysis and full interpretation of observable events during a usability test. Moreover, since we are very interested in user behaviour, verbal comments, facial expressions and eye movements, as well as keeping a record of keystrokes and mouse movement, we also use other software.
Archive for the ‘Methodology’ Category
How is usability testing different when done on-site instead of remote? Here is a list to help you weigh the pros and cons of each method.
ON-SITE USABILITY TESTING
Pros
- Better performance for testing different interactive systems (web, software, mobility, interactive TV, vocal interfaces, etc.) and eye tracking techniques
- All participants are accomplishing the same tasks in the same environment, resulting in more stable performance data
- Technical set-up for the tests is easier
- By being on-site, the moderator can stay close to the participant and observe non-verbal behavior, hesitations and react quickly
- Audio and video, especially facial expressions are recorded
- The Client has the opportunity to observe users in real-time. It also allows them to take a break from their busy schedule to recharge and learn about their users first-hand
Cons
- Knowing he is being watched, the participant’s behavior might be influenced by a desire to perform well, despite assurances that they are not being tested
- Recruitment can be difficult because there are many marketing research companies asking for people to participate in surveys
- The pool of potential participants is limited to the local population otherwise, the usability experts need to travel
- Absenteeism rate is higher because participants have to travel to the site of the test. Other factors include weather, workload and last-minute cancellations after work, etc., might affect attendance)
REMOTE USABILITY TESTING
Pros
- The participant is in his natural habitat, at home or at work, in a real-life situation
- The technology allows for tests to be conducted anywhere in the world and in the participant’s first language
- The participant is more comfortable because he is working with familiar equipment or technology
- The participant seems less stressed, is acting more like his usual self, and is possibly less tolerant
- The moderator can still see everything the participant is doing on screen and hear what he is saying
- The moderator gets a glimpse of the participant’s private life through the use of shared screens
- The client can observe the tests from a remote location
- The sytem allows for a complete recording of screen interactions and audio
- Recruitment is easier because, having eliminated the traveling factor, pool is not limited to the local population
Cons
- Logistical elements are greater in terms of technical requirements. We need to do more pre-testing and use more internal resources
- Last minute or uncontrollable technical problems are more likely
- High speed bandwidth is essential to avoid long delays
- The use of a cellular phone for capturing audio is not ideal and the use of a hands-free landline is recommended
- Planning is more complicated because of timezones
- It’s harder for a participant to explain what he is doing or feeling when discussions between the moderator and the participant is in the participant’s second or third language
- It’s impossible to record non-verbal behavior
Either way, don’t stop testing your products with users, their feedback is too important to leave it out!
A usability testing protocol is a document that describes how to conduct usability testing. It’s also the spring board for discussions between Yu Centrik and client’s product teams.
The main goal of testing products with users is to identify usability issues encountered by users when using a product.
Through usability testing, Uability experts can observe how the user interacts with the product, evaluate how user-friendly it is and identify which functions were easier to use and which ones were more difficult.
Testing objectives are established at the onset and evaluated through scenarios developed with the Client.
During usability testing, participants are asked to do a number of specified tasks while explaining, out loud, exactly what they are doing as they accomplish the different scenarios. When the focus is on performance metrics then Retrospective Think Aloud (RTA) is best.
A user-profile defining the caracteristics of participants needed for testing is developed with the Client. Recruited participants should be representative of the product’s target client segment and include a sufficient number of people.
On the day of the test, the participant is welcomed into Yu Centrik’s offices.The moderaror explains the process and what they will be asked to do.
Each testing session lasts 60 minutes with the exception of eye tracking tests which can last 90 minutes.
The participant works through each task scenario individually and discuss as he goes along or after completing each scenario. The moderator listens and asks follow-up questions while an observer takes notes. Each session is filmed and client representatives observe in real-time.
Qualitative data is also gathered after each task in order to understand the motivations, perceptions and experience of the user. Data is then compiled, summarized and analysed.
A report on the test results, including an analysis of the results and recommendentations, is submitted and presented to the Client.
When possible, usability experts find it useful to follow-up on recommendations made because it gives them a chance to ensure that they were properly understood.
A lot of companies who wish to create or improve their web sites leave out the process of development of information architecture. Is this because of a lack of understanding of its principles and of the way it’s represented? Or is this because of the “middle of the road” aspect between the abstract ideas and concepts of a strategy and the more concrete wireframes which are closer to the final product?
What is important to know is that because information architecture bridges the gap between the abstract and the concrete, it is not only valuable, but crutial. Without it, it is impossible to build a product that represents the objectives of the company and at the same time satisfies the users’ needs. It would be virtually impossible for a product or service to thrive in competitive business without a well thought-out design structure.
Information architecture is applicable to whatever interactive product we design, be it a Web site, an interactive TV or a cell phone application. They are all dynamic products where content and features change with time. That’s the reason why a website’s basic structure must allow it to grow without significant changes that may cause a huge impact for final users and for those who keep the product alive. By the way, the biggest advantage of a well-designed information architecture is that it makes the product easier for those who use the final product and for those who are in charge of its management.
During a project, the time committed to define the information architecture must not be seen as wasted time but rather as time needed to create a high quality product in short and long terms.
Recent research has shown that people can pass judgement on a stimulus after having been exposed to it for only 50 milliseconds (Lindgaard, Dudek, Fernandes & Brown, 2006). In other words, users may make a judgement as to the quality of a website almost immediately.
Feelings and first impressions have a big role to play in determining our behaviour. Take a typical user’s decision-making process for example. The first page that user sees (whether it be the home page or some other page on the site) is crucial, since it is the first contact the user has with a product or a company.
Evidence shows that most people react by leaving a site if it doesn’t immediately fulfill their expectations, meet their needs, or if it is too complicated. Users may quickly decide to leave a site for a number of reasons. Nielsen (Alertbox, August 25, 2003) puts it this way:
- If the site is hard to use, we leave;
- If the homepage doesn’t manage to establish what the company has to offer, or what we can accomplish on the site, we leave;
- If we get lost on a site, we leave;
- If the information is hard to read, or if it doesn’t answer our questions, we leave.
Furthermore, if the site isn’t visually appealing, or if it doesn’t manage to get our attention immediately and evoke positive emotions in us (sympathy, for example), we leave.
User interfaces must meet certain design criteria if they are to generate positive emotions: aesthetics, colour use, organisational structure and the choice of media used to attract the users’ attention, to name a few. If you’re interested in a more in-depth description of these heuristics, you may wish to review the following article by Alistair Sutcliffe, which is very interesting:
(http://csdl2.computer.org/comp/proceedings/hicss/2002/1435/05/14350137.pdf).
A site’s attractiveness can definitely have an impact on the user’s decision to explore the site further or to leave it immediately.
We’ve all been there. My experience was with trying to plan a little trip on the Web. The first site I visited was nothing but text; flashing objects everywhere; but not a single photo to make me dream - what a shame !. I barely had time to lift my head before my pointer was searching for the Back button. I couldn’t get back to my search engine fast enough to find a competitive site.
There are; however, moments when the user’s goals are stronger than anything. I remember jumping some pretty big hurdles to buy some little technological item at its advertised “Low Low Price”; a savings I wouldn’t have gotten through another site, despite its being easier to use and more attractive. My goal and my motivation (buying the product at a better price) prevailed over the poor visuals and the difficulty I had navigating the site. While we’re on the subject, you may want to take a look at this blog, which was written by my colleague Marcio Leibovitch on August 23, 2006. This entry discusses the paradox of certain products being adopted because they meet the user’s real need, despite having an interface that is less than ideal.
Let me close, then, by saying that aesthetics and usability are only some of the many factors (context and motivation) that drive users’ searching and buying behaviours, but that doesn’t make them any less important.
In our field it’s important that our clients understand our working process and the steps we take before we get to the final results. It’s important because they need to believe in what we do and also be comfortable to collaborate during the project development process. We don’t just close ourselves in our offices and come out months later with magical deliverables. The work is always done in close cooperation with the client and it’s crucial that no step is forgotten or neglected.
Thus, here’s a tool I highly recommend: The Elements of User Experience, by Jesse James Garrett. The book was published in 2002, and it clearly explains the necessary steps a project must follow in order to achieve a good user experience. The book doesn’t discuss technical details and doesn’t give us all the knowledge needed to become a user-centered design expert, but this is not the goal. What it gives us is a model for establishing a clear communication between the usability professional and their client. The book is based on a diagram originally created by Garret back in 2000 and is a valuable tool that comes in handy as a daily guide.
A very good book with a quick and pleasant read. Its value lies in a series of clear and simple explanations about the steps of a project and the reasons why we should not simply find shortcuts during product development – something that happens very frequently.
I strongly recommend it.
