Yu Centrik

In industrial design projects, including everything from improving the user experience of an interactive television system to performing open-ended user research for an upcoming product, designers are presented with the challenge of synthesizing best practices and contextual analysis into an effective human-computer interface.  I’ve encountered a significant amount of confusion with respect to the priority of quantitative and qualitative information.  The fact is, commercial design is a discipline which will forever walk the line between the subjective and objective.  This raises a major question:

How do we know when to rely on statistics and when to trust our creative instincts?

STATISTICS VS. CREATIVITY

The scientific approach to solving a problem suggests hypothesizing a solution based on previous work, testing it experimentally, and reporting results.  Through such theory and experimentation, we can draw conclusions which move us closer to understanding and solving real problems.  And so goes the science of user experience design.  Based on previous experience, best practices, experimental evidence, I can suggest a design solution which will likely optimize the interface between a certain technology and its users.  Performing usability tests allows me to validate my suggested interface and tweak the system, creating a predictably usable interface.  Simple, right?

Wrong.

What’s missing from this description is the artistic.  The fact of the matter is that human-computer interaction design can benefit substantially from a creative approach.  When our clients want systems which engage, attract and achieve a sense of flow, we need to move beyond the objective, incremental scientific process.  The artistic approach represents an increase in risk but also potential reward (especially in a growing, competitive industry).  By constantly evolving our art-form through our experience with innovative products on various projects in various contexts, we improve our ability to build designs which communicate with users; the deeper our insight into the human experience, the more depth we can provide in artful, “big picture,” interactive solutions.

So what’s the correct answer?  When should we trust the numbers, and when should we take creative risks?  While each project needs its own tailored approach, every designer must be aware that both methods are interrelated.  Their relationship is not simple, but I can certainly provide some insight directly from my experience researching, designing and testing human-computer interfaces.

SIX RULES TO BALANCE THE SCIENCE WITH THE ART

1.  A completely objective and numeric approach is a safe bet when time and budget are constrained.  In a quick project which doesn’t have time for abstract design methods and adequate user testing, a more objective approach is typically a good way to minimize risk and get a quick job done with minimal internal controversy.

2.  With a client who is willing to take risks for reward, a completely objective and numeric approach is a missed opportunity.  While interface usability can be improved with an objective scientific approach, the other elements of the user experience tend to be neglected.  Where possible, attempting to inject creative and innovative approaches where possible can help increase overall flow, mental modeling, engagement, hedonic factors (fun), and user satisfaction.  In a thriving industry, innovative solutions can be a major competitive edge.

3.  A completely improvised creative approach is never a good idea.  It may work occasionally for graphic and visual design, but principles and existing work in human psychology and industrial design allow us to avoid blindly hoping that our users will react well.  There is always a place for science, even in the most creative approaches.

4.  Facts not only predict user behaviour, but they also inspire creative design solutions.  Facts about how users interact with a system tend to logically suggest a solution.  For instance, the fact that “90% of users didn’t notice the button” would suggest that this button needs to be bigger and more prominent.  However, when taking all problems into account, it becomes clear that not everything on the screen can be prominent; we can’t simply enlarge every button.  A few behavioural facts may lead designers’ instincts to a deeper level.  It is here where the experienced designer’s creative approach can solve many problems at once with a more elegant design solution.

5.  The more testing you’ve planned, the more creative risk you can afford to take.  If your project has several iterative tests planned, you should inform your design team that innovation is expected.  Lateral and less-directed design methods like parallel prototyping and group workshops can lead to innovative and groundbreaking solutions which not only optimize the usability of the interface, but generate an engaging and attractive user experience which gets people talking.  Sure, there will be failed tests and wrong steps, but without experimenting, how can you possibly expect to innovate?

6.  Fix edge-cases when they don’t affect more common use cases.  When testing a design with many users, if a very small minority of users has trouble with a certain element, it may seem statistically irrelevant.  However, if such problems seem repairable without affecting other users, they’re still worth fixing.  Think about it, if only 1 out of 10 test participants have trouble with an element, the problem could either be negligible or it may translate to 10% of all your users, or maybe even more!  If you’ve identified a problem that’s easy to fix and doesn’t harm the experience of other users, why take the risk?

–

Hope this helps you walk the line!

It is often necessary to use fairly complex tools to perform usability tests on voice interfaces (cf. Nexus[1], Database Systems Corp.[2]). One of our projects involved a touch-activated automated telephone information system. We required a test solution that was flexible, easy to build, and which provided a relatively natural simulation of a telephone call that would allow us to capture the call and a certain number of events during said call. It also had to be easy to modify and not too expensive.

We therefore developed a simple flash utility that was adapted to simulate a user’s phone call into a voice interface. It had the following characteristics:

1. Easy to use interface;
2. Not demanding: running this kind of application only requires an Internet explorer and a FLASH player;
3. Simple to modify: tests were iterative, so changes were made to the application’s structure after each test (three in all);
4. Allows the user to manage streaming video and audio. Works with video and audio recording software;
5. Helps collect and analyze data: for every call, the tool allowed us to capture the type of button pressed, the steps taken through the voice interface, the amount of time spent listening to messages, etc…  All of these data are critical for performing an in-depth usability analysis of any voice interface.
6. Easy to transfer to other voice interface applications.
7. Not too expensive: a FLASH developer developed this tool for us in two weeks.

Tool Description (Front End)

A simple screen reproduces a virtual telephone, which only the moderator can see and operate. Participants hear the instructions and press the buttons on the external numeric keypad to move on to the following instructions. Voice prompts are played on the computer’s audio port.
The moderator can record the participant’s name and surname, as well as the type of scenario and persona. The moderator launches the application; stops it whenever the participants are finished with their tasks; can interrupt the process at any moment during playback; and can replay any section.

During the call, the buttons the participants touch appear on the screen one after the other. Thus, the moderator sees in real time what steps the participants took. It also helps avoid errors in note-taking and compile the interface’s critical points quickly between participants.
Logs are also available to retrace the in-session keypad interactions: number of times zero was pressed, time spent inactive, etc…

Control System (Back End)

The interface’s structure management application assigns a telephone keypad key to each menu option. This is how the voice application’s menu path, options, and messages are created, how mp3 audio files are associated to them, and how errors, inactivity, redirects and message interruptions are accounted for.

Example:

A menu is created from left to right (see the figure above).

1. Click on ¨+¨ to create a new thread. Click on ¨-¨ to delete one. A “thread” is a menu, a message, a choice of languages, etc.

2. Name it and assign it a type (menu, message, etc.) according to its internally-defined nomenclature. The mp3 audio file (recorded in English or French) should have the same name. Assign it a time. This is the time within which a user should react (like three seconds, as in our example).

3. In the right-hand section, each key on the keypad is assigned a function and a response is defined for inactivity or error (ex: what sound to play if no key has been pressed within three seconds).
As you can see, the structure can be modified very quickly.

Conclusion

The results showed that FLASH, which is most commonly used for graphic interfaces, can also be handy for prototyping voice interfaces.
It can be used to develop a flexible application quickly, when you need to replicate a telephone call and to easily record the user’s experience.

The system has the added advantage of being transposable to other voice applications. We see the potential to use it in remote usability tests in the future, since FLASH makes it possible to develop interactive multimedia tools online and in local environments.

Still, the tool needs to be improved by linking up with a real telephone. This would most resemble a real call made from home.

Webliographic References
[1] Franz Neeser. Testing IVR Systems, White paper. Nexus Telecom AG, Switzerland. May 2005.
http://www.nexus- ag.com/fileadmin/documents/Whitepaper/White_Paper_IVR_Testing_
Nexus8610_Ed_2.2.pdf

[2] Database Systems Corp.

Source image
Laptop computer image: openclipart.org/media/files/aurium/4163

So, where do I start?  If you’ve been following my Twitter @jayvidyarthi, you know that last week I experienced an overload of knowledge transfer at the annual CHI conference in Boston.  I wanted to write a blog updating all of you on some of the fantastic methodologies, processes and technologies on the fringe of our field.  First, let me tell you a little about my read on the main theme of the conference.

Tying directly into the Closing Plenary delivered by Kees Overbeeke from Eindhoven, I observed the underlying theme of HCI and design today as focusing on the humanization of technology.  Computer systems no longer have to be cold and digital.  Instead, they can behave more organically, installing themselves into our lives in a natural way.  Kees described how our relationships with technology can even go beyond such seamless integration.  He presented a lamp which gradually loses power.  As one pets the lamp, as if it were a cat or dog, it comes back to life.  This was an example of technology which can appeal to our emotions and connect with our users in a more profound way (while also being environmentally friendly!).  The tradeoffs between a practical and useful lamp and a “pet” lamp are still to be explored, but Kees’ point of view undoubtedly inspires questions about the future directions of human-computer interaction.

Although not every researcher is ‘dreaming of the impossible’ like Kees, a broad survey of projects from various angles at CHI 2009 show that seamless integration is the underlying theme of our technological future.  I decided to summarize a few interesting projects for you to check out, but by no means is this a comprehensive summary.  Check out the official CHI program summary to get many more descriptions and references on the fringe of HCI.

Some Highlights

• Eric Gilbert’s work on building a predictive model for social networks allowing a system such as Facebook to automatically distinguish your close friends from your distant acquaintances based on your behaviour with the system.  A short summary on Eric’s blog
• John Zimmerman is building a strong design philosophy centered around creating products which promote self-identity.  An application of product attachment theory to design specifically for role transition and self enhancement.  Really fantastic stuff for the creative theorist.  John’s Research Description
• Sunny Consolvo is working on the Ubifit Garden, a wallpaper for your mobile phone which measures how much time you spend sedentary, walking, running, biking, etc., and grows flowers and plants to indicate your physical exertion.  The system serves as a subtle ambient measure of your own exercise which creates a motivating factor within your cellphone.  A Poster Describing the Ubitfit Garden
• Tovi Grossman et. al. at Autodesk are designing and testing a new approach to measure the learnability of an interface: the Question-Suggestion model.  By having a coach directly interact with users, the method extracts specific problems and provides a more precise measure of interface learnability.  Extremely valuable for any kind of longitudinal usability testing of complex interfaces.  Description of the Paper @ ACM
• David Geerts from Leuven presented well-justified heuristics for the implementation of social functionality in interactive TV systems.  Hopefully I’ll hear more from David when I head to Euro ITV 2009 in Belgium to present Yu Centrik’s interactive TV research findings.  David’s Presentation Slides (jump to slide 33)
• Sarah Diefenbach and Mark Hassenzahl presented a thought-provoking piece on the interaction of Beauty and Usability on consumer product choice.  The paper served as a launching point for an intense panel discussion including a tirade on the definition of beauty from Bill Buxton.  Very engaging stuff.  Sarah and Mark’s Thought-Provoking Paper
• Leanne Hirshfield from Robert Jacob’s lab at Tufts University presented some early work on using fNIR brain imaging to get a read on human syntactic and semantic workload.  The eventual fruits of this work could lead to interfaces which can adapt to the users’ cognitive load by reading bloodflow in the brain.  If this project starts to fly, interfaces could get very interesting.  A Description of the Adaptive Interfaces using fNIR Project
• Alexander Hoffman is working on TypeRight, a keyboard which uses language dictionaries to adjust the resistance of keys on the keyboard based on the probability of each letter being pressed.  I tried using the keyboard and barring some minor technical issues, it seems to have potential to help us avoid type-os and increase our overall typing speed.  The TypeRight Project
• David Frolich presented a case study of StoryBank, a mobile-phone and touch-screen based system allowing children in rural India to develop narrated slideshow videos.  Being a kind of “local, embodied youtube”, the interfaces involved no language, enabling use by illiterate children in the village.  The findings were quite surprising, and the videos made were very powerful.  David was intent on stressing how our technology can help the developing world.  The StoryBankHomepage

Again, if you enjoy this type of work, check out the CHI program summary for more quick descriptions!  If you want details and have access to ACM’s digital library, check out the full proceedings.

Our usability test lab has a software environment allowing for capture, recording, analysis and full interpretation of observable events during a usability test. Moreover, since we are very interested in user behaviour, verbal comments, facial expressions and eye movements, as well as keeping a record of keystrokes and mouse movement, we also use other software.

How is usability testing different when done on-site instead of remote? Here is a list to help you weigh the pros and cons of each method.

ON-SITE USABILITY TESTING

Pros

• Better performance for testing different interactive systems (web, software, mobility, interactive TV, vocal interfaces, etc.) and eye tracking techniques
• All participants are accomplishing the same tasks in the same environment, resulting in more stable performance data
• Technical set-up for the tests is easier
• By being on-site, the moderator can stay close to the participant and observe non-verbal behavior, hesitations and react quickly
• Audio and video, especially facial expressions are recorded
• The Client has the opportunity to observe users in real-time. It also allows them to take a break from their busy schedule to recharge and learn about their users first-hand

Cons

• Knowing he is being watched, the participant’s behavior might be influenced by a desire to perform well, despite assurances that they are not being tested
• Recruitment can be difficult because there are many marketing research companies asking for people to participate in surveys
• The pool of potential participants is limited to the local population otherwise, the usability experts need to travel
• Absenteeism rate is higher because participants have to travel to the site of the test. Other factors include weather, workload and last-minute cancellations after work, etc., might affect attendance)

REMOTE USABILITY TESTING

Pros

• The participant is in his natural habitat, at home or at work, in a real-life situation
• The technology allows for tests to be conducted anywhere in the world and in the participant’s first language
• The participant is more comfortable because he is working with familiar equipment or technology
• The participant seems less stressed, is acting more like his usual self, and is possibly less tolerant
• The moderator can still see everything the participant is doing on screen and hear what he is saying
• The moderator gets a glimpse of the participant’s private life through the use of shared screens
• The client can observe the tests from a remote location
• The sytem allows for a complete recording of screen interactions and audio
• Recruitment is easier because, having eliminated the traveling factor, pool is not limited to the local population

Cons

• Logistical elements are greater in terms of technical requirements. We need to do more pre-testing and use more internal resources
• Last minute or uncontrollable technical problems are more likely
• High speed bandwidth is essential to avoid long delays
• The use of a cellular phone for capturing audio is not ideal and the use of a hands-free landline is recommended
• Planning is more complicated because of timezones
• It’s harder for a participant to explain what he is doing or feeling when discussions between the moderator and the participant is in the participant’s second or third language
• It’s impossible to record non-verbal behavior

Either way, don’t stop testing your products with users, their feedback is too important to leave it out!

A usability testing protocol is a document that describes how to conduct usability testing. It’s also the spring board for discussions between Yu Centrik and client’s product teams.

The main goal of testing products with users is to identify usability issues encountered by users when using a product.

Through usability testing, Uability experts can observe how the user interacts with the product, evaluate how user-friendly it is and identify which functions were easier to use and which ones were more difficult.

Testing objectives are established at the onset and evaluated through scenarios developed with the Client.

During usability testing, participants are asked to do a number of specified tasks while explaining, out loud, exactly what they are doing as they accomplish the different scenarios. When the focus is on performance metrics then Retrospective Think Aloud (RTA) is best.

A user-profile defining the caracteristics of participants needed for testing is developed with the Client. Recruited participants should be representative of the product’s target client segment and include a sufficient number of people.

­ On the day of the test, the participant is welcomed into Yu Centrik’s offices.The moderaror explains the process and what they will be asked to do.
­
Each testing session lasts 60 minutes with the exception of eye tracking tests which can last 90 minutes.

The participant works through each task scenario individually and discuss as he goes along or after completing each scenario. The moderator listens and asks follow-up questions while an observer takes notes. Each session is filmed and client representatives observe in real-time.

Qualitative data is also gathered after each task in order to understand the motivations, perceptions and experience of the user. Data is then compiled, summarized and analysed.

A report on the test results, including an analysis of the results and recommendentations, is submitted and presented to the Client.

When possible, usability experts find it useful to follow-up on recommendations made because it gives them a chance to ensure that they were properly understood.