Group Testing

Version 1.0 / 9 March 2004

Table of contents

• Group Testing: The Best of Both Worlds
• 1. What method is best for you, focus groups or usability testing?
• 2. You get a better result when practicing both
• 3. Case: Feeling good while developing at HUB

GROUP TESTING: THE BEST OF BOTH WORLDS

A number of methods are currently available to assist in developing and testing applications. Faced with these choices, how do you choose the right technique for the right circumstance? For example, is focus group research the best method to determine if the UI of an application is attractive, or would usability testing be even better? Would focus group testing show the optimal implementation for task flows, or would usability testing give a better answer?

Group testing kills two birds with one stone. A mixture of both focus groups and usability, it delivers the best of both worlds and provides an efficient way to gather important data on users and their behavior.

Back to top

1. What method is best for you, focus groups or usability testing?

In general, a focus group is a session in which current or potential customers gather to discuss issues and concerns about the features of the product presented to them. This somewhat informal technique helps assess user needs and feelings both before interface design and long after implementation.

A focus group usually comprises six to nine users, along with a moderator who keeps the group focused on issues and concerns related to the features of a user interface. Focus groups often bring out users' spontaneous reactions about the product, and ideas for both new features and marketing positioning.

Focus groups are easy to organize and useful for judging subjective measures. However, their success depends heavily on the quality of the moderator, and they only present the customers' descriptions of what they do - not the way they actually work with the product.

Usability testing is the observation and analysis of user behavior while they are using a product or product prototype to achieve a goal. Conducting a usability test is great way to refine an ongoing design or lay the groundwork for a future offering. The purpose of usability testing is to find out practical information about how users actually use a proposed product or service. Usability tests make it possible to get user feedback on specific features that are of particular interest to the designer or developer.

A focus group is not a usability test and vice versa. A focus group can be very effective at getting opinions, but it has nothing to say about performance. According to usability guru Jakob Nielsen, the proper role of focus groups is not to assess interaction styles or design usability, but to discover what users want from the system. The opposite can be applied to usability testing: It is an excellent technique for evaluation, but inadequate for assessing user needs. Knowing this, imagine what these two techniques could produce together!

Back to top

2. You get a better result when practicing both

A more efficient way to determine user needs and evaluate the product is group testing, a mixture of both focus groups and usability testing. When you combine the advantages of each technique, you get results like never before. Group testing is a cost-efficient way to get a lot of information in a short amount of time. An experienced researcher with previous knowledge of both usability testing in general and case-specific guidelines, not to mention the underlying technology, is a prerequisite for successful testing. Group testing is best when conducted in application prototyping phase, but it is suitable also when the product needs updating to a newer version or its quality needs to be verified.

Group testing can provide a wealth of usable information. For example, it yields product information about the efficiency and ease-of-use of a product, as well as marketing information about consumer behavior, pricing, and models of use. Conducted in larger groups, the test also gives insight into whether the product can be adapted to the processes of various groups and communities. In group testing, the real focus group users provide information while using the products and discussing them with their peers. In addition to offering feedback about new product features and qualities, the test reveals issues related to cultural usability and applicability to existing behavioral models. Further, group testing can be used for concept design and other phases of product development.

To sum up:

• Group testing is cost-efficient
• You get a wealth of usable information by the real focus group users in a short amount of time
• The results can be used in several product development phases

Seven steps to produce a group test: 1. Define the target group of the product 2. Recruit the test participants through various channels 3. Arrange the test situation, tasks and questions 4. Actual group test 5. Analyze the results 6. Write the report 7. Report the results to product developers

Back to top

3. Case: Feeling good while developing at HUB

In the last days of February, a group of Web game enthusiasts from Tampere got together at Idean Research UsabilityHUB to test several mobile games and have a good time. Unlike most groups of testers, this one consisted of hardcore players with a lot of gaming experience who could give expert feedback on games that were still in the developmental state.

The players were given three different games to play and a questionnaire to fill out that also included tasks for them to perform. Players were seated at three different tables; the testing was conducted in a relaxed manner, while usability researchers took notes, asked questions, and engaged participants in lively discussions. Unlike basic usability test laboratories, the HUB environment is decorated casually, with no special attempts to provide context authenticity. Random background music plays; a wide range of refreshments are available; and cozy sofas, plants and peaceful pictures on the wall create a relaxed, neutral space for game playing.

The sessions demonstrated the advantages of group testing in a very concrete way. Continuous discussion guarantees instant feedback on both problematic and positive situations. Instead of videotaping or recording the testing process to verify hypotheses, developers get immediate validation from the group. The atmosphere is free and easy, and any possible tension caused by the researcher/target arrangement is absent: Users can try out the games and applications freely, with no constraints. Early results are available immediately if needed. Multiplayer game sessions proved to be a success: Challenging group members in front of a larger audience is always a not-to-be missed opportunity!

Feedback from the players confirmed these impressions. Although tired from the all-day excursion, they were happy and had a good time at the testing.

All were experienced players and some even had previous testing experience, but they still enjoyed playing. Some players mentioned how important it felt to be part of game development. And yes, they found some problems in the games — which is a frequent part of the usability development lifecycle that results in more usable games and therefore more enjoyable gaming experiences in the future!

Related

UsabilityHUB

Back to top

Print the document

Rate This