Real-World .NET Applications

This research plan outlines the needs and goals of company X in order to conduct rapid user research on company X's and the competitors' products, and presents a schedule designed to meet this goal. It includes plans for five rounds of usability testing, four focus groups, and the beginning of an ongoing contextual inquiry process.A schedule of all research through the week of July 8 is included, and an estimated budget is proposed.

This research plan is valid between 5/22/2003 and 6/26/2003, at which point an updated plan will be submitted.

Research Issues

Based on conversations with representatives of design, information architecture, product development, marketing, and customer service, we have identified five large-scale issues that our research will attempt to shed light on.

• While many people use the core product comparison service, less than 1% (based on analysis of tracking cookies) ever purchase anything from the online shop.

• While the top levels of the content tree get a fair amount of use, the deeper levels, especially the product-specific sections, do not.

• The competitors' design is a lot less polished and much more chaotic, yet they get twice as much traffic with a similar amount of advertising.

• Other than knowing that they're comparing one product against another, there's little information about the circumstances under which people use the service.

• People often complain about being unable to find a specific product entry again after they've found it once.

Research Structure

The research will be broken into two parallel segments: interaction research and a profile of the existing user population.

Immediate User Research

In order to provide actionable results in time for the next release, we will immediately begin a testing process to evaluate the existing site interfaces.This will determine which elements of the design work best,which are most usable, and which features are most compelling,while finding out what doesn't work and shedding light on how users prioritize the feature set as a whole.There will be some competitive analysis included to uncover the strengths and weaknesses of the user experiences provided by competitors' products.

The techniques used will include four rounds of usability testing and, potentially, some focus group research.

• Usability testing:We will conduct four sets of one-on-one structured, task-oriented interviews with five to eight users apiece from company X's primary target audience, for a total of 20 to 32 interviews.The interviews will last about an hour apiece and will focus on how well people understand the elements of the interface, their expectations for structure and functionality,and how they perform key tasks.Videotapes of the interviews will then be analyzed for feature use trends and feature preference.There will be one round per week from 6/5 until 6/26.For each round, a report summarizing findings will be prepared within two to four business days of the completed research and presented to appropriate parties within company X. Each round will use the most recent prototype and will concentrate on the most pressing user experience issues at the time as determined by company stakeholders and previous research.

• A fifth set of tests will be of the same format with the same tasks,but will be conducted with the services provided by company Y and company Z.

• Focus groups: If no additional usability testing is needed before launch, we will conduct a series of three focus groups with six to eight users apiece from two key segments of the user base, member researchers and shoppers (as defined in the market segmentation studies obtained from marketing).These groups will concentrate on uncovering what the users consider to be the most valuable parts of the service and where the service performs below their needs and expectations.

• In addition, a competitive focus group will be conducted featuring users familiar with company Y's product discussing that company's product.

Existing User Profiling

In addition, we will begin a program to create a profile of the existing user base and to better understand how they comparison shop.This will (we hope) uncover opportunities for the service to expand into and provide a closer fit to people's lives, further encouraging its use.

• The technique used will be contextual inquiry with one to two people.

• Contextual inquiry:We will visit the homes or offices of three to five people representing a couple of the primary target audiences.We will schedule the visits for times when they expect to be comparison shopping for a specific item, and then we will observe and document (with video recording and handwritten notes) how they go about this task. We will create a model of the process they use to comparison shop based on analyzing the video and notes, enumerating what tools and techniques they use, what problems they face, and how they solve them.

Schedule

The following schedule lays out the planned research. Most work is done in parallel between several different tests in order to get the most research in the available time.The usability tests all involve about the same amount of preparation and recruiting, which can happen simultaneously for one test as the next test is being conducted and analyzed.

Focus groups involve a fair amount of preparation, but since the groups themselves are relatively short (two hours apiece), they can all be conducted in the same week (although the schedule allows for a week's slippage for the last of the regular focus groups).There's also a competitive focus group, which has its own deliverable.

The contextual inquiry project is not slated to be completed in this time period because it was determined that although understanding the use of the product in context is very important, understanding immediate usability needs is a higher priority.Thus, there are no deliverables listed for it on the schedule, but preparation for it is displayed.

(preparation weeks are shaded the light color; test and analysis weeks are shaded the dark color)

Budget

These are the projected budgets, broken out by total estimated time and total estimated costs.These are approximate figures based on experience, and they will be adjusted in future research plans to reflect actual amounts as the research progresses.

Deliverables

The results of each usability test will be sent in email as they are completed. Each email will include an outline of the procedures,a profile of the people involved,a summary of all trends observed in their behavior (as they apply to the initial research goals), problems they encountered, and a series of supporting quotations.A presentation of the results of each test will be scheduled for everyone affected.The presentation will allow the analyst to answer questions about the results and give further explanations of the proceedings of the test.

The results of all the regular focus groups will be collected into a single report, which will be sent by email as soon as it is complete. In addition to outlining the procedures used and providing a summary of the trends as they apply to the research goals, it will analyze any differences observed between various market segments.There will be a separate report from the final focus group that will compare the values and reactions of users of company Y's services to those observed with company X's.