Usability Testing for Nordstrom

Project overview

This study was developed for a Usability Studies course at UW taught by Rebecca Destello, a Research Manager at Facebook.

The full report on this usability study can be viewed here.

• This study was designed to assess the utility and usability of the BOPUS (Buy Online, Pick Up In Store) service on the Nordstrom iOS application, which allows for same day order pick up from a local Nordstrom store.
• We looked at BOPUS’ discoverability and ease of use throughout the customer journey, from browsing for products through the checkout process.
• We were mentored by Angela Sharer, a Senior UX Researcher at Nordstrom, in the design and execution of this study.

Nordstrom’s “BOPUS” product was heavily researched before it launched. However since then, Nordstrom added the “Nordstrom Local” feature to the iOS app. They tested the “Nordstrom Local” feature in the cities where it launched, but they weren’t able to look back at how the BOPUS product as a whole was working for the markets that don’t have the “Local” feature.

The team

Responsibilities on this study were shared by the entire team. As the only Nordstrom employee on the team, I managed much of the communication and logistics of the project.

Methods

Participants were recruited from friends and family to meet the following criteria:

• 25-45 year old women
• iPhone users
• Have never used the Nordstrom iOS application.
• Have used other mobile retail apps to purchase goods (within the past 6 months).
• Recent Nordstrom customers (within the past 6 months).

This study was conducted in the Nordstrom Corporate Office’s usability labs. Each participant was compensated with a $25 Nordstrom gift card. There was one moderator and one notetaker in the room. The other two group members and the group mentor (Angela Sharer) watched from the observer room.

When participants arrived at the study, we had them fill out a background questionnaire to get some more detail about their shopping habits. We also asked them a few verbal questions when the test started to break the ice. During the test, the notetaker and moderator recorded behaviors, observations, and participant answers on a pre-formatted form that can be found in the full report. The two team members in the observation room noted important quotes and participant behaviors on Post-It notes and placed them adjacent to printed screenshots of the relevant screen. After the tests we compiled the following data for each participant.

The full screener survey and other questionnaires are available for view in the full report on this study linked at the top of the page.

We spent a lot of time perfecting our task list and prompts for the usability test. We wanted to be able to compare the participants’ browsing with and without the BOPUS feature “turned on.” To do this, we had participants first browse with no particular timeline in mind, then we added the requirement that they pick up the items the same night in store.

Results (The Good & The Bad)

All of our participants had shopped at Nordstrom in the past six months, and had favorable views of Nordstrom overall.

Our report focused on two phases of the customer journey. Both involved the “Get it Fast” filter, which is the button you select to turn on the “BOPUS” feature. There is an image showing this button on a later slide.

This is how participants discovered the “Get It Fast” button. Only one used it immediately upon starting to browse.

Evaluate

Primary issues

Participants selected “Get It Fast”, entered their zip code, and selected their store. The feature on their search pages then read: “Get It Fast: Downtown Seattle - Pick up these items today” leading to the expectation that products populated using this feature would be eligible for pick up. Issues arose when participants selected colors and/or sizes and found that their selected sku/combination was not available.

Recommendation

In search, encourage users to filter by item size when applicable. To address color discrepancies, once the search results page populates, (after users select “Get It Fast”), we recommend only showing the number of colors available for “Get It Fast”.

Having already selected “Get it Fast” in search, entered their zip code, chose their stores, selected items and then confirmed those items were available for pick up, participants missed or misunderstood the required action needed on the product page to add to bag for BOPUS. Instead of selecting the “Buy & Pick Up” tab on PDP, participants skipped over this section which was defaulted on “Free Delivery” and added to their bag. Once reviewing their bag, participants noticed that items were set to ship instead of pick up as was expected.

Recommendation

The correct action needed to be taken by customers is to switch to the “Buy & Pick Up” tab, confirm their store is selected, then select “Buy & Pick Up in Store”, then click “Buy & Pick Up” to add to bag. We recommend that when the “Get It Fast” filter is on, the shipping option on the product detail page default to “Buy & Pick Up”.

When reviewing orders prior to checking out, participants were unable to f ind detailed information on when their order would be ready for pick up.

Inconsistencies within the shopping journey created additional confusion. The “Get It Fast” filter says “pick up these items today”, the product detail pages say “We’ll notify you when it’s ready and where to find the Order Pickup area”, and Bag and Review say “Today” and include today’s date.

Because this pickup details wasn’t available, participants made assumptions about when their orders would be ready for pick up. One participant expected 15-20 minutes, one within 2 hours, and the others expected sometime today, but had no clear indication on timing.

Recommendation

To help customers better plan their BOPUS experience and to prevent customer dissatisfaction, in addition to date available, we recommend including a clear timeframe for when the order is expected to be ready prior to order completion.

Secondary issues

The Nordstrom UX team followed up with us a few times about our findings on the app filters. They used our study to inform future research on filtering issues on the app. Some examples include:

• When clicking into a PDP, the main image on the PDP didn’t always match what the participant had seen in the thumbnail in SBN. For example, P2 filtered on blue sweaters, but when she clicked on a thumbnail of a blue sweater, it pulled up the PDP of the same sweater in orange (Frequency: 3 / 5 participants).

• P3 wanted to select “blouses and collared shirts [and] bodysuits” while filtering on categories to view, but only one category may be selected at a time (Frequency: 1 / 5 participants).

• When selecting filters, clicking “apply” felt like an extra step to some participants (Frequency: 3 / 5 participants).

• The order in which filtering options and search results were listed was not what participants expected (Frequency: 2 / 5 participants)

Recommendation

Conduct more thorough research on filtering in app. This is a robust feature and we were only able to skim the surface with our study. The Filtering function deserves much more probing.

Reflections on our process:

• Our open-ended scenario encouraged participant monologues, despite our fear that tasks were too “simple” and may not find anything of significance.
• We also learned how to be better study moderators, as we were given feedback on our physical, facial, and verbal responses to participant monologues and inquiries.
• Conducting a pilot earlier in the process would have helped us scope what data to collect, set up our equipment, and provide ample time to troubleshoot the technical issues found in the early stages of our study.
• We took turns moderating and taking notes during the usability studies. In retrospect this made consistency a challenge but it was important to us that each person had a chance to experience leading a study. In the future, a set notetaker and moderator would help us improve consistency while also making data easier to clean and analyze.