AB Testing

Project Summary

In this project, A/B testing conducted found what new hires wanted on a support page. The testing compares two versions of a web page to see which one performs better.

The business goal is part of an onboarding initiative within the company. The product owners include Corporate Communications, Information Technology, and Human Resources. As owners, their goal is to push self-service within the workforce.


Business Goal

The business goal is to place on-boarding documentation into one central area for employees and the new hires.

The common denominator is new hires; all staff at one time or another was a new hire, and resources are relevant, no matter how long a person has been with a company.


UX Role

UX/UI focused on research methods, including brainstorming techniques and feedback from surveys.

Feedback gathered over three days came from various departments, which formed a foundation for qualitative research.

The project required working closely with:

  • Business Analyst
  • IT Service Delivery Managers
  • New Hires

Iterative Session #1

Employees in the company cafe and around the outer offices participated in the sessions. The feedback was candid and real.

As the user’s primary focus is to walk through the layout, I observed looking for relevant content that mattered to them

First Set of 2 Designs

UX Insight and Findings

As noted above, version “A” is what currently exists; version “B” is a revised version based on feedback from surveys.

Conversations revealed that version “A” was confusing to most; the employees could not relate to the defined groups on the page.

As noted above, version “B” was attractive due to the community support feed and the list of items to scan under “Help Center,” this approach aided with users’ short-term memory.

The top section remained the same for familiarity factors.

AB Test Session


Iterative Session #2

The turnaround time for each iteration was one day. The session consisted of five employees, with each employee taking around five minutes to review the wireframes.

The group tests were no more than 20 minutes long.

Second Set of 2 Designs

UX Insight and Findings

After reviewing feedback from version “A” and “B”; additional iterations required more testing.

As noted above, the version “C” allows employees to contribute feedback to the Support Community.

The functionality was present in version “B,” but the title changed to “Latest Conversations,” which invites employees to contribute ideas versus reading support posts.

People read from left to right; the feed moved to the left; this was intentional to make the meal more prominent for users.

Both columns expanded in width for legibility purposes. Self-service is encouraged by placing the “Contact Us” at the bottom of version”C.”

As noted above, version “D” is promoting search on the top of the page; the section encourages discoverability. The middle section is limited to three groupings and recategorized. The bottom section displays favorite links, making resources readily available, with one click.


Iterative Session #3

Third Set of 2 Designs

UX Insight and Findings

Noted above, to the left, the search remained prominent at the top, an element that stays consistent through the support section. Search is the first option for self-service.

Users related to the three groupings below, restructuring the middle part, reduced cognitive load for users; it became the second choice as their eyes moved down the page.

“Popular Links” and “Latest Conversations” moved below the main groupings as the third option, creating engagement among co-workers. As the fourth option, the “Contact Us,” “Live Chat,” and “Report an Issue” are listed on the bottom of the page, making them less prominent.

The layout of the page utilizes a grid layout, which allows for an easy read amongst groupings. The interaction cost of this design, coming from the homepage, is one click. The page focuses explicitly on new hire tasks, based on feedback from surveys.


Takeaway

The next steps recommended are to make a Proof of Concept (POC), building the site with content and limited functionality. It would determine if the material is cohesive throughout, and the interaction correlates with a user’s task.

Usability tests with a POC would confirm that the assumptions made above are valid and reassure that you’ve met and exceeded user requirements.

Going through discoverability and research will reduce interaction costs, saving your company time and money.

AB Session