← All work

Verizon · Digital Channels20265 min readAI

Finding the Problems That Repeat Across Nine Verizon Purchase Flows

A seven-reviewer heuristic evaluation of nine purchase flows, sorted with help from AI, that showed which usability problems repeat and gave teams one shared list to work from.

1,328
Reviewer comments coded
27
Problem patterns seen in 5+ of 9 flows
15 of 17
Problem types found in 8+ of 9 flows

Role: Lead researcher

  • Study design
  • Heuristic evaluation
  • Qualitative coding
  • Mixed methods
  • AI-assisted synthesis
  • Workshop facilitation
01Context

Is this one flow’s problem, or all of them?

This study covered nine of Verizon’s digital purchase flows: five for new customers, four for existing ones, across the app, desktop, and mobile web. When someone finds a confusing label in one flow, it gets filed as that flow’s bug. When the same label is confusing in all nine, it’s a habit, and fixing it is a different job with a different owner.

I led the study. I planned it, ran some of the review sessions myself, and did the analysis and the write-up. I also built the workflow that sorted the comments, which used AI for the first pass.

In case the term is new: a heuristic evaluation is an expert walkthrough. Reviewers go through a flow screen by screen and judge each step against well-known usability rules. It tells you where experts think people will trip. It can’t tell you how many shoppers actually do.

02Method

How we ran it

We wanted this done fast. Reading and sorting 1,328 comments by hand wasn’t going to be fast, so I used AI for a first pass and kept every decision with the research team.

Seven reviewers took part. Each covered some of the flows and nobody covered all nine. They worked through a deck of screenshots for each flow and left a comment on any step that bothered them. Every comment was coded against the same seven issue types, so a confusing term in one flow could be counted alongside a confusing term in another.

Exhibit A: What went into the study
Exhibit A: What went into the study
FlowContextStepsReviewersCommentsIssues
Add a line, new phoneExisting customer · app104136108
Add a line, own phoneExisting customer · app164177123
Phone upgradeExisting customer · app104159118
Home Internet shopExisting customer · app911515
Home Internet buyNew customer · desktop836451
Fios buyNew customer · desktop155244160
Own phone, device firstNew customer · desktop1315353
Own phone, device firstNew customer · mobile web154229166
Own phone, plans firstNew customer · desktop134251185

How to read it: each blue square is a step in the flow, and the filled dots show how many of the seven reviewers covered it. The pale bar is how many comments reviewers left, and the dark bar is how many issues those comments merged into. Coverage varies a lot between flows, so compare patterns inside a flow more than totals between flows.

Exhibit B: How comments became findingsReviewers’ words first, sorting at scale, checks at the end
01Reviewers comment on each step
02Comments coded to a shared scheme
03Duplicates merged into issues
04Issues grouped into findings and themes
05Steps placed on 8 journey stages
06Every grouping checked twice

I wrote the rules for the AI pass. It could only use what the reviewers had written, and it wasn’t allowed to rate severity or suggest fixes of its own. Then I checked it three ways: a script to confirm every issue landed in exactly one finding, and two more passes comparing each group against what the reviewers had actually said. Every suggested fix in the report is the reviewers’ own.

03Findings

What came back

The short version is that the same problems showed up almost everywhere.

Exhibit C: From comments to themesEach step condenses the one before
1,328CommentsLeft by seven reviewers across 109 steps
979IssuesDuplicate comments merged
190FindingsRecurring patterns within each theme
17ThemesShared across the nine flows

Here’s what one finding looks like from the inside. Five comments from four reviewers, in four different flows, ended up as a single finding. One of those comments is mine.

Exhibit D: One finding, traced back to the comments behind it

What reviewers wrote (paraphrased)

Existing customer · app · choosing a number

I can't tell whether 'No' is selected. The outline around it is too faint.

New customer · mobile web · choosing a number

Same problem here: I can't tell if 'No' is picked, because the outline is so subtle.

Existing customer · app · device page

The selected option has a black outline, but it doesn't stand out enough to show which one is chosen.

New customer · desktop · picking a planMy comment

I picked a plan and the only thing that changed was a lighter 'Remove' button. The plans I didn't pick still had bold 'Select' buttons, so mine looked less chosen than theirs.

New customer · desktop · offers and gifts

The 'Select' buttons on the other items stay highlighted instead of the 'Remove' button on the one I added, so I can't see what's added or how to undo it.

What it became

FindingThe selected option is hard to tell apart from the unselected ones

The selected state of cards, boxes, and buttons shows up only as a subtle outline or a changed button label, so reviewers couldn't tell what was chosen. In places, the chosen item looks less chosen than the ones left alone.

Reviewers' own suggestion: A thicker border, a check mark or 'Selected' on the card, and a highlighted Remove button, used the same way across the experience.

↓ one of 10 findings in this theme

ThemeSelections are not confirmed or shown clearly

31 issues, found in 8 of 9 flows

Comments are paraphrased. Each one stayed its own issue here, since each came from a different flow or step.

  1. 01
    15 of the 17 problem types showed up in at least eight of the nine flows.

    27 specific patterns appeared in five or more flows. That’s a lot of overlap.

  2. 02
    Most findings had more than one reviewer behind them.

    159 of the 190 findings were raised by at least two different reviewers. The most widely repeated one, no progress tracker or competing ones, came from six of the seven reviewers across seven flows. Single issues look different: 855 of the 979 came from one reviewer, so the agreement shows up in the patterns.

  3. 03
    Wording, cost, and text size were the big three.

    Unclear wording had 136 issues. Costs and conditions that were unclear or shown late had 96, and text that was too small, faint, or out of view had 95.

  4. 04
    Most comments landed early in the journey.

    Entry pages got 269 issues and device and line setup got 235. Be careful with that one: stages with more steps and more reviewers collect more comments, so it tells you where comments pile up, not which stage matters most.

Exhibit E: Where the comments pile upIssues by journey stage
269
S1Entry
125
S2Plan
235
S3Device & line
93
S4Offers
64
S5Add-ons
38
S6Number
119
S7Cart
36
S8Checkout

Stages with more steps and more reviewers collect more comments, so this shows where comments gather and not which stage matters most.

Putting the counts next to the comments told me two things. Six of the 17 problem types show up at every one of the eight stages, which changed how I read all those separate “this wording is confusing” comments. It’s one language problem that keeps happening, and a lot of small ones would have been the wrong way to see it.

The other was where specific problems sit. 24 of the 29 issues about add-ons with no clear way to decline are at the add-ons step, and 29 of the 44 issues about carts that are hard to add up are at the cart. The reviewers’ own words say what was going wrong in each place.

Exhibit F: Where each problem type shows upIssue counts by journey stage
Recurring problem
S1Entry
S2Plan
S3Device & line
S4Offers
S5Add-ons
S6Number
S7Cart
S8Checkout
All
Jargon, abbreviations and unclear wording9/9 flows
43
8
36
3
9
8
16
13
136
Costs, fees and conditions unclear or revealed late8/9 flows
25
15
26
2
4
4
10
10
96
Important text too small, faint or out of view8/9 flows
30
7
21
14
8
3
9
3
95
Hard to tell where you are, where to go, or how to get back9/9 flows
38
9
24
8
0
0
12
1
92
Controls behave unexpectedly or inconsistently8/9 flows
27
8
33
8
3
4
6
1
90
Oversized, loud or undifferentiated layout9/9 flows
22
9
19
7
2
4
7
1
71
Marketing, ads and pop-ups get in the way of the task9/9 flows
24
5
10
10
1
1
3
0
54
Forced choices, late errors and no undo8/9 flows
2
8
22
5
2
0
9
3
51
Gifts, perks and promotions are confusing9/9 flows
9
11
4
15
1
1
6
0
47
Cart and order summary are hard to add up8/9 flows
0
3
2
3
0
6
29
1
44
Information contradicts what was shown earlier8/9 flows
8
13
7
2
3
0
2
1
36
Product names are inconsistent or unexplained8/9 flows
23
4
5
2
0
0
1
0
35
Selections are not confirmed or shown clearly8/9 flows
1
2
10
2
3
6
5
2
31
Not enough information to compare and choose8/9 flows
7
8
10
6
0
0
0
0
31
Add-ons offered without introduction or a clear way to decline8/9 flows
0
1
0
4
24
0
0
0
29
Content buried in tooltips, accordions and small windows6/9 flows
3
8
6
1
3
1
2
0
24
Assumes the shopper already knows the offer6/9 flows
7
6
0
1
1
0
2
0
17
All themes
269
125
235
93
64
38
119
36
979

How to read it: darker means more issues raised. A count shows how often something was raised, and says nothing about how bad it is. Dashed cells mean no issue was recorded.

Reviewers also left 252 notes under “What works”, in seven patterns. The two biggest are prices and fees, and the cart, which are also the two areas people criticized most. So some flows already handle these well.

One more thing, and this one is my own read, kept apart from the reviewers’ findings in the report: everything about what a shopper will pay adds up to the biggest single area, 140 issues across several themes.

04Outcome

What happened next

There was pushback, as you’d expect when you critique screens people built. Teams defended specific screens and wondered whether the reviewers had the right context. I’d built the workbook so every finding links back to the reviewers’ own comments, so when someone challenged a finding, we could open the comments and look.

Executives reacted most to the cross-flow view, where you could see one problem sitting in flow after flow.

Then came the triage workshops, with people from different functions in the room. Every finding went into one of three piles.

Exhibit G: How findings were triagedCross-functional workshops

Quick wins

Small, contained fixes a team could take on right away.

Strategic bets

Larger changes that needed roadmap space and agreement across teams.

Backlog and icebox

Valid findings that were parked, with the evidence kept on file.

A few things changed because of it.

  1. 01
    Roadmaps got reprioritized.

    The workshops fed straight into what teams planned next.

  2. 02
    Design system changes.

    When a problem repeats across nine flows, it usually lives in a shared component, so the findings sent people to the design system.

  3. 03
    More testing with real shoppers.

    This was an expert review, so the obvious next step was usability testing on the highest-priority findings.

  4. 04
    Better tracking.

    Where teams couldn’t yet measure a problem, analytics and KPIs were added so fixes can be judged on behavior.

Delivery is still going. Findings are becoming tickets and epics, and regression and QA benchmarks are being set so fixed problems stay fixed. I’m leaving out outcome numbers because they don’t exist yet.

Limits worth knowing

This is experts reading screens, so it can’t say how many shoppers struggle, and the counts show how often something was raised, not how bad it is. The groupings are my reading of the comments, checked twice against the reviewers’ words, but no second researcher reviewed them.