Verizon · Digital Channels20265 min readAI
Finding the Problems That Repeat Across Nine Verizon Purchase Flows
A seven-reviewer heuristic evaluation of nine purchase flows, sorted with help from AI, that showed which usability problems repeat and gave teams one shared list to work from.
- 1,328
- Reviewer comments coded
- 27
- Problem patterns seen in 5+ of 9 flows
- 15 of 17
- Problem types found in 8+ of 9 flows
Role: Lead researcher
Is this one flow’s problem, or all of them?
This study covered nine of Verizon’s digital purchase flows: five for new customers, four for existing ones, across the app, desktop, and mobile web. When someone finds a confusing label in one flow, it gets filed as that flow’s bug. When the same label is confusing in all nine, it’s a habit, and fixing it is a different job with a different owner.
I led the study. I planned it, ran some of the review sessions myself, and did the analysis and the write-up. I also built the workflow that sorted the comments, which used AI for the first pass.
In case the term is new: a heuristic evaluation is an expert walkthrough. Reviewers go through a flow screen by screen and judge each step against well-known usability rules. It tells you where experts think people will trip. It can’t tell you how many shoppers actually do.
How we ran it
We wanted this done fast. Reading and sorting 1,328 comments by hand wasn’t going to be fast, so I used AI for a first pass and kept every decision with the research team.
Seven reviewers took part. Each covered some of the flows and nobody covered all nine. They worked through a deck of screenshots for each flow and left a comment on any step that bothered them. Every comment was coded against the same seven issue types, so a confusing term in one flow could be counted alongside a confusing term in another.
| Flow | Context | Steps | Reviewers | Comments | Issues |
|---|---|---|---|---|---|
| Add a line, new phone | Existing customer · app | 10 | 4 | 136 | 108 |
| Add a line, own phone | Existing customer · app | 16 | 4 | 177 | 123 |
| Phone upgrade | Existing customer · app | 10 | 4 | 159 | 118 |
| Home Internet shop | Existing customer · app | 9 | 1 | 15 | 15 |
| Home Internet buy | New customer · desktop | 8 | 3 | 64 | 51 |
| Fios buy | New customer · desktop | 15 | 5 | 244 | 160 |
| Own phone, device first | New customer · desktop | 13 | 1 | 53 | 53 |
| Own phone, device first | New customer · mobile web | 15 | 4 | 229 | 166 |
| Own phone, plans first | New customer · desktop | 13 | 4 | 251 | 185 |
How to read it: each blue square is a step in the flow, and the filled dots show how many of the seven reviewers covered it. The pale bar is how many comments reviewers left, and the dark bar is how many issues those comments merged into. Coverage varies a lot between flows, so compare patterns inside a flow more than totals between flows.
I wrote the rules for the AI pass. It could only use what the reviewers had written, and it wasn’t allowed to rate severity or suggest fixes of its own. Then I checked it three ways: a script to confirm every issue landed in exactly one finding, and two more passes comparing each group against what the reviewers had actually said. Every suggested fix in the report is the reviewers’ own.
What came back
The short version is that the same problems showed up almost everywhere.
Here’s what one finding looks like from the inside. Five comments from four reviewers, in four different flows, ended up as a single finding. One of those comments is mine.
What reviewers wrote (paraphrased)
I can't tell whether 'No' is selected. The outline around it is too faint.
Same problem here: I can't tell if 'No' is picked, because the outline is so subtle.
The selected option has a black outline, but it doesn't stand out enough to show which one is chosen.
I picked a plan and the only thing that changed was a lighter 'Remove' button. The plans I didn't pick still had bold 'Select' buttons, so mine looked less chosen than theirs.
The 'Select' buttons on the other items stay highlighted instead of the 'Remove' button on the one I added, so I can't see what's added or how to undo it.
What it became
The selected state of cards, boxes, and buttons shows up only as a subtle outline or a changed button label, so reviewers couldn't tell what was chosen. In places, the chosen item looks less chosen than the ones left alone.
Reviewers' own suggestion: A thicker border, a check mark or 'Selected' on the card, and a highlighted Remove button, used the same way across the experience.
↓ one of 10 findings in this theme
31 issues, found in 8 of 9 flows
Comments are paraphrased. Each one stayed its own issue here, since each came from a different flow or step.
- 0115 of the 17 problem types showed up in at least eight of the nine flows.
27 specific patterns appeared in five or more flows. That’s a lot of overlap.
- 02Most findings had more than one reviewer behind them.
159 of the 190 findings were raised by at least two different reviewers. The most widely repeated one, no progress tracker or competing ones, came from six of the seven reviewers across seven flows. Single issues look different: 855 of the 979 came from one reviewer, so the agreement shows up in the patterns.
- 03Wording, cost, and text size were the big three.
Unclear wording had 136 issues. Costs and conditions that were unclear or shown late had 96, and text that was too small, faint, or out of view had 95.
- 04Most comments landed early in the journey.
Entry pages got 269 issues and device and line setup got 235. Be careful with that one: stages with more steps and more reviewers collect more comments, so it tells you where comments pile up, not which stage matters most.
Stages with more steps and more reviewers collect more comments, so this shows where comments gather and not which stage matters most.
Putting the counts next to the comments told me two things. Six of the 17 problem types show up at every one of the eight stages, which changed how I read all those separate “this wording is confusing” comments. It’s one language problem that keeps happening, and a lot of small ones would have been the wrong way to see it.
The other was where specific problems sit. 24 of the 29 issues about add-ons with no clear way to decline are at the add-ons step, and 29 of the 44 issues about carts that are hard to add up are at the cart. The reviewers’ own words say what was going wrong in each place.
How to read it: darker means more issues raised. A count shows how often something was raised, and says nothing about how bad it is. Dashed cells mean no issue was recorded.
Reviewers also left 252 notes under “What works”, in seven patterns. The two biggest are prices and fees, and the cart, which are also the two areas people criticized most. So some flows already handle these well.
One more thing, and this one is my own read, kept apart from the reviewers’ findings in the report: everything about what a shopper will pay adds up to the biggest single area, 140 issues across several themes.
What happened next
There was pushback, as you’d expect when you critique screens people built. Teams defended specific screens and wondered whether the reviewers had the right context. I’d built the workbook so every finding links back to the reviewers’ own comments, so when someone challenged a finding, we could open the comments and look.
Executives reacted most to the cross-flow view, where you could see one problem sitting in flow after flow.
Then came the triage workshops, with people from different functions in the room. Every finding went into one of three piles.
Quick wins
Small, contained fixes a team could take on right away.
Strategic bets
Larger changes that needed roadmap space and agreement across teams.
Backlog and icebox
Valid findings that were parked, with the evidence kept on file.
A few things changed because of it.
- 01Roadmaps got reprioritized.
The workshops fed straight into what teams planned next.
- 02Design system changes.
When a problem repeats across nine flows, it usually lives in a shared component, so the findings sent people to the design system.
- 03More testing with real shoppers.
This was an expert review, so the obvious next step was usability testing on the highest-priority findings.
- 04Better tracking.
Where teams couldn’t yet measure a problem, analytics and KPIs were added so fixes can be judged on behavior.
Delivery is still going. Findings are becoming tickets and epics, and regression and QA benchmarks are being set so fixed problems stay fixed. I’m leaving out outcome numbers because they don’t exist yet.
This is experts reading screens, so it can’t say how many shoppers struggle, and the counts show how often something was raised, not how bad it is. The groupings are my reading of the comments, checked twice against the reviewers’ words, but no second researcher reviewed them.