DSP Payments

Fixed Invoice Portal

client

Amazon

industry

Logistics / Fintech (enterprise technology)

project

Fixed invoice portal

services

Research Ops, UX Research

The challenge

DSPs, the small delivery businesses that deliver to your door, were frustrated by Amazon’s opaque invoices and dispute process—50% of DSPs identified fixed payments as their primary pain point.

before

My task

Identify what is causing DSP frustration and re-dispute cycles.

Project goal
  • Reduce DSPs’ fixed invoice disputes, including invalid disputes and re-dispute cycle.
  • Eliminate DSP frustration and anxiety when reviewing their monthly fixed invoices.
Early insight

Payments and invoice disputes were sensitive topics—DSPs were angry, frustrated, and exhausted—causing sessions to run over and observers to engage the participant.

Actions

I revised my research protocols, provided additional training to observers, and gave participants a time and place to vent before we got into the meat of the interview.

With participants breaking down from frustration and generally overwhelmed by their financial challenges when going over their invoice processes I added time and a parking lot into the interviews so we wouldn’t lose our opportunity to learn from them. I realigned the research process and discussion guides to let the participants express their feelings and know that they were heard and that we were taking steps to address their issue. In later iterations of this project, I re-invited these participants to evaluate the UI against their needs, effectively testing our ability to synthesize their pain and gain trust from listening to our customers. Participating stakeholders received additional training and interview instructions to minimize unneeded engagement.

Tested UI

Insight

DSPs did not understand Amazon’s ‘math’.

  • Across the Payments experience, DSPs pointed out issues calculating what was owed. Their numbers never matched Amazon’s.
  • Lacking an ability to easily understand Amazon’s payments, DSPs would dispute their invoices.
  • When disputes were not accepted, DSPs would become frustrated and resubmit. Some DSPs calculated they were waiting on $50k or more due to disputes.
Action

After I discovered that it was a math issue, I pivoted, working with UX and Product to focus on the best way to communicate Amazon’s calculations in the UI. I took UX comps into research and identified the best language and visual UI to communicate Amazon’s math. Including visual ‘math’ alleviated DSPs’ concern over the numbers, and the conversations helped us identify the best method (and contents) to communicate the complex numbers.

Insight

Accounting vocabulary was not common among DSPs, creating confusion and indiscriminant clicking until they found the document they sought.

  • When navigating between pages, DSPs could not differentiate between invoice types, clicking indiscriminately based on date until they found the right invoice.
  • Lacking understandable language, DSPs struggled to navigate the UI.
Action

Working closely with the Content Strategist, PM and UX Designer, we prototyped multiple vocabulary changes to humanize the language used throughout the UI. We landed on explicit, clear terms that articulated the ‘what’ in the invoice. This resulted in accurate targeting with no “hunting” behaviors.

“Great job. New system is unbelievable. Previous system was a nightmare. The difference is night and day. Awesome, awesome, awesome. I love it! This is slick!”

DSP owners / research participants

“This tool, the way it’s set up, it is gorgeous. I was able to navigate through everything using just the tool alone. I didn’t go back and look at my fleet counts, I stayed right in the tool and I worked within the tool. I didn’t go out at all. I didn’t write anything down. I didn’t export anything to Excel. I just used the tool.”

DSP owner / research participant

Outcomes
Q1 A/B pilot testing:
  • Reduced invalid disputes by 76%
  • Reduced inconclusive disputes by 69%
  • Reduced disputes due to confusion by 52%
  • Reduced pre-payment disputes by 28%

The Invoice and Dispute product launch accomplished three objectives:

  1. It reduced fixed invoice disputes by showing DSPs how Amazon calculates their payments each month, and provides in-line feedback loops reducing the number of invalid disputes and the re-dispute cycle. Overall fixed invoice dispute rate dropped 700 basis points (bps), from 18% in March, to 12% in April and 11% in May, with at least 90 bps directly attributable to UX changes.
  2. It streamlined DSP’s invoice review and dispute process, reducing the time it takes for DSPs to dispute their invoices. Reduced DSP invoice review time by 75%, from ~4 hours/month to >1 hour (based on post-launch interviews).
  3. As a result of the above, it reduced the cognitive load and overall frustration DSPs experience reviewing their monthly fixed invoices. Reduced the number of DSPs who mentioned the fixed payments experience as their primary payments pain point from 50% in Q1 to 3% in Q2.