Customer delivery profile 2.0

Overview

At Amazon, our mission is to provide our delivery drivers with data that lets them know as much about a building as someone who has delivered there 1000 times. One way we do this is by collecting delivery instructions and other data about delivery points from drivers and customers. Customer Delivery Profile (CDP) is a feature that was built to allow Amazon.com customers to provide delivery instructions (referred to as address attributes) to help Amazon deliver to their addresses successfully. CDP surfaced four simple questions that mapped to our most common delivery defects.

With the original launch of CDP, the experience was static for all customers, and did not fully meet the needs of commercial customers. On average, we found that 43.3% of customers that interact with CDP provide at least one attribute, that number is much lower for commercial addresses (36.6%) vs. single family homes (43.1%) vs. apartments & other multi-tenant residential buildings (54.9%). We believed this was because our current attributes like access information (e.g., building entrance codes, callbox information, etc.) and unattended delivery preferences (e.g., leave the package on my back porch), were most relevant for residential properties.

Project goal

Our goal was to transition from showing each customer the same experience to surfacing a personalized CDP that only asks contextually relevant questions based on customer need. Our plan was to customize the experience based on the customer’s property type (house, apartment, business, other) and defect history (unable to locate, unable to access, no safe location, business closed).

Customer Delivery Profile (CDP) 1.0

Understanding & defining the Problem

Discovery

I started by mapping out the customer journey and identifying all existing touch points and workflows across platforms (desktop and mobile) and customer types (Prime, non-prime, and commercial). I wanted to familiarize myself with the experience and in turn discovered several variations in the execution of how we collect address attributes from customers. This helped identify existing problems, visualize the scope of the effort, and provided a starting point for prioritizing the workstreams.

CDP entry points across the customer journey—from shopping through delivery.

Documentation of all flows—including mobile and desktop.

Identified inconsistencies and alternate flows.

Research

To fully understand our customer’s needs, I planned and executed multiple research activities:

  • Baseline usability test. I initiated a usability study to baseline the current customer experience and gain qualitative insights to inform the new CDP 2.0 Personalization design effort. Because the mobile and desktop experiences differed slightly, I tested both using a task-based usability study on UserTesting.com. A total of 16 (8 desktop, 8 mobile) Amazon customers were recruited and screened for a mix of delivery address types (house, apartment, business or other). Individuals were also screened for a recent negative delivery experience to target customers that would be more inclined to provide the attributes that CDP collects.

  • Data analysis. Working with my Product Manager, we also kicked off a deep dive of existing data collected via CDP. We wanted to analyze the attributes customers were providing—specifically free-text inputs—to determine the quality of feedback and identify patterns or issues.

  • Amazon Business research insights. Another primary goal was to improve the experience for our commercial customers. I partnered with the Amazon Business team to learn more about these users and the problems they experienced with deliveries.

The Opportunity

The problems we needed to solve followed three main themes:

  • Lack of personalization. Showing all customers the same questions, in the same order, wasn’t the most effective way to collect the data we needed. Using a property type (home, apartment, business, other) attribute, I could customize what we asked and the order we asked it for each customer—increasing engagement and relevance of the data collected.

  • Data collection was not optimized for commercial customers. Our original set of attributes over-indexed on residential customers and didn’t address the needs of our business customers. I would need to introduce new attributes specifically designed for the business customer—for example, the ability to provide business hours or the times which we could successfully deliver packages.

  • Existing usability issues. In addition to a laundry list of usability issues identified, I discovered the entire feature was not at all accessible. I audited the existing experience and lead the effort to bring the product into compliance with the Web Content Accessibility Guidelines (WCAG).

Crafting the solution

Customization by Property type

To achieve customization by property type, I began by thinking through how customers might provide this information. I explored several concepts and after many rounds of reviews with stakeholders and user testing, we arrived at our final design for property type selection and the order of attributes associated with each type.

Optimize for the business customer

To improve the experience for our business customers, I partnered with the Amazon Business team to learn more about their customer’s unmet needs. After benchmarking several business hour collection experiences, I began to iterate on multiple design concepts for CDP’s delivery hours attribute. Through internal stakeholder reviews we narrowed down to 3 concepts to test with customers. For this study, I built 3 prototypes and asked participants to enter 2 delivery hours configurations—one simple and one more complex with split (lunch) hours. The primary goal of the study was to identify usability problems and opportunities with each design, as well as validate the language being used.

Results

Following launch, the new personalized CDP improved customer engagement from 23.3% to 32.6%, resulting in 12.1MM in savings. The product today provides foundational inputs to Amazon’s route planning and delivery execution. 670K delivery drivers use these inputs to arrive at the accurate customer doorstep while delivering over 30MM packages every day. CDP continues to help Amazon create the best-in-class at-stop experience for drivers at complex delivery locations—business parks, high rise mixed-use buildings and garden style apartments.