Augmented Reality Proof-of-Concept

Role: Product Owner/UX Lead

Team: Art director, 3D designer, Unity developer, Technical analyst

Timeline: H2 2017

After months of working behind the scenes to create the Emerging Tech (ET) group, I was excited to finally launch it through an all-agency presentation in September 2017. Leading up to that, I knew that it was critical we began work on a proof-of-concept (POC) to ensure that we weren’t just talking the talk, but that we were walking the walk as well. With an ET POC, my goal was to create showpieces that we could use to:

1. Up-level client perceptions of POP’s capabilities

2. Catalyze new business development conversations

3. Share out our own learnings with the larger design and tech community

Apple's launch of ARKit in the summer of 2017 marked a watershed moment for augmented reality: for both creators and users. Third-party developers now had a lower barrier to entry in creating AR applications. For users, the launch of iOS 11 in the fall of 2017 meant that 500 million iPhone users would gain mainstream access to augmented reality.

The moment was ripe, and I targeted ARKit as our technology platform for the inaugural ET POC. In my typical UX design process, I start with the user and the problem first, and I remain platform-agnostic as long as possible. In other words, focus on the nails first, before I go looking for the right tool. In this unique case though, we wanted to showcase our experience with an emerging technology, ARKit. So after I made that initial decision, I then shifted the conversation to the user and their problem.

Discovery: Generative Research and Ideation

I called a meeting for initial ideation, gathering ET group members as well as a few other designers and developers who had expressed interest in AR. In advance of the meeting, I asked everyone to come to the table with their own POC ideas. I set two key thought starter questions:

What utility or value would the user get from using AR? How can we deliver something simple, done very well, within a timeboxed 4 weeks?

I would typically take this time to research competitive and comparative experiences. Since iOS 11 hadn’t been released yet, we couldn’t use any AR apps (outside of Pokemon Go and Snapchat) firsthand, so I watched ARKit demo videos from Twitter. I could immediately see the ubiquity of "AR as sticker," which in my opinion, underscored the challenges of AR productization. In other words, “AR as sticker” highlights the novelty factor of AR and grabs attention for one-time use, but I saw there is very little compelling return visits to existing ARKit applications.

At our initial ideation meeting, I kicked off with a Crazy 8s exercise. Each person, regardless of role, was given a stack of 8 Post-it notes on which to sketch or write their idea within the span of 8 minutes. Group brainstorming can disservice ideation, so my goal was to ensure that everyone had the time and space to speak up on their individual ideas. After the exercise concluded, I asked each person to walk the group through each of their ideas, discuss any questions that arose, before placing their Post-It note ideas on the whiteboard.

I had intentionally invited both designers and developers alike because I knew: First, this would foster healthy debate around blue-sky AR envisioning balanced with what functionality is actually supported by ARKit. Second, since this was an R&D investment project, it would always be deprioritized in favor of billable client work; but if I got everyone involved early on, then they would feel more bought in and committed. I then grouped the Post-It notes based on use case categories (e.g., AR for tabletop interactivity, for modeling a product before purchase, for gaming, etc.). We then voted for the ideas we’d most like to execute, with each person having 3 “ballots” to cast.

Getting Executive Buy-In

Next, we needed to get approval from our CEO. I created a pitch deck based on the top 4 ideas. A presentation may have felt like overkill for an internal approval on an R&D POC, but I had learned from my agency/consulting experience that a little of “pitch theater” is key for drumming up excitement amongst stakeholders who may not be as close or invested in the work itself. For each idea in my pitch deck, I highlighted a one-liner for what it was, the unique utility that AR provided to the end user, and the value POP could get out of executing this idea.

What are we doing? ARKit tent shopping

What problem are we solving? It’s hard for customers to determine which tent is right for their needs. This helps them visualize it in real life, moving them more quickly to a confident purchase.

Why’s this matter? In addition to its utility for customers, this POC offers us PR value in the immersive spin we can take on ARKit (e.g., stepping into the tent as a portal, haptic feedback through phone, changing environments outside). In terms of BD, it helps us show other eCommerce clients how we solve customer pain points by engaging customers in fun and different ways.

Potential future extensions:

Step-by-step tent assemblyCompare sleeping bags (both rolled up and expanded)Build your campsiteTry on ski gogglesMulti-user shopping (incorporating geolocation, social)

I presented to our CEO and CTO, and they immediately loved the Tent Pitch idea — How might we use augmented reality to make it easier to confidently shop for tents within your own home? It solved a real pain point for outdoor recreation enthusiasts, the idea delivered on the potential of augmented reality, and we hadn’t seen anyone else do it yet. I also learned from my CTO the importance of a POC that is wholly-owned by POP. Even though REI is one of our clients, and this POC would be a clear fit, we would need to stay away from any other branding and make sure that the POC could be whitelabeled for our other retail clients (e.g., Target, The Home Depot) to continue telling that story of, how does AR help customers try before they buy?

Define: Targeting the Use Case

Having received executive approval on our ARKit idea, we moved onto defining the experience of the POC itself. Unlike traditional web and mobile apps, AR is not confined to a 2D rectangular screen. This meant that in addition to creating the AR app itself, we would need to think about the situational considerations around the AR app: e.g., Where is the user? What is the user doing before the AR activation? How do they activate it? How are they interacting with the physical versus augmented reality? What is their next step after the AR experience? To start answering these questions, our art director and I whiteboarded out potential steps for the user flow that our POC would support. The key for us was to keep the user flow as targeted as possible — to execute on few things very well with our POC, rather than oversell user expectations with a technology as foreign as AR.

I realized that our POC had three tiers of users: the end user, who we defined as an iPhone user who was a regular online shopper and outdoor enthusiast; POP clients, with whom we intended to share this POC to ignite BD conversations; POP client engagement (CE) team, the account directors who worked closely with our clients and whose jobs depended on closing new business.

With this in mind, our art director and I created storytelling pieces to convey our vision. He created a storyboard based on our whiteboarded flow, while I created a concept writeup. I shared these with our ARKit project team, as well as our CEO, CTO, and CCO, to get alignment on our vision. I also shared with a CE team member to get their input on how this could impact their client conversations.

Balancing the User Delight and User Benefit of AR

In defining our ARKit concept, my key goal was to balance its inherent novelty (creating user surprise and delight) with its unique utility (providing user value). Thus our concept allowed the end user to place an AR tent in their physical environment, walk around it to gain a sense of its real-life scale, come up close to examine tent features — all features that paid off what utility can AR provide that no other technology can — as well as climb into the tent for an “Easter egg” of being transported to the Grand Canyon and hearing eagles soar overhead — which paid off the inherent fun of AR.

Design and Develop: Kick-Off

With alignment on the concept and storyboards, I gathered our team to define requirements and set the definition of done for our POC. We kicked off design and development: Our art director and 3D designer worked closely together to create our own tent model. Our developer started blocking out the process of placing an AR object that was fixed in the physical environment independent of where the user stood. I worked with our art director to define and design the user’s screens that lead up to the AR activation (while the user is at home shopping for tents on their phone) as well as the in-AR UI.

Unexpected Design Challenges of ARKit

Before we started this project, I did not fully realize the limitations of ARKit, namely its limited partial ground plane detection, which meant that sometimes the phone would not correctly recognize where the ground actually was in relation to the user. This challenged the UX because it meant that when the user tapped their phone to place an AR tent, it would occasionally float in mid-air. From this problem, I realized how critical onboarding is for ARKit apps, which isn’t apparent from most of the recorded demos available online, which usually begin filming after the user has set the phone to successfully detect the ground. I re-focused our attention on onboarding, which we broke into four key steps.

1. Once the user has activated AR, we added a visual cue to prompt to them to turn their phone to landscape orientation for better viewing of the tent.

2. We included direction for the user to tap their phone screen to place the tent.

3. At which point, a placeholder mark appears on the screen to convey the size and placement of the tent.

4. The user taps the check button to confirm their desired placement, and the tent appears.

Validating Our Decisions Through User Testing

I assumed that this onboarding is critical to a successful AR user experience. To validate our design decisions, I tested with 6 iOS phone users in the office (given the need to be scrappy with our recruiting given the demands of a POP-investment project). I uncovered two key unexpected findings.

First, our testers were delighted by crawling into the tent (or shortcut toggling to be inside it). They marveled at the sound effects of the bird calls overhead, and they would pan around to try and guess what nature landscape they had “teleported” to. Before designing this experience, I knew the importance of balancing utility with user delight through AR; what I didn’t expect was just how important that experience of tent teleporting would be! It left a huge smile on most testers’ faces, and they continued remarking how “cool” it was even after the session. It was clear to me that this is what would stick with users, even knowing the novelty would wear off over time.

Second, four of our six testers expressed concerns about the true scale of the tent; they weren’t sure how much to trust it. This was problematic, since it underpins the value proposition of the experience. I collaborated with our developer, and he added a product feature button that would trigger a ruler to reinforce the actual height, width, and depth of the tent. I also found William Steptoe's research that cited increased user trust in the AR object when the entire screen was slightly stylized. However, we were unable to implement this due to timing constraints.

Measuring Success

At the end of an unrelated meeting with our REI client, I shared the POC with him. Coincidentally, he was in the early stage of developing a similar POC using Google’s ARCore. He was impressed by our AR initiative and by our understanding of proven pain points around tent shopping, even though we hadn’t been exposed to their user research on this issue. As a result, he invited us to REI’s Innovation Days hackathon, where we formed a team, hacked on an in-store AR campsite experience, and took home first place out of 20 teams. We are also in the process of closing a SOW on our first official AR engagement together. Ultimately, the ARKit POC fulfilled the two key goals I’d set upfront: not only did we uplevel client perception of our capabilities (after working with REI for over a year on exclusively email campaign production), but we also closed a new business opportunity. In terms of what the future holds for the ARKit POC, I am working with POP’s marketing team to create a case study, and I have shared the POC with the rest of our business development team to showcase with their own clients.