Drawing floor plans with AR

Every new restaurant that uses the OpenTable GuestCenter product needs to configure their floor plan and schedule before they can take reservations on OpenTable.com. One of the major obstacles to self-service restaurant onboarding is this task of building a floor plan. Today, account managers can spend up to two hours building out a floor plan before meeting with the restaurant because they don't want to make the restaurant owner sit through the many steps of the drag and drop configuration. This ultimately costs the company at least 8000 hours a year (conservatively $256K).
How we create floor plans today:

Our goal for the 3-day company hackathon

Use Apple's ARKit to build a tool that makes adding a floor plan fast and fun.

Why augmented reality?

We considered alternative technologies that could also speed up the process such as scanning a floor plan from a photo, or using the phone accelerometer to walk around and tap to add tables. However, AR captures more info about the physical space and tables which means we can make better guesses and require less input from the user.

The multi-device challenge

While the owner would build a floor plan using an iPad, all of the rest of the restaurant setup process is completed on a computer. How would we send a user from their computer in the back office to their phone or iPad at the front of the restaurant and then back to their computer again? While we didn't build out all of the solution during the hackathon, we did capture the flow of how it would work.

Designing the AR experience

AR detection of the table allows us to pre-populate the table shape and size. By detecting height, we might even be able to guess if it's a high-top or bar seat.

For every table, the owner has to add a table number, the rest of the fields are pre-populated based on the data we capture by scanning the table. We could guess at the table number based on the last table number, however restaurants don't necessarily number their tables sequentially from 1 to 100. Using a custom keyboard, we could suggest what we think is the next table number so it's easily accessible, but isn't annoying if we're wrong.

Flags on the previously added tables make it clear which tables have been completed and what is left to do.

The steps above to add a table should be fast and allow the user to keep moving through the restaurant quickly without needing to make a lot of changes. However, the final step should always be to review the layout of the tables and quickly check for any miss-named tables.

Hackathon results:

In 3 days, we were successfully able to build and demo an app that could detect a table, add it's name and attributes, and keep track of all the added tables in the room. We also won the People's Choice Award among over 40 teams!

Considerations for future work

3 days allowed us to showcase how OpenTable can put AR to a practical use in our restaurant products. To fully integrate this into our product in the future, there are some things we need to think about:

Having the user switch devices while in the middle of onboarding always runs the risk of distractions and losing the user's focus. However, the risk is probably lower than the time-intensive and tedious process of dragging and dropping in a GUI. We'd want to test with users to see where the drop off points are and how to address them.
What's happening on the computer when the user is on the iPad? Can the user go anywhere else in the web product during that time? What happens if they close the browser window?
We built this feature into our iPad app because every restaurant owner needs an iPad to use the GuestCenter product in their restaurant. However, what if they want to onboard before they buy an iPad? Is the experience ideal with a smaller screen?

The team

I was the team lead and designer, 1 other designer, a restaurant onboarding specialist, 3 product managers, and 2 engineers.