Quick Bites & Specialty Food Markets

*The filters on the left-hand side show new categories, Quick Bites and Specialty Food Markets, which enable diners to discovery options more in line with what they are looking for.*

Employer
Tripadvisor (tripadvisor.com)

Team
-Marketing (email communication to owners, PR messaging)
-Industry Relations (to work with industry partners around the world to help define requirements and communicate changes)
-ML engineers (for building and training a model)
-Software engineers (for creating the filters)

Problem
There are so many different types of eateries and restaurants. If I’m looking for a romantic place to celebrate an anniversary, I won’t be interested in a deli counter. Alternatively, if I need to grab a quick bite between appointments, I wouldn’t consider a traditional sit-down restaurant.

User stories
As a diner, I want to easily distinguish between a traditional restaurant with table service, and a place to grab a quick bite so that I am confident that the place I choose to eat will fit my event/schedule.

As a restaurant owner, I want to be discovered by prospective diners when they are searching for a place like mine to eat so that I can get more traffic to my business.

Business case
Being a B2C feature, this change was less about direct impacts on churn or revenue, and more about creating an excellent diner experience. We actually prioritized this project due to some international industry partners encouraging us to do so—saying that in certain markets (particularly in Europe) this mismatch of expectations between a grocery store or quick bite place vs a sit-down restaurant caused confusion and frustration for both diners and owners. Owners of Michelin star restaurants want to be compared apples to apples with other sit-down restaurants...not with a pizza counter.

Solution

Definitions
First things first, we needed to nail down definitions for Quick Bites and Specialty Food Markets.

We decided that Quick Bites places typically have at least one of these characteristics:
-Diners pay before receiving their order (e.g. at a counter where you place an order)
-Primarily a take out / take away establishment
-Drive through or pick up windows
-Does not include table service
-Limited seating
-Diners order from a limited menu

And Specialty Food Markets are places where you can buy ingredients to make your own food.

Machine Learning Model
As we had done with other Establishment Types on the site, we needed to develop a Machine Learning algorithm to identify Quick Bites and Specialty Food Markets. There are many factors that go into these models which I cannot share publicly, but one of the many inputs is a question that we pose to diners when they are looking at a restaurant’s listing.

For Quick Bites, we ask: Is this a place where you pay before receiving your order?

For Specialty Food Markets we ask: Is this a place where you buy ingredients to cook your own food?

Coming up with these questions required lots of consultation with stakeholders. As a PM, I want to make sure I am considering things like:

Questions that are specific enough to create an accurate representation, but simple enough to understand by the user.
Questions that translate well into other languages (this is tricky!)
Questions that, when answered in the affirmative, indicate that the establishment may be a QB or SFM. For example, it’s insufficient to ask “Does this restaurant offer table service” and then assume that if a user answered “No” that it means that it’s a Quick Bites place.

Aside from the questions, we also needed to train the model with existing eateries we knew to fit our definitions of QB and SFM. I used SQL queries to create a list of such establishments, and manually checked the initial set of 100 restaurants that the model categorized as either a QB or SFM to ensure that it was on the right track.

This QA step caught some important issues. For example, Specialty Food Markets were really leaning more towards fish markets, and so we adjusted the initial data set to be more inclusive of other types of markets as well.

Screen Shot 2020-05-17 at 2.24.04 PM.png

Rollout & Communication
Rolling out features like this needs to be done with care. Tens of thousands of restaurants around the world depend on Tripadvisor to send them traffic, and so we want to make sure that we don’t miscategorize eateries or miscommunicate changes.

Unfortunately, we knew we’d get some backlash from this feature. Suppose you are the owner of a deli that had been listed as the #1 Restaurant in Paris for the last 3 years. Of course you will be frustrated to find that you are now the #1 Quick Bites place, and that you have lost your standing in the overall Restaurants category. However, we felt that this needed to be done for the overall good of diners.

In order to ease this transition, we first categorized restaurants as QB and SFM, and had the filter selected by default when a diner landed on a list page. We felt that this would give any restaurants who may have been miscategorized the time to reach out to our support team. A few weeks later, we removed the default selection

I worked closely with marketing to outline three variations of the same email announcing this change:

To those who were not impacted (their category was not changing)
To those newly categorized as Quick Bites
To those newly categorized as Specialty Food Markets

I also filmed overview videos and helped create internal documentation that we circulated within Sales & Support teams in order for each rep to be able to handle any issues or objections with the new categories.

Finally, we rolled this out in an A/B test to 50/50 traffic and kept an eye on some key indicators, such as bounce rates on list pages. Overall, the metrics looked good and we decided to fully roll out the feature.

Reception

The new categories received a lot of press coverage in Italy (one of our key industry relations played a huge role in getting this project prioritized). The change was featured in many publications including the ones below.

As expected, some owners were also frustrated by the change, and our support team did their best to explain to them the ‘why’ behind a change. We did also get feedback from owners understanding that this was overall a beneficial change to the site.

Quick Bites & Specialty Food Markets

Solution

Support Chat