We like to run lots of experiments in and out of campaigns to test how users behave in metasearch and understand how we can help drive our clients more bookings at higher ROIs. In a recent experiment we set up a controlled metasearch environment and tracked how users responded to price, position, and advertiser-types.
You can get a sense of this experience through the mockup below.
By running this experiment we wanted to answer a few interesting questions:
It’s not a great surprise that in a value-conscious marketplace, users most frequently choose the offer with the lowest price. As you can see in the data below, users also are more likely to choose the lowest price when it’s a better deal.
Another way to think about this is that according to this experiment the “low price” in a metasearch ad auction is 8x more likely to get clicked.
If you’ve spent much time in the digital marketing world you know that typically CTR declines significantly with ad position, and metasearch is no different.
Our experiment allowed for an ad with a higher price to be shown in position one (the Left position) in order to measure whether position can draw users away from a lower price offer.
We believe this has been shown in the data above; if position had no impact on selection and CTR, we would expect 9% of users to have selected the left offer in the “Lower Price Right” scenario instead of 15% of users as was observed.
We’ve seen in previous experiments that there is some user preference towards Suppliers (hotel brand direct) vs. OTAs, but haven’t been able to measure it this as cleanly in the past.
When everything is the same, we observed a meaningful bias to Suppliers. This continues to exist in scenarios where the Supplier is more expensive. A 1% difference in the rate only had a small impact on the selection rate of Suppliers. Only when there was a 10% difference in price did the selection rate begin to even out.
We were lucky to be able to survey users after their selection and found that brand preference worked for OTAs as well, just not as frequently as it did for Suppliers. Here is what users self reported as the reason for their booking choice:
As you might expect by now, Price was confirmed again as the greatest factor in ad selection. Trust and Supplier Preference were second and third. Trust is interesting to us because it often benefits the OTAs; users were very familiar with these brands. While overall we saw a preference for Suppliers, there were definitely users that cited much greater trust levels for the OTAs.
Themes that surfaced in areas of Supplier preference included better treatment, better customer service, price matching, and the assumption that booking direct would lead to more upgrades. Users that showed Supplier Preference would often mention bad experiences with OTAs, and assumed that booking with an OTA would be more expensive in the long run, result in worse service, get them into a lesser quality room, or cause other issues with their booking.
In OTA preference, users cited better selection, deals, and better rooms for the prices. They often would call out that the OTA they chose was a trusted brand and in many cases cited recent TV ads as a source for that trust. Some users felt that they would get a better room as a result of booking with an OTA.
This was a fun experiment to work on because it nicely confirmed some assumptions that we frequently hear from the general industry and exposed some new insights for us.
If you have any questions on this experiment or want to see the data split in a different way, please comment below or e-mail us directly.