Booking Holdings CFO says even AI hyperscalers don’t know their ROI — but he’s learning from his ‘AI coach’

Booking Holdings CFO Ewout Steenbergen suggests that even the AI-powered giants are uncertain about the return on investment from their large language model development. At the Fortune AIQ Summit, he stated that these companies may have assumptions and hypotheses, but they are unsure about the actual ROI, which is a positive sign for others.

Companies like Booking that utilize these models rather than creating them pay less, provided they monitor expenses. Steenbergen explained to Fortune's Sheryl Estrada that AI will make travel easier for customers and cut marketing costs for the company. Currently, these gains are minimal, with referrals from AI accounting for under 1% of total room nights.

However, significant improvements are taking place internally. In customer service, bookings have increased in the high-single digits while costs have slightly decreased, leading to a substantial reduction in cost per booking. In engineering, the company's 9,000 engineers are producing 30% more code that passes testing and quality control.

To control AI spending, Booking employs an "effective model cost routing" system where basic and open-source models handle simple tasks, while more expensive models tackle complex ones. The hotel industry is unlikely to be fundamentally changed by AI, but the customer experience will evolve. Currently, about five platforms are typically visited before booking, and AI could streamline this process and manage disruptions such as flight delays affecting reservations.

AI tools have shown modest gains for Booking, with customers booking slightly faster, converting at higher rates, and canceling less often. The company's ultimate aim is growth, not cost reduction. Steenbergen shared an example of using AI to proactively adjust travel plans if bad weather is forecast. This could lead to more loyalty, trust, and brand value for Booking.

The company is also seeing growth in connected trip transactions, where customers book multiple travel verticals for the same trip, increasing at a low-double-digit rate at Booking.com. Additionally, merchant bookings have reached 73% of gross bookings, and level 2 and level 3 "Genius" members make up more than 30% of active customers.

Steenbergen is also benefiting from an "AI coach" and two agents who assist him with board presentations, strategic plans, and earnings call preparation.