QSRs: Vision AI

QSRs: Want a Side of Vision AI with That?

January 25, 2023

Christina Cardoza

Vision AI

When you think of AI, you don’t typically think of restaurants. But food service was one of the industries most disrupted by the pandemic and its aftermath. The rise of third-party delivery services has also had an impact on the situation, where a mismanaged meal delivery can damage the reputation of the restaurant as well as that of the driver. And it turns out that AI has a lot to offer this new culinary landscape.

Our guest is Hauke Feddersen, VP of Operations at the smart software automation provider PreciTaste. He’ll talk about the challenges of QSR kitchens that are pressure cookers at the best of times, and how edge technology and vision AI can create efficiencies there that lead to fresher food reaching customers, faster. Because, as Feddersen reminds us, with QSRs “it’s not the food that is fast, it’s service that is fast.”

What’s driving the current demand for AI in the food service industry?

I’m a firm believer in the fact that there can only be a solution if there’s a problem. And this industry is facing problems. There is a strong, unmet demand for labor, and a lot of labor churn. And the churn brings with it the fact that a lot of established best practices, a lot of established know-how, tends to get lost.

The demand patterns have also shifted since the beginning of the pandemic, which makes it very difficult for operators in the kitchen because they have such limited data. Their only window into the reality out in the restaurant is the KDS, the kitchen display system. The KDS tells them what has been ordered in the past, but they don’t have a system that forecasts what will happen next.

So we are turning restaurants into data-driven operations. AI is incredibly good at solving an equation with almost unlimited variables—whether it’s traffic patterns, historical sales, the sales of the last hour, the sales of the last days—to predict demand better than any human could. This helps the kitchen crew by taking the cognitive load off them, and making sure that the individual station basically just has to do what’s on the screen.

And there is so much more delivery now. All of a sudden the customer that used to be standing in front of you is somewhere else entirely, waiting for a delivery. That puts huge implications on order accuracy. “You forgot the Happy Meal for my kids!” “My apologies. And here’s an extra Happy Meal toy for you.” Everybody is happy. But you can’t do that if the customer is 10 miles away and the food has just been delivered. You have to get it right the first time.

In 2020 PreciTaste launched an auto-accuracy verification tool to go with its QSR brain platform. Cameras mounted to the ceiling see what is happening in the restaurant and can see what is being added to that bag, so they can see if the Happy Meal toy has been put into the bag or if it is missing, and they can see that the correct bag is handed out the window to the correct customer or delivery driver.

“We are turning restaurants into #data-driven operations. #AI is incredibly good at solving an equation with almost unlimited variables” – Hauke Fedderson, @PreciTaste via @insightdottech

How else are third-party delivery services affecting restaurant operations?

This situation changes how the customer is perceived by the restaurant. The delivery customer is not necessarily your customer, and you don’t know that person; they are anonymized by the platform. When you know your customer and have a direct interaction with them, you own the entire customer experience from order to delivery. Now, all of a sudden, restaurants are just one part in the middle of the transaction. But when things go wrong, the feedback to them is still prompt, and it’s expensive. Refunds for inaccurate orders to platforms like Uber Eats are very, very severe.

The kitchen is already a stressful environment, and it’s sometimes close to magic that teams are able to churn out the amount of meals they do in one hour, and have them all delivered and all reach the right customer. So the best thing that we can do to help that is to reduce that stress, reduce the cognitive load, make sure that the processes flow and that inventory is available at all times so that this very well-oiled machine doesn’t have to stop.

Talk about the investments in technology needed to implement these AI solutions.

This is a nickel-and-dime business, and there is not a lot of money to be wasted, therefore the investments need to be targeted and solution oriented. The KPI improvements must be real for the customers.

We are strong believers in edge AI. Everything we do runs on small form factor computers like the Intel^® NUC. And one of the main reasons is price. We intend to have the fully fledged solution installed for between $2,000 and $5,000 max, including all the cameras, all the edge devices, and all the networking kit that is required.

And if security cameras are already installed that are TSP compliant—meaning IP cameras—then we absolutely love using existing video streams. Vision AI does not need perfect imagery; very few pixels are actually sufficient to run very sophisticated models. Our model is: What the human eye can see, we can teach the computer to see. As soon as that data is digitalized, we upload it into the brain part of our edge AI installation, and that then makes predictions based on what it has seen.

So the role of edge AI in this is very important for multiple reasons. First: cost. Cloud-AI platforms tend to be very, very expensive over time. Second: the seamless integration and the low-latency inference that we get from these devices independent of internet. Even if you cut the internet, our solution will continue to run.

And the third, very important aspect: managing PII, personal identifiable information. We mount the edge device that captures data from an ordinary security camera only a few feet away from the camera. The vision data, the PII part, is thrown away immediately, and the only thing left is: There are six people waiting to order; there are 12 cars in the drive-through and two of them have ordered already.

Can you share any PreciTaste use cases?

My favorite one is Chipotle. Chipotle runs an amazing operation—it’s scratch kitchen at its finest. Raw ingredients being cut, being spiced, being marinated, being cooked in the restaurant itself; they start with raw avocados and raw tomatoes in the morning to make their delicious guacamole. Our solution is about inventory sensing at the front-of-house makeline, as well as the digital makeline in the back of house that is used for delivery orders. It is always sensing how much inventory is present, how fast the inventory is depleting, and then advising the crew on what to cook next, and when.

For example, the chicken process, just because it’s so artisanal, so scratch kitchen, takes them 25 minutes from the instruction to the crew member, “Please make chicken now,” to the chicken hitting the front-of-house makeline. So you have to know 25 minutes in advance when you need to restock. And of course the demand patterns vary throughout the day.

So, at lunchtime a full pan still means you have to cook more right away. An hour later, half a pan means you can leave it for another 10, 15 minutes—because it’s still great food—and please cook something else first; you’ll only need to cook more chicken 20 minutes from now. The AI is very, very good at predicting what will happen, and helping the crew get to the point where they never stock out, but they can serve the freshest food possible.

At the beginning of each project we always have a passive phase, in which we capture how well the restaurant is performing without our help. And then, after we switch on our software suite, we compare that to the benchmark. That’s actually our biggest selling argument, just saying, “This was before, and this is now.” It’s always worked.

What is the value of working with Intel and its technology?

The Intel^® NUC 12th Generation is a powerhouse in the tiniest imaginable form factor, and it is extremely reliable. We can mount them anywhere, even if the restaurant doesn’t have a server closet or a proper office. I just really like working with those devices. Same goes for the Intel^® RealSense^™.

OpenVINO^™ has also helped us. With OpenVINO we can port our models to run on the CPU or the integrated GPU. This unlocks an abundance of devices that we can potentially use, which has been especially important in the last two years during the supply chain crisis for digital components.

Where else do you see AI going within the food service industry?

I think we’ll see a lot more implementation—in front of house that’s visible to the customer, as well as back of house to optimize processes. It’s about producing more with less: more food with a smaller crew, or with the same size crew that now does 20% more.

And we’ll see the technology being specialized, so that those third-party solutions can deliver multiple different food items from one kitchen—you can have Mexican food paired with Italian food, paired with sushi, paired with wings—all delivered by one driver. Then everybody around the table can enjoy the food they want, and they won’t be limited to either all getting the same thing or having to pay three delivery fees.

I also think that our industry will get a lot more precise in its predictions in order to eliminate the stigma that QSR food sits around all day, which some people still associate with these kinds of restaurants. It will be fresher food, and the food can be more interesting if it doesn’t have to be optimized for shelf life. Personally, as a consumer, I’m very excited about that part as well.

Are there any final thoughts you’d like to leave us with?

The thing I like the best is that this is not science fiction. This is technology that is out there today, and it’s improving the lives of customers already, unbeknownst to them. What I would love to see in the future is a sign or a badge outside a restaurant or in your favorite delivery app that says, “Hey, this restaurants optimizes quality utilizing state-of-the-art vision-AI technology. So, don’t hesitate, don’t worry—you’ll get the best quality from this location every time, because it’s managed by a system that is entirely designed to do just that.”