AI • IOT • NETWORK EDGE

Businesses of all types struggled through the pandemic, but public venues like museums were taken completely out of commission. This lasted for more than a year in some places, and even after restrictions lifted, these facilities had to remain clean and safe while patrons overcame their suspicion of public spaces.

As we’ve all come to learn, one of the many paradoxes of pandemic life is that human staffers who once assisted guests can now be a deterrent. This has left public spaces like the Ontario Regiment Museum with no choice but to turn to technology for answers. And the response so far is familiar, but not at all what you’d expect (Video 1).

Meet Master Corporal (MCpl) Lana. She is a health screener who greets visitors, as well as a docent capable of providing information about the museum and its events. She also happens to be an AI.

Or, more accurately, an Animated Virtual Assistant (AVA).

The Anatomy of an Animated Virtual Assistant

Developed by Canadian high-tech startup CloudConstable, MCpl Lana is just one of several customizable AVA characters originally designed to improve user experiences in smart city, healthcare, entertainment, and other high foot traffic use cases. When the pandemic hit, those use cases expanded to include touchless health screening and contact tracing in all types of public spaces.

“We’ve got the largest collection of operational military vehicles in North America,” says Dan Acre, Operations Manager at the Ontario Regiment Museum. “And when I say operational, they drive. We’ve got tanks from World War II to the present day.

MCpl Lana is just one of several customizable AVA characters originally designed to improve user experiences in #SmartCity, #Healthcare, entertainment, and other high foot traffic use cases, @CloudConstable via @insightdottech

“One of the ways we get an important part of our revenue is doing shows on a monthly basis,” he continues. “We have a lot of people come, pay to see the show, go to the gift shop, buy rides on the vehicles. That was our normal way of operating, but during COVID that was extremely restricted by the number of people that could come.”

With a limited number of people allowed in the museum at any given time, MCpl Lana was redeployed from her original role as a museum guide to assist with the check-in process. This meant using the AVA’s sophisticated perception sensing suite comprising an LWIR thermal scanner for temperature screening, microphones that connect to cloud-based speech engines, and two Intel^® RealSense^™ cameras for anonymized people counting, facial, and gesture recognition (Figure 1).

The RealSense cameras allow Lana to count guests, scan tickets or QR codes, and capture “air touch” inputs so users can make selections without contacting surfaces. Images and video taken by the cameras feed into a nearby Intel^® NUC 9 Pro, where AI inferencing models developed and optimized using the OpenVINO^™ Toolkit are executed in real time to enable even more-advanced functionality.

“If you’re a member of staff and we know who you are, she can actually recognize you by facial recognition, greet you, and run through the required screening questions. And once you’re done, welcome you and say, ‘Thanks for coming. Have a good day!’” explains Michael Pickering, President and CEO of CloudConstable.

“There’s a head pose model as well that we use to know where you’re looking. We’re trying to detect nodding to say ‘Yes’ or shaking your head to say ‘No,’” he continues. “MCpl Lana wants to know if you’re off to the right or left and looks towards you to try to make virtual eye contact. We want to figure out the main person she’s interacting with, and presumably that’s somebody looking at the Animated Virtual Assistant.”

In addition to vision processing, the NUC 9 Pro’s Core^™ processor features Intel^® Active Management Technology that permits secure remote management of the AVA platform so CloudConstable engineers can maintain and upgrade the system without having to be physically on-site or train museum staff.

“When COVID came out, it was a perfect system to do screening, ask questions, eventually take the temperature of the people with infrared sensors, and the system was smart enough that when you gave a couple of negative answers, you couldn’t come in,” Acre says.

Animated Virtual Assistants Take the Field

Obviously, the adaptability of the AVA platform from use case to use case is one of its core strengths. Not only can the animated character’s appearance be modified to fit the setting, but what and how it communicates with users can be refined as well. At the museum, for example, MCpl Lana has been tailored to respond to frequently asked questions like whether the gift shop charges sales tax or where the bathrooms or specific exhibits are.

But the museum is also investigating other unique ways to customize the AVA platform for its purposes. These include the introduction of a virtual medal exhibit in which patrons can use the AVA platform to scan Canadian military medals, then have Master Corporal Lana identify and provide historical context about them.

The museum is also working on a massive interactive exhibit that simulates a tank battle run around the actual airfield the museum sits on. As part of this initiative, CloudConstable, in collaboration with architectural design firm Cord Design, is working to contextualize the physical space of the airfield by creating an interactive map you can “drive around on” that includes 3D models of the area’s buildings, surroundings, and topography.

Ultimately, they hope to use the Intel Game Dev AI Toolkit and Gaia ML to populate the map with virtual combatant tanks and create an environment that lets users experience what it’s like to be a tank commander or gunner operating a classic war vehicle. Again, AVA will be there to assist as a guide.

Use Case Flexibility Goes Virtual

In uncertain times, flexibility is paramount. Platforms like CloudConstable’s AVA bring that adaptability to public spaces in an integrated package full of hands-free technology capable of performing a variety of functions that formerly required human staff, and even some tasks even humans can’t perform.

Thanks to off-the-shelf and open-source technology building blocks, flexible AI is now reporting for duty in public spaces everywhere.

This article was edited by Christina Cardoza, Associate Editorial Director for insight.tech.