Intelligent AI

Bringing Intelligent AI into the Physical World

September 16, 2022

Christina Cardoza

Intelligent AI

The ongoing supply chain crisis has brought the workings of the nation’s cargo transportation system out of the shadows and into the mainstream news. It’s pretty apparent that a system that had functioned well for decades was not exactly up to the challenge the past couple of years threw at it. Modernization, digital transformation, and AI innovation was—and still is—desperately needed, or the whole country feels the consequences.

That’s where Scott Chmiel, Business Development Manager for Cloud Solutions at Arrow, a technology solutions provider; and Steen Graham, Co-Founder and CEO of Scalers.ai, come in. Between their two companies, they help customers navigate the intelligent IoT partner ecosystem—and not only in the environment of smart ports. Deploying AI in the physical world is applicable to all kinds of industries. And the benefits go beyond business to have a ripple effect on society at large.

What challenges do businesses currently face in their digital transformation efforts?

Scott Chmiel: The challenges have changed because the complexity of solutions has increased so much. In the past, everything was contained in a single piece of hardware or software, but now we’re adding cloud, we’re adding complexity, we’re adding technologies that not only require more from a tech standpoint but different skill sets from a development standpoint. Solutions now have to be integrated and deployed into existing customer environments that differ from one to another. Connected devices now require additional operational security. And, obviously, we can do things that weren’t possible before, such as machine learning and AI. It’s possible to solve business problems that we couldn’t even address in the past.

Steen, what is your perspective on these efforts?

Steen Graham: The challenge is deploying artificial intelligence and IoT in the physical world. Take the situation of a port. Obviously ports, and the infrastructure for ports, have been around for decades, and there are various existing applications that are working just fine there, but then you want to implement new technology. So how do you actually deploy these cloud-native methodologies—including artificial intelligence—on the existing infrastructure to do things like analyze efficiency and monitor CO₂ emissions? Combining existing infrastructure with new infrastructure, from both a hardware and a software perspective, is critical to driving industry transformation and addressing the challenges in our supply chain.

The current federal government administration has been fantastic in supporting port modernization. But, interestingly, ports are actually managed by their local municipalities, so what those local leaders do has impacts on a national scale. Unions are also critically important to the situation. For example, one of the port jobs that has been sustained in the United States is crane operations. What we’ve automated is the front-end part—removing the containers from the ship—but we still have heavy investment in these human-performed, union-based roles in loading and unloading the trucks. So those three parties: The federal government, the local municipalities, and the unions are all incredibly important in this current crisis.

How do businesses go about making impactful technology changes?

Scott Chmiel: The first step is understanding what business outcome they’re seeking. What are they trying to accomplish, and who are the stakeholders? In the example of the Port of Los Angeles, there’s not just one company; there’s the municipality, the people handling the containers, the truck drivers, dozens if not hundreds of subcontractors—who all have to dance around each other to run the port. Our solution focuses on their challenges around safety, as well as just tracking in and out.

Combining existing #infrastructure with new infrastructure, from both a hardware and a software perspective, is critical to driving #IndustryTransformation and addressing the challenges in our #SupplyChain. @Arrow_dot_com and Scalers.ai via @insightdottech

Steen Graham: To answer the second part of the question, what Scott and I looked at was a no-compromise solution. From a simplistic, operating-system perspective, there are two pervasive operating systems in the world: Windows and Linux. Cloud-native workloads in modern AI applications are written in Linux, whereas a lot of existing workloads and applications have been written in Windows. By adding cross-platform capabilities to some of these technologies we’ve been able to retrofit the AI applications on existing infrastructure to make sure they work better together. Layering on modern cloud-native attributes and AI capabilities was really the approach we used in this particular solution.

What’s driving this cross-platform interoperability?

Scott Chmiel: Often it’s the existing hardware. And the technology, the infrastructure, can be applied to many different solutions—whether it’s a retail application, within a smart port, or in a warehouse—all the same types of challenges are there, and the same technology can be used and customized or repackaged. It brings additional value to the existing hardware they have, and adds value to it with things they couldn’t do before. In the example with the smart port, it was adding safety, and that’s applicable to retail, too: Before a crane moves through a warehouse, you want to make sure wherever it’s going is clear of people who might be in its way.

Steen Graham: From a technical point of view, we were given a gift—notably by Microsoft and Intel^®—with the underlying technology. We use the acronym EFLOW, for Edge for Linux on Windows—or, more accurately, Azure IoT Edge for Linux on Windows. That is what gives us that no-compromise capability across Windows and Linux. And the hidden gem there is that Intel invested in hardware-acceleration capabilities via its integrated-graphics capability that allow us to do these workloads on deployed Intel-based CPUs without having to upgrade to expensive GPS. Now we can run multiple AI models, multiple camera feeds on affordable, off-the-shelf technologies like Intel’s NUC platform, and Windows and Linux as well. It’s an incredible array of technology that allows us to deploy these modern workloads and make sure they’re interoperable with existing infrastructure.

How is EFLOW used in the port example?

Steen Graham: The EFLOW technology was only released late last year, so we are still in the engagement phase. From a business-outcome perspective, the problem that we were trying to address was the bottleneck associated with turn times: the operational-technology metric of how fast containers can be loaded and unloaded. So how do we optimize the turn times of those cranes? How fast can they be loaded and unloaded? How do we make sure the truck is in the right place at the right time? All while providing an enhanced safety experience for the workers on-site. And we are also tracking CO₂ emissions, so another metric we’re looking at is how efficient the hybrid cranes are that many ports are using alongside their diesel cranes.

What other use cases or challenges might EFLOW solve?

Scott Chmiel: There are lots of opportunities: Transportation, industrial, and retail are a few different verticals. I know there’s a strong focus on retail from both Microsoft and Intel: The opportunities are there to do workload consolidation—consolidation of surveillance and point of sale, where one machine could do both. Or there could be new services that couldn’t be done before; once you have a visual element with the transaction, what kind of value can you generate out of that?

The code, the underlying technology, can be repurposed for any of those verticals. A lot of the work has already been done for them with the accelerators and the tools that Microsoft and Intel with OpenVINO^™ have provided.

Steen Graham: Healthcare is another possible industry. If you look at medical-imaging equipment, such as ultrasound, a lot of ultrasound vendors are Windows-based applications, but they’re looking to add new AI-based features. An example is that anesthesiologists occasionally have challenges finding veins in their patients. You could use ultrasound equipment to determine with accuracy the location of the vein. You take existing Windows-based ultrasound equipment, and then overlay modern deep-learning.

We’ve also seen an incredible demand in using computer vision to do defect detection in the manufacturing process, and I think that’s an incredible use case. If you do in-line AI defect detection, you can find the products that are having quality issues earlier in the manufacturing flow. And if you address those problems earlier in the flow, you actually end up using less fossil fuel to run through the rest of the process.

Can you talk about the partnerships that go into this process?

Steen Graham: Arrow is always looking to figure out how it can make one plus one equal three across its partnerships. So Scott came to us with an incredible idea about showcasing the value of this underlying EFLOW technology, and we were able to take technologies from Intel and Microsoft—and a number of open-source projects as well—to build that solution code. Where Scalers comes in is in really understanding how to fit all those things together into a high-fidelity enterprise AI solution, and then providing that solution, as well as building the custom AI models for deployment.

Scott Chmiel: Arrow calls itself an orchestrator and aggregator—whether it’s bringing the different technologies, services, or components together, or helping out with design. It’s hard for one company that has a vision or a challenge to have the all resources or the skill sets in-house to do everything for an end-to-end solution. So what Arrow looks to do is work with that end user and bring in appropriate partners. We help them pick the right solutions, not only for their end use but looking at the longevity, the overall life cycle, of that solution as well. Smart ports—that’s not something that’s going to be deployed and done within the course of a couple of years. And it should also be something that’s repeatable. The company that’s developing that solution, or that is bringing these pieces together, can reuse it and create more scale, create more value across the ecosystem.

Is there anything else we should know about EFLOW or this topic?

Steen Graham: I think as we talk about the cost of development and software engineering, it’s incredibly important that we write the code to integrate these partnerships. There are so many incredible companies with great technologies, but what many times is missing is the single line of code that connects the APIs to really drive transformation. As an industry, we really have to come together on the deployment challenge, because building capabilities in the cloud is fantastic, and it’s really affordable and easy to do these days. Where the challenge occurs is deploying it in the physical world, and the continuous learning, the transfer learning, the continuous annotation requirements to do that.

And, finally, although we’re getting really good at synthetic data and creating AI models with small data sets, if we really want to move society forward, we have to be able to build models with high fidelity on good data sets. And we have to do it with explainable AI, so that we know why it’s making its determinations in order to make sure it’s as inclusive as possible, as well as accurate.

Scott Chmiel: I’m always amazed when I talk to companies in specific verticals—whether it’s somebody running a warehouse, somebody in a port, somebody in surveillance or the medical industry—the amount of knowledge they have about what they do. Their particular solutions are amazing. And as these solutions get more complex, I want to make sure people understand that there’s no need to go it alone. We’re no longer in the days of building a device that does one thing—it’s not just an MRI that does visioning; it’s how it integrates with the whole hospital. But companies don’t need to figure that out alone. And they really can’t do it alone with these more complex solutions. The bar is moving down for what can be done; it’s amazing the business solutions that couldn’t be solved in the past that now can be.