SMART CITIES

Simulators are used across every aspect of life to test a facsimile of real environments. They are vital to predicting the actions and reactions of complex systems—from flight simulators and movie sets, to weather models and economic forecasts. The IoT is no different.

The complexity of deployments and the number of IoT devices in a typical network are on the rise. This makes the chance of an unanticipated problem that could wreak havoc more likely. The flexibility and scalability that make IoT deployments attractive also make them difficult to test at scale. This is one of the complications that can doom a new project.

A  Cisco report states that 60 percent of IoT projects stall at the proof-of-concept stage. Only a fraction are considered a complete success. Reports like this speak to the difficulty of moving from a system that works on paper to one that achieves its full potential. Simulating various components of the network before bringing the entire system online is one of the best ways to ensure a successful project.

Simulation is important when the number of connected devices or their distribution grows beyond a few rooms or handful of products. And it’s far more cost-effective to simulate an extensive network than it is to actually build one. Testing rare scenarios, fault tolerance evaluation, or measuring end-to-end latency are additional use cases.

The chance of encountering an unexpected event grows as the number of sensors increases. By simulating these events in advance, operators can ensure they are dealt with appropriately.

MQTT Protocol: IoT Friendly

Simulators come in all shapes and sizes—both commercial software and open-source alternatives. They can be differentiated by a number of factors such as scalability and supported protocols.

One popular protocol is MQTT (Message Queuing Telemetry Transport), a publish-subscribe messaging system. It is prevalent for IoT networks because it’s lightweight, with a small code footprint. It also works well in areas where bandwidth is limited.

Applications like the MIMIC MQTT Simulator from Gambit Communications are designed to create a large environment of sensors, actuators, and other MQTT clients.

An IoT simulator can focus on a specific messaging protocol, like MQTT, or support multiple protocols, including CoAP, REST, and AMQP. Picking the right one requires first understanding what kind of simulation is needed and which aspect of the total solution is under test.

Simulating the architectural behavior of individual endpoints is different from that of a client-broker model. The MQTT standard relies on this model for passing messages across a network. Gambit treats the MQTT as an interface to be simulated. This enables it to effectively represent any device that uses the protocol and then scale up the total number to whatever level is required.

Simulation at Scale

In some environments a simulator can help you understand subtle differences in how protocol support has been implemented by various brokers.

When Gambit tested MQTT 5.0 support with a number of services, it found several scenarios that could have caused unexpected behavior in a production environment. Testing the applications against a simulator before deployment helps prevent unwelcome surprises.

These components can be programmed to interact in different ways, controlled by the end user. Operators can scale simulations up and down, program specific events, and measure the impact of these changes on the platform under test in real time.

The MIMIC Simulator is capable of positioning hardware based on the Intel^® IoT Gateway platform as a Device Under Test (DUT), and can create thousands of virtual sensors. Each sensor can be defined as a different type, such as temperature, pressure, and location. They can also be programmed to predictably generate unique traffic, or to simulate various failure modes.

MIMIC supports dynamic rules that can adjust a simulation mid-run and introduce new parameters. This is important for evaluating how an analytics engine behaves under real-world conditions when confronted with unexpected behavior. This could be malfunctioning sensors, sudden failure of a group of sensors, or radio interference. Tools that support only static rules can’t be adjusted to test these kinds of scenarios (Figure 1).

Performance Testing and Edge-Case Evaluation

The Gambit solution can be used to examine how other IoT components perform as the number of connected devices grows. Can the customer’s analytics engine scale effectively to meet current or future demand? Can it handle unexpected corner cases in an appropriate way?

Simulating different arrangements of publishers and subscribers or significantly increasing the total number of connections are good ways to benchmark the flexibility and performance of an IoT solution.

The video below (Video 1) shows the effect on end-to-end latency when 10,000 sensors are being brought online.

As the number of sensors increases, so does the total (simulated) traffic load. Overall latency begins to creep up. A number of variables can impact latency. These include the distance between sensors and brokers, the message payload size, QoS settings, and more. An IoT simulator should provide the ability to adjust these settings as needed.

The IoT, as an industry, is still in its infancy. Simulators were initially designed for cloud testing, because that’s where the majority of IoT processing was happening. The shift to edge computing is driving the need for IoT simulators to add features that integrate with edge services. Gambit has focused on adding these capabilities to its MQTT simulator to ensure this growing test area is also thoroughly covered.