News

Telemetry buffering on IoT gateways during connectivity loss

A practical matrix for selecting what an IoT gateway stores, evicts, and retransmits after a connection loss, considering MQTT 5.0 and Sparkplug B standards.

Taras Zabolotnyi 20.10.20267 min read

Building Management Systems (BMS) and IoT integration architects face the challenge of maintaining telemetry data integrity—from temperature and humidity sensors to energy meters—during network interruptions. The reliability of data transmission from edge gateways to cloud platforms or central systems depends on the careful configuration of the gateway's telemetry buffer. Without proper attention to this aspect, data may be lost or transmitted out of sequence, leading to corrupted event logs and incorrect system states.

MQTT QoS limitations: why it is not enough for gateway buffering

The MQTT protocol offers three Quality of Service (QoS) levels: 'at most once' (QoS 0), 'at least once' (QoS 1), and 'exactly once' (QoS 2). While QoS 1 and 2 provide protocol-level acknowledgment mechanisms between client and broker, they do not inherently define local telemetry storage during prolonged gateway disconnection. Therefore, an architect must combine MQTT settings with custom application-level policies for buffering, ordering, and deduplication.

MQTT 5.0 introduced mechanisms for better session and message management. One such mechanism is the Session Expiry Interval. This parameter, set by the client during connection, determines how long the MQTT broker retains session information (subscriptions and queued messages) after disconnection. If set to zero, the session is deleted immediately. Session behavior is detailed in the OASIS MQTT 5.0 specification.

There are two session types: Clean Start = true (MQTT 5.0) means the broker does not retain any session information after disconnection. A new session is created upon every reconnection. Clean Start = false (MQTT 5.0) allows the broker to retain session state, including subscriptions and unacknowledged messages, after disconnection. This enables the client to resume a previous session upon reconnection. However, the duration of a Persistent Session on the broker is limited by the Session Expiry Interval.

Another mechanism is the Message Expiry Interval. Clients can set this interval for each PUBLISH message, specifying how long the broker should store the message for offline subscribers or as a retained message. If not defined, the message has no expiration and is stored by the broker until delivery or deletion according to other session rules. Further information: OASIS MQTT 5.0 specification.

Despite these improvements, they primarily concern broker behavior. An edge gateway, as a client, still requires its own buffering strategy to ensure that data collected from devices (e.g., sensor readings) is not lost if the connection to the broker is interrupted or if the broker cannot store the session long enough. Even with these mechanisms, if the gateway loses connectivity entirely and cannot send messages to the broker, data collected from devices may still be lost unless the gateway maintains its own persistent buffer.

Sparkplug B: advanced mechanisms for context and order preservation

Sparkplug B is a specification built on top of MQTT that standardizes topic namespaces, payload formats, and state management. It resolves interoperability issues arising from the flexibility of pure MQTT. More details: Eclipse Sparkplug specification.

Key Sparkplug B mechanisms include:

BIRTH messages: Upon connecting an edge node to an MQTT broker, it publishes an NBIRTH (Node BIRTH Certificate) message containing a full definition of all metrics (tags), their data types, current values, and metadata. For every device managed by the edge node, a DBIRTH (Device BIRTH Certificate) message is published. This allows any subscribing application to immediately obtain the full schema and current state of all devices in the network. See the Eclipse Sparkplug specification.
State Management: Sparkplug B defines a clear lifecycle for edge nodes and devices. Host applications publish their own state (ONLINE or OFFLINE) as a retained message. Edge nodes can be configured to wait until their primary host is ONLINE before publishing data, preventing data loss while the host is unavailable. See the Eclipse Sparkplug specification.
Sequence Numbers: Each message from an edge node (except NDEATH) contains a sequence number that increments from 0 to 255. A jump in the sequence indicates lost or out-of-order messages, which may prompt the host application to request a 'rebirth' to resynchronize the state. See the Eclipse Sparkplug specification.

Buffer management strategies: volume, priority, and duplicates

An edge gateway requires a local buffer if telemetry must survive a connection loss to the broker. Storage for such a buffer should be chosen based on a verifiable requirement: after a controlled process restart or a full gateway reboot, records not yet acknowledged by the central system must remain available for retransmission.

Storage limits

Gateways have limited storage resources. It is necessary to define the maximum buffer size and the eviction policy for when the buffer overflows. Popular policies include:

FIFO (First-In, First-Out): The oldest messages are deleted first.
LIFO (Last-In, First-Out): The newest messages are deleted first (less desirable for telemetry).
Priority Eviction: Low-priority messages are deleted first to free up space for more critical data.

Monitoring the buffer file size in production and setting up alerts for abnormal growth is critical.

Data integrity

During network outages, ensuring data integrity is a top priority. NIST recommendations for IoT device cybersecurity emphasize the importance of resilience and data integrity. If data is buffered on disk, the security of this storage must be ensured. See NIST SP 800-213: IoT Device Cybersecurity Guidance.

Duplicate handling

Upon connection recovery or when using QoS 1 in MQTT, duplicate messages may occur. Strategies for duplicate handling at the application level or through protocol mechanisms are essential. Sparkplug B, thanks to the use of sequence numbers, helps detect and ignore duplicates or out-of-order messages.

Connection recovery: how the gateway retransmits accumulated data

After network connection recovery, the edge gateway must initiate the process of retransmitting accumulated data and synchronizing state. Important aspects of this process include:

Automatic reconnection: The gateway should automatically attempt to reconnect to the MQTT broker, preferably using an exponential backoff algorithm to avoid broker overload.
Retransmission of buffered data: After successful reconnection, the gateway begins sending accumulated messages from the buffer. It is important that this process occurs in order if critical to the application.
State synchronization: Sparkplug B plays a key role in state synchronization. Upon reconnection, the edge node publishes new NBIRTH and DBIRTH messages, providing the full current state of all its metrics. Host applications can request a 'rebirth' to clear their cache and restore the full state. See the Eclipse Sparkplug specification.
Keep Alive interval: The MQTT Keep Alive parameter allows the gateway and broker to detect connection breaks. A shorter Keep Alive interval speeds up disconnection detection and the start of buffering but increases network traffic. See the OASIS MQTT 5.0 specification.

The Retain flag in MQTT stores the last message for a specific topic on the broker and does not replace the telemetry log on the gateway. In Sparkplug, full state synchronization relies on BIRTH messages, so retain should not be used as a workaround for local buffer policy. This is a separate solution from session duration and message life expectancy.

Telemetry buffer strategy selection matrix

Criterion / Strategy	Strategy 1: Maximum Retention	Strategy 2: Critical Data Priority	Strategy 3: Minimal Buffering
What to store on the gateway?	All telemetry data.	Only critical high-priority data.	Minimal volume, only the latest data.
Buffer limits (eviction policy)	Large persistent buffer; FIFO on overflow.	Medium persistent buffer; priority eviction.	Small buffer; rapid eviction (LIFO or FIFO).
Out-of-order messages	Order preserved locally; Sparkplug Sequence Number detects gaps during retransmission.	Order preserved for critical data; Sparkplug Sequence Number detects gaps.	High risk of order loss; Sparkplug Sequence Number helps detection.
Loss of device context/state	Minimized by persistent local buffer; full state restored via NBIRTH/DBIRTH.	Critical data context kept locally; full state restored via NBIRTH/DBIRTH.	Higher risk of history loss; current state restored via NBIRTH/DBIRTH.
Duplicate handling	Stable record ID and acknowledgment check; sequence number helps detect gaps.	Same rules for critical data; retransmission does not create a new business record.	Explicit deduplication before writing; QoS 0 also accepts loss risk.
Connection recovery (speed, data integrity)	Full retransmission from buffer; synchronization via BIRTH.	Fast retransmission of critical data; synchronization via BIRTH.	Fast recovery, but significant data loss possible; synchronization via BIRTH.
QoS requirements (MQTT)	QoS 1/2 for all data.	QoS 1/2 for critical data; QoS 0 for less important data.	QoS 0 for most data.
State retention requirements (Sparkplug)	Full STATE recovery via BIRTH; Host STATE (OFFLINE/ONLINE).	Priority STATE recovery via BIRTH; Host STATE (OFFLINE/ONLINE).	Basic STATE recovery via BIRTH.

A rigorous approach to telemetry buffer management at the edge gateway is a fundamental aspect of ensuring business process continuity and data reliability in IoT systems. Understanding the nuances of MQTT and Sparkplug B, as well as proactive planning for buffering and recovery strategies, allows architects to build resilient and reliable solutions capable of withstanding unpredictable network failures.