MQTT Protocol: Lightweight Messaging for IoT and Real-Time Systems

MQTT Fundamentals

MQTT (Message Queuing Telemetry Transport) was designed in 1999 for satellite telemetry links — bandwidth-constrained, high-latency, unreliable connections. Those constraints make it ideal for IoT sensors, mobile clients, and any system where connection quality is unpredictable.

The core of MQTT is a publish/subscribe model mediated by a broker:

Sensor (Publisher)                 Broker                 Dashboard (Subscriber)
       |--- PUBLISH temp/sensor1 24.3 --->|                          |
       |                                   |--- PUBLISH temp/sensor1 24.3 -->|
       |                                   |                          |
       |                                   |<-- SUBSCRIBE temp/# ----|  (wildcard)
       |--- PUBLISH temp/sensor2 21.1 --->|--- PUBLISH temp/sensor2 21.1 -->|

Publishers and subscribers are decoupled: they do not know about each other. The broker routes messages based on topics — hierarchical strings like home/livingroom/temperature or factory/line1/machine3/vibration.

Topic Wildcards

MQTT supports two wildcard characters for subscriptions:

• + — single-level wildcard. home/+/temperature matches home/livingroom/temperature and home/bedroom/temperature but not home/livingroom/sensor/temperature.

• # — multi-level wildcard (must be last). home/# matches everything under home/, including home/livingroom/temperature/celsius.

import paho.mqtt.client as mqtt

def on_connect(client, userdata, flags, rc):
    print(f'Connected with result code {rc}')
    client.subscribe('home/#')        # All home topics
    client.subscribe('factory/+/status')  # All machine status topics

def on_message(client, userdata, msg):
    print(f'{msg.topic}: {msg.payload.decode()}')

client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message
client.connect('broker.example.com', 1883)
client.loop_forever()

Broker Architecture

Popular MQTT brokers:

QoS Levels

MQTT's Quality of Service levels determine message delivery guarantees between client and broker (note: QoS applies to each hop independently — publisher→broker and broker→subscriber can have different QoS levels):

QoS 0 — At Most Once (Fire and Forget)

The publisher sends the message once. No acknowledgment. No retry. If the broker is down or the network drops, the message is lost. Best for: sensor telemetry where frequent updates make individual loss acceptable (temperature every 5 seconds).

Publisher ---- PUBLISH (QoS 0) ----> Broker  (no ACK, no retry)

QoS 1 — At Least Once

The publisher stores the message and retransmits until it receives a PUBACK from the broker. The broker delivers to subscribers at least once. Messages may be delivered multiple times if ACKs are lost. Subscribers must be idempotent.

Publisher --- PUBLISH (QoS 1, msgId=42) --> Broker
Publisher <-- PUBACK (msgId=42) ----------- Broker

QoS 2 — Exactly Once

A four-way handshake guarantees exactly-once delivery. The publisher waits for PUBREC, then sends PUBREL, then waits for PUBCOMP. No duplicates, no loss. Use for: financial transactions, command execution, state changes where duplicates cause real harm. Cost: 4× the network round trips of QoS 0.

Publisher --- PUBLISH (QoS 2, msgId=99) --> Broker
Publisher <-- PUBREC (msgId=99) ----------- Broker
Publisher --- PUBREL (msgId=99) ----------> Broker
Publisher <-- PUBCOMP (msgId=99) ---------- Broker

Choosing QoS in practice:

• Metrics and telemetry → QoS 0 (loss tolerable, high frequency)
• Notifications and alerts → QoS 1 (delivery important, idempotent OK)
• Commands and transactions → QoS 2 (exactly once mandatory)

Advanced Features

Retained Messages

A retained message is stored by the broker and immediately delivered to any new subscriber. This solves the 'first message' problem: without retention, a new subscriber must wait for the next publish to see the current state:

# Publish with retain=True — broker saves the last message
client.publish('device/thermostat/setpoint', '21.5', qos=1, retain=True)

# Any new subscriber immediately receives '21.5' without waiting
# Perfect for: current state, configuration, device status

To clear a retained message, publish an empty payload to the same topic:

client.publish('device/thermostat/setpoint', '', qos=1, retain=True)

Last Will and Testament (LWT)

LWT lets a client register a 'death notice' with the broker at connection time. If the client disconnects ungracefully (network failure, power loss, crash), the broker publishes the LWT message automatically:

client = mqtt.Client(client_id='device-001')
client.will_set(
    topic='devices/device-001/status',
    payload='offline',
    qos=1,
    retain=True,
)
client.connect('broker.example.com', 1883)

# On clean connect, publish 'online'
client.publish('devices/device-001/status', 'online', qos=1, retain=True)

This pattern — retained status topic + LWT — gives you real-time device presence tracking for an entire fleet without any application-level heartbeat logic.

Shared Subscriptions

MQTT 5.0 added shared subscriptions, enabling load balancing across multiple subscribers without each receiving every message:

# Three worker processes all subscribe to the shared group
# Each message is delivered to exactly ONE worker (round-robin or random)
client.subscribe('$share/workers/sensors/#')

MQTT over WebSocket

Browsers cannot make raw TCP connections, so MQTT runs over WebSocket for browser clients. The MQTT protocol is identical — only the transport changes:

// Browser: MQTT.js over WebSocket
import mqtt from 'mqtt';

const client = mqtt.connect('wss://broker.example.com:8883/mqtt', {
  clientId: `browser-${Math.random().toString(16).slice(2)}`,
  username: 'user',
  password: 'secret',
  keepalive: 60,
});

client.on('connect', () => {
  client.subscribe('dashboard/#', { qos: 1 });
  client.publish('presence/browser', 'online', { retain: true });
});

client.on('message', (topic, payload) => {
  const data = JSON.parse(payload.toString());
  updateDashboard(topic, data);
});

The broker must have WebSocket support enabled (Mosquitto example):

# /etc/mosquitto/mosquitto.conf
listener 1883
protocol mqtt

listener 8883
protocol websockets
cafile /etc/letsencrypt/live/broker.example.com/chain.pem
certfile /etc/letsencrypt/live/broker.example.com/fullchain.pem
keyfile /etc/letsencrypt/live/broker.example.com/privkey.pem

MQTT 5.0 Features

MQTT 5.0 (2019) introduced several features that close gaps in MQTT 3.1.1:

Reason codes — every PUBACK, SUBACK, and DISCONNECT now carries a reason code (similar to HTTP status codes). Previously, failure was silent.

User properties — arbitrary key-value metadata on any packet. Enables correlation IDs, content-type headers, and routing hints without wrapping payloads.

Request/response pattern — a client can set Response-Topic and Correlation-Data on a PUBLISH. The responder sends its reply to the response topic, enabling synchronous RPC patterns over MQTT:

# MQTT 5.0 request/response (Python paho-mqtt 2.0)
import paho.mqtt.client as mqtt
import paho.mqtt.properties as properties

props = properties.Properties(properties.PacketTypes.PUBLISH)
props.ResponseTopic = 'replies/client-001'
props.CorrelationData = b'req-42'

client.publish('commands/device-007/reboot', '', properties=props)

Topic aliases — assign a short integer alias to a long topic string. Reduces per-message overhead significantly when publishing to the same topic thousands of times per second.

Session expiry interval — controls how long the broker retains session state (subscriptions, queued QoS 1/2 messages) after disconnect. Set to 0 for ephemeral browser sessions, or hours/days for mobile devices that reconnect after sleep.