MQTT + Kafka + OPC UA: A Modern OT/IT Data Backbone

The convergence of OT and IT is accelerating, and data pipelines must now handle both real-time control data and enterprise analytics streams. The combination of MQTT, Kafka, and OPC UA is emerging as the new reference architecture for scalable, event-driven manufacturing systems.

Why Combine These Technologies

• OPC UA handles structured, secure communication with machines.
• MQTT efficiently moves small, frequent messages across networks.
• Kafka manages long-term streaming, replay, and analytics workloads at enterprise scale.

Architecture Overview

The pipeline looks like this:

[PLC/Robot] -- OPC UA --> [Edge Gateway] -- MQTT --> [Broker] -- Kafka Connect --> [Data Lake or Cloud]

Each component plays a distinct role: OPC UA ensures reliable machine-level data, MQTT bridges the edge, and Kafka provides the durable event backbone for IT systems.

Integration Patterns

• Edge to Cloud: MQTT brokers forward data to Kafka via connectors or REST proxies.
• Backpressure Handling: Kafka’s buffering prevents overload when OT networks slow down.
• Schema Registry: Avro or Protobuf definitions maintain consistent data typing.

Security and QoS

Each layer adds reliability and control:

• MQTT QoS 1–2 ensures message delivery confirmation.
• Kafka replication maintains durability.
• OPC UA certificates secure machine access.

Case Example: Electronics Manufacturer

An electronics factory connected 900 PLCs using OPC UA at the edge. MQTT forwarded production events to a Kafka cluster, which fed Power BI dashboards and AI quality models. Data latency dropped from 5 seconds to 300 ms end-to-end.

Related Articles

• Unified Namespace in Practice: Topic Taxonomy That Scales
• Building an Enterprise UNS: Naming, Ownership, and Data Governance
• UNS Anti-Patterns: How Good Intentions Create Data Swamps

Conclusion

MQTT, Kafka, and OPC UA together form a resilient and flexible OT/IT backbone. This hybrid architecture bridges deterministic machine data with scalable enterprise analytics — unifying operational visibility from shop floor to boardroom.