This article describes monolithic architecture and consist of high level diagram, description of data flow between various components and some architecture choices made.
Please note that NDU v2.2, the platform supports microservices deployment mode. Although microservices option is preferable for highly-available and horizontally scalable scenarios, many NDU customers find it useful to be able to start with a single NDU instance and scale in the future.
We also recommend to use this mode for development and prototyping.
In the monolithic mode, all NDU components are launched in a single Java Virtual Machine (JVM) and share the same OS resources. Since NDU is written in Java, the obvious advantage of the monolithic architecture is minimization of required memory to run NDU. You can launch and run NDU process with 256 or 512 MB of RAM in a constrained environment. The obvious disadvantage is that if you overload one component with messages, like MQTT transport, it may impact other components as well. For example, if the limit of OS for your NDU process is 4096 file descriptors, you can not open more then 4096 MQTT sessions from device and websocket user sessions in parallel.
NDU provides MQTT, HTTP and CoAP based APIs that are available for your device applications/firmware. Each of the protocol APIs are provided by a separate server component and is part of NDU “Transport Layer”. The full list of components and corresponding documentation pages are listed below:
Each of the transport components pushes data to the rule engine and also may use core services to issue requests to the database to validate device credentials, etc.
Since NDU uses very simple communication protocol between transport and core services, it is quite easy to implement support of custom transport protocol, for example: CSV over plain TCP, binary payloads over UDP, etc. We suggest to review existing transports implementation to get started or contact us if you need any help.
NDU rule engine is responsible for processing the incoming messages with user defined logic and flow. You can lear more about the rule engine using corresponding documentation page.
Core services are responsible for handling:
NDU node uses Actor System to implement tenant, device, rule chains and rule node actors. Platform nodes can join the cluster, where each node is equal. Service discovery is done via Zookeeper. NDU nodes route messages between each other using consistent hashing algorithm based on entity id. So, messages for the same entity are processed on the same NDU node. Platform uses gRPC to send messages between NDU nodes.
Note: NDU authors consider moving from gRPC to Kafka in the future releases for exchanging messages between NDU nodes. The main idea is to sacrifice small performance/latency penalties in favor of persistent and reliable message delivery and automatic load balancing provided by Kafka consumer groups.
It is possible to push messages from NDU to external systems via the Rule Engine. You can push data to external system, process data and report the results of the processing back to NDU for visualization. Please review rule engine documentation and guides for more details.