Open-source engine enables high-performance data processing for Internet of Things devices

The Berlin Institute for the Foundations of Learning and Data (BIFOLD) announces the open-source release of NebulaStream, a next-generation stream processing engine built for the unique challenges faced in IoT environments.

A paper titled “NebulaStream: An Extensible, High-Performance Streaming Engine for Multi-Modal Edge Applications,” has been published in the journal Companion of the 2025 International Conference on Management of Data.

The software is freely available under an Apache 2.0 license. Furthermore, the NEEDMI Project, which leverages NebulaStream to implement a smart co-pilot for an intensive care unit (ICU), recently won the SIGMOD 2025 Best Demo Honorable Mention award. The demo showcases NebulaStream’s ability to integrate and process multi-modal, multi-frequency data streams, to improve the health assessment of patients in a smart ICU.

Built for the edge

NebulaStream was designed to process data directly where it is generated—at the sensor, edge device, or gateway. The system supports diverse hardware architectures (ARM, x86) and compiles user queries into hardware-tailored code.

Adaptive resource management ensures robust performance, even under changing conditions or without Internet connectivity. Target applications include predictive maintenance, environmental monitoring, smart manufacturing, or smart medicine applications.

Flexible, extensible, ready for production

NebulaStream comes with out-of-the-box support for common stream tasks, such as data alignment and inference. Developers can easily add custom operators, formats, and optimizations, making the system highly adaptable to specialized requirements—without having to deal with low-level internals.

Demos

NebulaStream powers innovative applications across diverse domains. The IoTropolis project demo simulates a smart city to demonstrate how NebulaStream addresses data management challenges in IoT, eHealth, and Smart Grid scenarios.

In the health care sector, a NEEDMI project demo was developed to showcase how NebulaStream can be employed to support medical staff in intensive care units by enabling real-time data processing at a patient’s bedside.

From research to real-world impact

NebulaStream was developed as part of a collaborative research initiative led by BIFOLD, the DIMA Group at TU Berlin, and the IAM Group at DFKI. First presented in a scientific preprint in 2020, the system has now been released as open-source in the SIGMOD 2025 conference.

“We want to thank all contributors that made this possible, with their huge efforts over the last few years. Let us celebrate this together,” says Steffen Zeuch, Project Lead and System Architect of NebulaStream.

Provided by
Berlin Institute for the Foundations of Learning and Data – BIFOLD