29 projects
Logstash
Logstash is a server-side data processing pipeline that ingests data from multiple sources simultaneously, transforms it, and then sends it to a destination of choice. It is commonly used to collect logs and other time-series data for search, analysis and visualization in Elasticsearch.
Contributors
5,700
Organizations
1,308
Software value
$5.5M
Apache Flink
Apache Flink is a distributed stream processing and batch computation framework. It provides high-throughput, low-latency data streaming engines as well as support for event-driven applications and batch processing. The framework enables stateful computations over data streams and features automatic memory management, fault tolerance, and exactly-once processing semantics.
Contributors
4,683
Organizations
567
Software value
$99M
Apache Kafka
Apache Kafka is a distributed event streaming platform designed for high-throughput, fault-tolerant handling of real-time data feeds. It enables building real-time streaming data pipelines and applications that can process, transform, and react to streams of events.
Contributors
4,567
Organizations
750
Software value
$45M
Strimzi
Strimzi provides a way to run an Apache Kafka cluster on Kubernetes in various deployment configurations.
Contributors
3,640
Organizations
864
Software value
$334M
Apache RocketMQ
Apache RocketMQ is a distributed messaging and streaming platform with low latency, high performance and reliability, trillion-level capacity and flexible scalability. It supports pub-sub and push-pull messaging models, providing a reliable message-based asynchronous communication and streaming data processing solution for distributed applications.
Contributors
3,598
Organizations
272
Software value
$10M
Apache Pulsar
Apache Pulsar is a cloud-native, distributed messaging and streaming platform designed to handle high-performance workloads. It provides pub-sub messaging, queuing, stream processing, and event streaming capabilities with built-in multi-tenancy, geo-replication, and persistent storage features.
Contributors
3,528
Organizations
540
Software value
$29M
Hazelcast
Hazelcast is an open-source distributed computing platform that provides in-memory data storage and processing capabilities. It offers features like distributed caching, distributed data structures, distributed computing, and clustering for building scalable applications.
Contributors
2,979
Organizations
462
Software value
$64M
Sarama
Sarama is a Go library for Apache Kafka.
Contributors
2,157
Organizations
616
Software value
$2M
AutoMQ
AutoMQ is a cloud-native alternative to Kafka by decoupling durability to S3 and EBS. 10x Cost-Effective. No Cross-AZ Traffic Cost. Autoscale in seconds. Single-digit ms latency.
Contributors
1,854
Organizations
206
Software value
$44M
Apache Storm
Apache Storm
Contributors
1,138
Organizations
201
Software value
$14M
Redpanda
Redpanda is a streaming data platform that serves as a Kafka-compatible event streaming engine, offering high performance, low latency, and simplified operations. It is designed to handle real-time data streams and event processing with features like JVM-free architecture, thread-per-core model, and cloud-native capabilities.
Contributors
1,054
Organizations
286
Software value
$37M
Pravega
Pravega is a storage system that uses Stream as the main building block for storing continuous and limitless data.
Contributors
769
Organizations
163
Software value
$70M
Apache BookKeeper
Apache BookKeeper - a scalable, fault tolerant and low latency storage service optimized for append-only workloads
Contributors
739
Organizations
110
Software value
$13M
TiFlow
TiFlow is a data replication and migration tool that enables real-time data flow between different databases and systems. It includes CDC (Change Data Capture) capabilities for capturing and streaming database changes, and supports integration with TiDB database ecosystem.
Contributors
512
Organizations
61
Software value
$20M
EventStream
EventStream is a Node.js library that provides a powerful streaming interface for working with event-based data. It enables transforming, filtering, and manipulating streams of events using functional programming patterns, with support for both synchronous and asynchronous operations.
Contributors
509
Organizations
250
Software value
$38K
Franz-Go
franz-go contains a feature complete, pure Go library for interacting with Kafka from 0.8.0 through 3.8+. Producing, consuming, transacting, administrating, etc.
Contributors
493
Organizations
153
Software value
$4.1M
Bento Stream Processor
A stream processing engine and framework for building real-time data pipelines and analytics applications
Contributors
489
Organizations
104
Software value
$12M
Materialize
Materialize is a streaming database that processes real-time data streams and maintains materialized views, enabling fast SQL queries over streaming data. It allows users to build real-time applications and analytics by transforming complex streaming data into queryable views.
Contributors
463
Organizations
92
Software value
$34M
Conduit data streaming
Conduit is a streaming data library for Haskell that provides composable, memory-efficient processing of data streams. It enables handling large datasets and I/O operations through a unified streaming interface.
Contributors
435
Organizations
165
Software value
$466K
Apache Celeborn
Apache Celeborn is an elastic and high-performance service for shuffle and spilled data.
Contributors
417
Organizations
58
Software value
$6.9M
Benthos
Benthos is a high-performance stream processing engine that enables users to build resilient data pipelines. It connects various sources and sinks, performs data transformations, and handles message routing with a declarative configuration approach.
Contributors
406
Organizations
83
Software value
$5.2M
Nussknacker
Nussknacker is a visual workflow designer and runtime engine that enables users to create and deploy real-time data processing applications. It provides a low-code platform for building streaming processes using Apache Flink, with features for ETL operations, data enrichment, and complex event processing.
Contributors
241
Organizations
16
Software value
$13M
Substreams
Substreams is a blockchain data processing engine that enables streaming of on-chain data through a series of WASM modules. It provides high-performance parallel processing capabilities for extracting, transforming, and analyzing blockchain data in real-time.
Contributors
197
Organizations
42
Software value
$5M
Tremor
Tremor is an early stage event processing system for unstructured data with rich support for structural pattern matching, filtering and transformation.
Contributors
180
Organizations
93
Software value
$13M
Pump
Pump is a streaming data pipeline library that allows piping between Node.js streams with error handling and automatic cleanup. It provides a simpler alternative to manually handling stream events and error propagation.
Contributors
80
Organizations
43
Software value
$6.5K
Combined Stream
A Node.js module that allows combining multiple streams into a single stream, enabling sequential reading from multiple sources as if they were a single source
Contributors
63
Organizations
29
Software value
$30K
streamx
A minimal and efficient Node.js streaming library that provides a lightweight alternative to the built-in streams implementation, focusing on performance and simplicity
Contributors
54
Organizations
33
Software value
$82K
Split2
Split2 is a Node.js module that splits text input into lines using a delimiter. It provides a transform stream interface for breaking up text streams by newlines or custom separators, with support for both string and buffer inputs.
Contributors
53
Organizations
32
Software value
$15K
Kafka Docker Playground
🐳✨ Fully automated Apache Kafka® and Confluent Docker based examples // 👷♂️ Easily build examples or reproduction models
This project hasn't been onboarded to LFX Insights.