LFX Platform

Know more about LFX Platform

LFX Insights

High-Performance Distributed Storage

Software systems designed for high-throughput, low-latency access to distributed data storage across clusters and networks, often used in scientific computing and big data environments.

17 projects

41,495 contributors

$2.6B

Elasticsearch

Elasticsearch is a distributed, RESTful search and analytics engine capable of addressing a growing number of use cases. As the heart of the Elastic Stack, it centrally stores data for lightning fast search, fine‑tuned relevancy, and powerful analytics.

Contributors

18,228

Organizations

3,592

Software value

$190M

CockroachDB

CockroachDB is a distributed SQL database system designed to be scalable, consistent, and highly available. It combines the benefits of traditional relational databases with the horizontal scalability and resilience of NoSQL systems, offering features like automatic replication, distributed transactions, and self-healing capabilities.

Contributors

4,371

Organizations

1,113

Software value

$132M

Alluxio

Alluxio is a distributed system that enables data orchestration across different storage systems and computation frameworks. It provides a unified namespace and data access layer, improving performance through memory-centric architecture and intelligent caching while maintaining compatibility with existing applications.

Contributors

3,352

Organizations

259

Software value

$19M

Apache Hadoop

Apache Hadoop is a distributed computing framework that enables processing and storage of large data sets across clusters of computers using simple programming models. It is designed to scale up from single servers to thousands of machines, with each offering local computation and storage.

Contributors

2,311

Organizations

268

Software value

$190M

DAOS Project

DAOS (Distributed Asynchronous Object Storage) is a high-performance scale-out storage system designed for next-generation supercomputing and data-intensive applications. It provides a native key-array interface and supports both traditional files and object storage, optimized for non-volatile memory and high-performance fabrics.

Contributors

2,180

Organizations

132

Software value

$183M

Gluster

Gluster is a scalable, distributed file system that enables users to aggregate various storage bricks over Infiniband RDMA or TCP/IP interconnect into larger storage volumes. It provides features like replication, quotas, geo-replication, snapshots and bitrot detection, making it suitable for data center, cloud and container storage deployments.

Contributors

1,801

Organizations

257

Software value

$21M

ScyllaDB

ScyllaDB is a high-performance NoSQL database that offers Apache Cassandra compatibility with significantly better throughput and lower latency. It is designed as a drop-in replacement for Cassandra, leveraging close-to-hardware programming and C++ to deliver improved resource utilization and reduced operational overhead.

Contributors

1,779

Organizations

395

Software value

$25M

YTsaurus

YTsaurus is a distributed storage and processing platform designed for managing large-scale data. It provides a comprehensive suite of tools for data organization, processing, and analysis, supporting features like distributed execution, data replication, and resource management across clusters.

Contributors

1,506

Organizations

29

Software value

$1.6B

Apache HBase

Apache HBase is a distributed, scalable, big data store designed to provide quick random access to huge amounts of structured data. It is a NoSQL database that runs on top of Hadoop HDFS, offering real-time read/write access to large datasets and supporting high-throughput applications.

Contributors

1,478

Organizations

131

Software value

$41M

Apache Cassandra

Apache Cassandra is a highly scalable, distributed NoSQL database management system designed to handle large amounts of data across multiple commodity servers, providing high availability with no single point of failure. It offers robust support for clusters spanning multiple datacenters, with asynchronous masterless replication allowing for low latency operations for all clients.

Contributors

1,358

Organizations

194

Software value

$54M

Pravega

Pravega is a storage system that uses Stream as the main building block for storing continuous and limitless data.

Contributors

767

Organizations

159

Software value

$70M

ChubaoFS

ChubaoFS is a distributed file system for cloud native applications.

Contributors

603

Organizations

102

Software value

$80M

XRootD

The XRootD central repository https://my.cdash.org/index.php?project=XRootD

Contributors

548

Organizations

123

Software value

$9.7M

Apache Accumulo

Apache Accumulo is a distributed key-value store built on Apache Hadoop, ZooKeeper, and Thrift. It features cell-level access control, server-side programming mechanisms, and robust data management capabilities designed for high performance, scalability, and security.

Contributors

471

Organizations

54

Software value

$19M

TrueNAS WebUI

TrueNAS WebUI is a web-based user interface for managing TrueNAS systems, providing a modern interface for storage management, system configuration, and monitoring of TrueNAS storage appliances

Contributors

465

Organizations

53

Software value

$35M

TileDB

TileDB is a universal data engine that enables efficient storage, querying and management of multi-dimensional array data. It provides a novel storage format and APIs for handling dense and sparse arrays with support for multiple data types, cloud storage integration, and parallel I/O operations.

Contributors

215

Organizations

62

Software value

$13M

Oxia

Oxia is a distributed key-value store designed for high performance and scalability, optimized for handling large volumes of data with strong consistency guarantees. It provides a reliable storage solution for distributed systems with features like replication and fault tolerance.

Contributors

62

Organizations

11

Software value

$4.6M

Looking for a project that’s not listed?