15 projects
Nomad
Nomad is a distributed, highly available, datacenter-aware scheduler that can deploy a mix of microservices, batch, containerized and non-containerized applications across on-prem and cloud infrastructure at scale
Contributors
4,462
Organizations
1,345
Software value
$24M
Dask
Dask is a flexible parallel computing library for analytics that provides dynamic task scheduling optimized for computation and integrates with Python data science libraries like NumPy, Pandas and Scikit-learn. It enables parallel and distributed computing through intuitive APIs and scales Python code from multi-core machines to clusters.
Contributors
3,576
Organizations
900
Software value
$6.9M
Apache Hadoop
Apache Hadoop is a distributed computing framework that enables processing and storage of large data sets across clusters of computers using simple programming models. It is designed to scale up from single servers to thousands of machines, with each offering local computation and storage.
Contributors
2,322
Organizations
277
Software value
$190M
Volcano
Volcano is a batch system built on Kubernetes.
Contributors
1,513
Organizations
299
Software value
$54M
Quartz Scheduler
Quartz is an open source job scheduling library that enables Java applications to schedule and execute jobs at specified times and intervals. It provides a rich set of features for defining triggers, handling job persistence, clustering, and managing scheduled tasks in enterprise applications.
Contributors
1,405
Organizations
223
Software value
$1.6M
Apache Linkis
Apache Linkis builds a computation middleware layer to facilitate connection, governance and orchestration between the upper applications and the underlying data engines.
Contributors
786
Organizations
40
Software value
$19M
Slurm Workload Manager
Slurm: A Highly Scalable Workload Manager
Contributors
608
Organizations
74
Software value
$35M
Koordinator
A QoS-based scheduling system brings optimal layout and status to workloads such as microservices, web services, big data jobs, AI jobs, etc.
Contributors
427
Organizations
69
Software value
$27M
DIRAC
DIRAC Grid
Contributors
399
Organizations
51
Software value
$8.5M
HTCondor
HTCondor is a specialized workload management system for compute-intensive jobs. It provides a job queueing mechanism, scheduling policy, priority scheme, resource monitoring, and resource management. HTCondor places jobs on distributed computing resources and manages their execution, handling job submission, monitoring, and completion across clusters and grids.
Contributors
398
Organizations
51
Software value
$29M
Armada
Armada is an application to achieve high throughput of run-to-completion jobs on multiple Kubernetes clusters. It stores queues for users/projects with pod specifications and creates these pods once there is available resource in one of the connected Kubernetes clusters.
Contributors
224
Organizations
81
Software value
$9M
Flux Framework
core services for the Flux resource management framework
Contributors
203
Organizations
50
Software value
$13M
Mesos
Apache Mesos
This project hasn't been onboarded to LFX Insights.