57 projects
Laravel Framework
Laravel Framework is an open-source PHP framework for building web applications, offering features such as routing, authentication, and database management.
Contributors
24,107
Organizations
4,609
Software value
$11M
Celery
Celery is a distributed task queue system that enables asynchronous task processing and scheduling in Python applications. It supports real-time operations, scheduled tasks, and complex workflows with features like task routing, retries, and monitoring.
Contributors
9,022
Organizations
2,310
Software value
$4.1M
Ray
Ray is an open-source unified framework for scaling AI and Python applications. It provides a simple, universal API for building distributed applications and includes libraries for machine learning, serving, streaming, and more. Ray enables developers to parallelize single-machine code with minimal code changes and scale applications from a laptop to a cluster.
Contributors
8,956
Organizations
1,525
Software value
$52M
Sidekiq
Sidekiq is a background job processing system for Ruby applications that uses Redis for job storage and provides features like scheduling, retries, and monitoring through a web interface. It enables efficient handling of asynchronous tasks and is designed to be enterprise-ready with high performance and reliability.
Contributors
5,649
Organizations
1,808
Software value
$819K
Apache Beam
Apache Beam is a unified programming model and framework for building and executing batch and streaming data processing pipelines. It provides a portable API that enables developers to write data processing code once and run it on various execution engines like Apache Spark, Apache Flink, and Google Cloud Dataflow.
Contributors
4,769
Organizations
635
Software value
$95M
Apache DolphinScheduler
Apache DolphinScheduler is the modern data orchestration platform. Agile to create high performance workflow with low-code
Contributors
4,475
Organizations
240
Software value
$13M
Rundeck
Rundeck is an open-source automation service and job scheduling platform that enables users to automate routine operational procedures and orchestrate workflows across servers and environments. It provides a web-based interface for defining, scheduling, and monitoring jobs, with support for access control, audit logging, and integration with various tools and systems.
Contributors
4,044
Organizations
706
Software value
$21M
Prefect
Prefect is a workflow orchestration platform that enables users to build, schedule, and monitor data pipelines and machine learning workflows. It provides a Python-based framework for creating resilient, distributed workflows with features like automatic retries, caching, and real-time monitoring.
Contributors
3,874
Organizations
783
Software value
$34M
Hangfire
An easy way to perform background job processing in .NET and .NET Core applications. No Windows Service or separate process required
Contributors
2,681
Organizations
414
Software value
$2.8M
Nextflow
Nextflow is a workflow management system and domain-specific language for writing data-intensive computational pipelines. It enables scalable and reproducible scientific workflows using Docker/Singularity containers, with support for major cloud providers and HPC environments.
Contributors
2,447
Organizations
491
Software value
$17M
Apache Hadoop
Apache Hadoop is a distributed computing framework that enables processing and storage of large data sets across clusters of computers using simple programming models. It is designed to scale up from single servers to thousands of machines, with each offering local computation and storage.
Contributors
2,332
Organizations
270
Software value
$189M
Resque
Resque is a Redis-backed Ruby library for creating background jobs, placing them on multiple queues, and processing them later.
Contributors
2,052
Organizations
774
Software value
$281K
RQ
Simple job queues for Python
Contributors
1,982
Organizations
542
Software value
$701K
BullMQ
BullMQ - Message Queue and Batch processing for NodeJS and Python based on Redis
Contributors
1,967
Organizations
585
Software value
$3.1M
Agenda
A lightweight job scheduling library for Node.js that allows defining and managing recurring jobs with flexible timing options, persistence, and concurrency control
Contributors
1,558
Organizations
404
Software value
$2.1M
Temporal
Temporal is a distributed, scalable workflow orchestration platform that enables developers to build and operate reliable distributed applications. It provides durable execution, automatic retries, and workflow versioning, allowing teams to write business logic as code while the platform handles the complexity of distributed systems.
Contributors
1,526
Organizations
374
Software value
$26M
Quartz Scheduler
Quartz is an open source job scheduling library that enables Java applications to schedule and execute jobs at specified times and intervals. It provides a rich set of features for defining triggers, handling job persistence, clustering, and managing scheduled tasks in enterprise applications.
Contributors
1,409
Organizations
227
Software value
$1.6M
Spring Batch
Spring Batch is a framework for writing batch applications using Java and Spring
Contributors
1,363
Organizations
199
Software value
$6.9M
node-cron
Cron for NodeJS.
Contributors
1,166
Organizations
297
Software value
$153K
Windmill
Windmill is an open-source developer platform for building internal tools and workflows. It provides a low-code solution for creating backend scripts, APIs, and UIs with features like resource management, scheduling, and version control. The platform enables developers to write scripts in multiple languages and automate business processes.
Contributors
936
Organizations
249
Software value
$27M
Node Schedule
A flexible cron-like and not-cron-like job scheduler for Node.js that allows scheduling tasks (jobs) with various timing rules including specific dates, recurring rules and custom intervals
Contributors
927
Organizations
241
Software value
$86K
Enqueue
Message Queue, Job Queue, Broadcasting, WebSockets packages for PHP, Symfony, Laravel, Magento. DEVELOPMENT REPOSITORY - provided by Forma-Pro
Contributors
868
Organizations
230
Software value
$2.5M
Schedule
A Python job scheduling library that lets you run functions periodically at predetermined intervals using a simple, human-friendly syntax. It supports various scheduling patterns including fixed intervals, cron-style definitions, and one-off delayed execution.
Contributors
852
Organizations
191
Software value
$96K
Oban
Oban is a robust job processing and background job scheduling system for Elixir applications. It provides features like job queuing, retries, unique jobs, cron scheduling, and real-time job monitoring, all backed by PostgreSQL for reliability and persistence.
Contributors
736
Organizations
273
Software value
$597K
Machinery
Machinery is an asynchronous task queue/job queue based on distributed message passing in Go. It supports task scheduling, retries, rate limiting, and multiple message brokers including Redis and AMQP. The library enables developers to distribute tasks across worker nodes and process them asynchronously.
Contributors
711
Organizations
176
Software value
$784K
Sidekiq-Cron
Scheduler / Cron for Sidekiq jobs
Contributors
659
Organizations
236
Software value
$133K
OCA Queue
A Python framework for managing and processing asynchronous job queues, providing tools for task scheduling, worker management, and distributed job execution in Odoo applications
Contributors
640
Organizations
99
Software value
$498K
Action Scheduler
Action Scheduler is a scalable, robust job queue and task scheduler for WordPress, designed to handle recurring events, delayed jobs, and background processing. It stores tasks in WordPress posts and processes them using the WordPress cron system or WP-CLI.
Contributors
636
Organizations
108
Software value
$409K
Scalding
Scalding is a Scala library built on top of Cascading that makes it easy to write MapReduce jobs in a concise, type-safe way. It provides a domain-specific language for expressing complex data transformations and analytics on Hadoop.
Contributors
576
Organizations
121
Software value
$2.6M
Taskcluster
Taskcluster is Mozilla's task execution framework that handles continuous integration, release processes, and automated tasks. It provides a flexible queue service for managing tasks and dependencies, real-time task status and logs, and tools for creating and debugging task chains.
Contributors
571
Organizations
108
Software value
$19M
Apache Gobblin
Apache Gobblin is a distributed data integration framework that simplifies common aspects of big data integration such as data ingestion, replication, organization and lifecycle management. It provides a unified solution for orchestrating data movement and transformation from various sources to destinations at scale.
Contributors
478
Organizations
42
Software value
$12M
gocron
gocron is a Go library that provides job scheduling functionality similar to cron, allowing developers to run Go functions periodically at pre-determined intervals using a simple, fluent API
Contributors
449
Organizations
135
Software value
$273K
cron-utils
A Java library for parsing and manipulating cron expressions, providing utilities to work with cron scheduling patterns including validation, human-readable descriptions, and time calculations
Contributors
443
Organizations
79
Software value
$526K
HTCondor
HTCondor is a specialized workload management system for compute-intensive jobs. It provides a job queueing mechanism, scheduling policy, priority scheme, resource monitoring, and resource management. HTCondor places jobs on distributed computing resources and manages their execution, handling job submission, monitoring, and completion across clusters and grids.
Contributors
396
Organizations
53
Software value
$29M
cron-parser
A Node.js library for parsing and validating cron expressions, allowing conversion of cron schedule strings into structured time and date patterns
Contributors
382
Organizations
121
Software value
$174K
Parallel
A Ruby library that enables parallel processing and execution of code blocks, tasks, or tests across multiple CPU cores, with support for various parallel processing models and thread management
Contributors
359
Organizations
141
Software value
$64K
Ganga
Ganga is an easy-to-use frontend for job definition and management
Contributors
330
Organizations
56
Software value
$3.6M
rufus-scheduler
Rufus-scheduler is a Ruby job scheduler that allows scheduling of recurring jobs using a cron-like syntax. It enables running blocks of code at specified times and intervals, supporting both one-time and recurring tasks with time zones and various scheduling patterns.
Contributors
301
Organizations
112
Software value
$204K
HPCC Systems Platform
HPCC Systems Platform is an open-source, enterprise-grade big data analytics computing platform that allows processing and analysis of massive data sets across parallel computing clusters. It provides a complete end-to-end data lake management solution with built-in ETL capabilities, high-performance distributed computing, and a declarative programming language called ECL.
Contributors
290
Organizations
10
Software value
$89M
workerpool
A JavaScript library that provides a pool of web workers for parallel processing and task execution in both Node.js and browser environments, enabling efficient distribution of CPU-intensive tasks across multiple threads
Contributors
289
Organizations
92
Software value
$213K
node-resque
Node.js Background jobs backed by redis.
Contributors
270
Organizations
95
Software value
$220K
ProActive Workflows & Scheduling
ProActive Workflows & Scheduling is an open-source platform for workflow automation and job scheduling in distributed computing environments. It enables users to design, execute and manage workflows across multiple computing resources, supporting task orchestration, parallel processing, and distributed computing workloads.
Contributors
245
Organizations
11
Software value
$8.9M
Apache Tez
Apache Tez
Contributors
241
Organizations
31
Software value
$8.5M
Tron
Tron is a centralized job scheduling system that enables running distributed cron jobs across multiple machines. It handles job scheduling, monitoring, notification and dependency management while providing a web interface for job configuration and status tracking.
Contributors
205
Organizations
31
Software value
$1.5M
limits
A Python library that provides rate limiting utilities and decorators to control access to resources and functions, supporting multiple backend storage options like Redis and Memcached
Contributors
161
Organizations
59
Software value
$318K
Worker Farm
A Node.js module that enables parallel processing by spawning worker processes to handle CPU-intensive tasks, with support for multiple workers, automatic load balancing, and graceful error handling
Contributors
127
Organizations
55
Software value
$31K
Queue
A simple queue implementation for managing asynchronous operations in JavaScript, allowing controlled execution of tasks with configurable concurrency limits.
Contributors
108
Organizations
33
Software value
$39K
Better Queue
A fast, feature-rich queue implementation for Node.js that supports priority, concurrency control, retries, progress tracking and rate limiting
Contributors
95
Organizations
26
Software value
$74K
fastq
Fast, in memory work queue
Contributors
93
Organizations
36
Software value
$48K
Pegasus WMS
Pegasus WMS is a workflow management system that automates the execution of complex computational workflows across distributed computing resources. It transforms abstract workflow descriptions into concrete execution plans, handles data management, job scheduling, and fault tolerance for scientific applications.
Contributors
81
Organizations
12
Software value
$43M
Tinypool
A lightweight, fast worker thread pool implementation for Node.js that enables parallel processing and task distribution across multiple threads
Contributors
75
Organizations
36
Software value
$115K
Are We There Yet
A Node.js progress tracker that provides a way to show progress toward a goal, including nested progress indicators for complex operations with multiple steps and substeps
Contributors
54
Organizations
22
Software value
$9.5K
libEnsemble
libEnsemble is a Python library for coordinating the concurrent evaluation of dynamic ensembles of calculations. It enables the efficient management of parallel workflows, particularly for optimization, sampling, and uncertainty quantification tasks in scientific computing applications.
Contributors
53
Organizations
12
Software value
$1.3M
batch
A simple batch processing library for Node.js that allows executing multiple asynchronous operations in parallel with configurable concurrency limits and error handling
Contributors
44
Organizations
22
Software value
$25K
ActiveJob::TrafficControl
Database-backed Active Job backend
This project hasn't been onboarded to LFX Insights.
XXL-JOB
XXL-JOB is a distributed task scheduling platform with high reliability and concurrency. It supports multiple scheduling types, dynamic task allocation, and real-time task monitoring. The platform features a web-based management interface, fault tolerance, and horizontal scaling capabilities for handling large-scale job scheduling scenarios.
This project hasn't been onboarded to LFX Insights.