14 projects
vLLM
The mission of the Project is to develop an open-source library for fast LLM inference and serving.
23,083
2,459
$78M
Ollama
Ollama is an open-source project that allows running, managing and serving large language models (LLMs) locally. It provides a simple API to run models like Llama 2, Mistral, and others on your own hardware, with features for model management, customization, and efficient inference.
16,780
2,715
$30M
LiteLLM
LiteLLM is a unified interface for calling AI large language models (LLMs) that provides a consistent API for working with various LLM providers including OpenAI, Anthropic, Cohere, Hugging Face, and others. It simplifies LLM integration by standardizing model inputs/outputs and handling provider-specific requirements.
8,612
1,587
$44M
SGLang
SGLang is a structured generation language and runtime system designed for efficient text generation and large language model inference. It provides a Python-based DSL for defining generation workflows with features like speculative decoding and token healing.
6,052
688
$31M
Spring AI
An Application Framework for AI Engineering
2,945
412
$11M
llama-cpp-python
A Python binding for llama.cpp, enabling integration of the llama.cpp library for running large language models locally. It provides a Python interface to load and run various LLM models compatible with llama.cpp, supporting both CPU and GPU acceleration.
2,622
424
$822K
LocalAI
LocalAI is an open-source project that provides a self-hosted, local alternative to OpenAI's APIs. It allows running LLMs (Large Language Models), text generation, embeddings, and image generation locally without requiring cloud services, supporting multiple model formats and hardware acceleration.
1,674
322
$14M
SillyTavern
A web-based chat interface for interacting with large language models and AI assistants, featuring character creation, conversation management, and customizable settings. It provides a frontend for various AI backends and models.
1,638
115
$6.8M
Unstructured
Open source libraries and APIs to build custom preprocessing pipelines for labeling, training, or production machine learning pipelines.
1,323
221
$38M
Quivr
Opiniated RAG for integrating GenAI in your apps 🧠 Focus on your product rather than the RAG. Easy integration in existing products with customisation! Any LLM: GPT4, Groq, Llama. Any Vectorstore: PGVector, Faiss. Any Files. Anyway you want.
1,279
148
$512K
Quarkus LangChain4j
Quarkus LangChain4j is an integration project that enables the use of LangChain4j, a Java framework for building applications with large language models (LLMs), within Quarkus applications. It provides Quarkus extensions and configurations for working with AI/ML models and language processing capabilities.
317
67
$6.8M
Instill Core
Instill Core is an open-source MLOps platform that provides infrastructure for building and deploying AI applications. It enables integration of various AI models and data sources through a unified API and pipeline system.
136
26
$2.1M
Agno
Agno is a lightweight library for building multi-modal Agents