LFX Platform

Know more about LFX Platform

LFX Insights

On-Device Neural Engines

Frameworks and runtimes optimized for executing neural network models directly on edge and mobile devices.

21 projects

53,386 contributors

$610M

llama cpp

A port of Facebook's LLaMA model in C/C++, focusing on efficient inference and deployment of large language models on consumer hardware. The project provides optimized implementations for running LLaMA models with minimal dependencies and memory requirements.

Contributors

11,890

Organizations

1,799

Software value

$29M

YOLOv5

YOLOv5 is a computer vision model and framework for real-time object detection, offering state-of-the-art performance, easy training and deployment capabilities, and extensive documentation. It implements the YOLO (You Only Look Once) architecture with improvements for speed and accuracy.

Contributors

8,826

Organizations

705

Software value

$804K

ONNX

ONNX is an open format built to represent machine learning models. ONNX defines a common set of operators - the building blocks of machine learning and deep learning models - and a common file format to enable AI developers to use models with a variety of frameworks, tools, runtimes, and compilers.

Contributors

8,035

Organizations

995

Software value

$51M

OpenVINO

OpenVINO (Open Visual Inference and Neural Network Optimization) is an open-source toolkit that enables fast development of computer vision and deep learning applications, optimizing neural network models for Intel hardware. It provides tools for model optimization, inference engine deployment, and cross-platform performance acceleration.

Contributors

4,340

Organizations

389

Software value

$83M

ncnn

ncnn is a high-performance neural network inference framework optimized for the mobile platform

Contributors

3,513

Organizations

310

Software value

$28M

Triton Inference Server

The Triton Inference Server provides an optimized cloud and edge inferencing solution.

Contributors

3,341

Organizations

504

Software value

$5.3M

Apache TVM

Apache TVM is an open source machine learning compiler framework for CPUs, GPUs, and machine learning accelerators. It aims to enable machine learning engineers to optimize and run computations efficiently on any hardware backend.

Contributors

3,270

Organizations

388

Software value

$24M

MLC LLM

Universal LLM Deployment Engine with ML Compilation

Contributors

1,709

Organizations

237

Software value

$2.9M

XLA

XLA (Accelerated Linear Algebra) is a compiler and runtime system for machine learning that optimizes and executes computational graphs across different hardware platforms. It provides hardware abstraction and optimization capabilities for deep learning frameworks, enabling efficient execution of ML models on various accelerators like GPUs and TPUs.

Contributors

1,698

Organizations

203

Software value

$56M

ExecuTorch

ExecuTorch is a runtime solution for deploying PyTorch models on mobile and edge devices, focusing on efficient model execution and optimization for resource-constrained environments. It provides tools for model compilation, portability across different hardware platforms, and streamlined deployment of AI models.

Contributors

1,532

Organizations

160

Software value

$30M

GGML

GGML is a tensor library for machine learning that enables efficient neural network inference on CPU. It provides low-level primitives for implementing deep learning models with a focus on performance and memory efficiency, particularly for running large language models on consumer hardware.

Contributors

1,531

Organizations

274

Software value

$7.9M

TT-Metal

TT-Metal is a software framework for programming Tenstorrent AI accelerator chips, providing low-level hardware access and control capabilities for machine learning workloads

Contributors

1,451

Organizations

79

Software value

$112M

MLX

MLX: An array framework for Apple silicon

Contributors

1,050

Organizations

249

Software value

$5.6M

Neural Network (NN) Streamer

? Neural Network (NN) Streamer, Stream Processing Paradigm for Neural Network Apps/Devices.

Contributors

458

Organizations

49

Software value

$34M

XNNPACK

High-efficiency floating-point neural network inference operators for mobile, server, and Web

Contributors

424

Organizations

93

Software value

$105M

Optimum Intel

🤗 Optimum Intel: Accelerate inference with Intel optimization tools

Contributors

304

Organizations

27

Software value

$1.8M

Archived

ShaderNN

The mission of the Project is to develop a lightweight deep learning inference framework optimized for Convolutional Neural Networks.

Contributors

14

Organizations

3

Software value

$35M

InternLM

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

This project hasn't been onboarded to LFX Insights.

MNN

MNN is a blazing fast, lightweight deep learning framework, battle-tested by business-critical use cases in Alibaba. Full multimodal LLM Android App:[MNN-LLM-Android](./apps/Android/MnnLlmChat/README.md)

This project hasn't been onboarded to LFX Insights.

Neuro-One

On-device Neural Engine

This project hasn't been onboarded to LFX Insights.

Torch-TensorRT

PyTorch/TorchScript/FX compiler for NVIDIA GPUs using TensorRT

This project hasn't been onboarded to LFX Insights.
Looking for a project that’s not listed?