Open-source serverless platform for event-driven functions on Docker and Kubernetes
AI InfrastructureEdge Computing
Open-source serverless platform for event-driven function execution on Docker and Kubernetes. Nuclio packages code into containerized functions and exposes them through HTTP, cron, and stream-based triggers.
AGPL-3.0 Kubernetes-native hybrid LLM gateway, MCP orchestrator, and chat UI
Agent & MCP ToolingAI Infrastructure
AGPL-3.0 hybrid platform combining LLM gateway, MCP orchestrator, and chat UI into one Kubernetes-native stack. Orchestrator spawns each MCP server as an isolated K8s pod — strong isolation, but K8s-only in practice.
AI gateway unifying 1,600+ LLM endpoints with guardrails, caching, and MCP
Agent & MCP ToolingAI Infrastructure
MIT AI gateway from Portkey AI that routes traffic across 1,600+ LLM endpoints through one OpenAI-compatible API. Combines routing, virtual keys, guardrails, caching, observability, and MCP gateway support in a Node.js / Go proxy that runs on Docker or Kubernetes.
Go LLM gateway with virtual keys, budgets, and MCP client/server
Agent & MCP ToolingAI Infrastructure
LLM gateway from Maxim AI written in Go, with OpenAI-compatible routing, virtual keys, budgets, MCP client and server, and a plugin-based governance pipeline. Apache 2.0 core with a commercial Bifrost Enterprise tier.
Open-source LLM observability proxy with traces, prompts, and cost analytics
AI Infrastructure
Open-source LLM observability platform from Helicone that proxies requests to 20+ providers while capturing traces, cost data, prompts, and user analytics. Self-hostable via Docker Compose with ClickHouse, Postgres, MinIO, and a web UI.
Rust AI gateway with multi-provider routing, semantic caching, and telemetry
AI Infrastructure
AI gateway from LangDB implemented in Rust, with multi-provider routing, virtual keys, semantic caching, and structured observability. Published under the Elastic License v2 — source-available with a restriction on managed-service redistribution.
OpenAI-compatible LLM gateway with built-in user accounts, wallets, and quotas
AI Infrastructure
Open-source LLM gateway from songquanpeng that proxies 30+ providers behind an OpenAI-compatible API with built-in user accounts, token wallets, and hard quota enforcement. MIT-licensed with active feature development continuing in the new-api fork.
AGPL LLM gateway with OIDC SSO, token wallets, and quota enforcement
AI Infrastructure
AGPL-3.0 LLM gateway from QuantumNous, forked from one-api and extended with OIDC SSO, expanded provider coverage, and a redesigned admin UI. Proxies 40+ providers via OpenAI-compatible API with a token-wallet and quota system.
LLM plugin suite for Kong Gateway with routing, guardrails, and MCP exposure
AI Infrastructure
Apache 2.0 Kong Gateway plugin suite for multi-provider LLM routing, prompt guardrails, token-based rate limiting, and MCP exposure. Core and several plugins are OSS; AI Rate Limiting Advanced, Semantic Routing, and OIDC auth are Kong Enterprise.
LLM extensions for Envoy Gateway with CEL policy, JWT auth, and MCP OAuth
AI Infrastructure
Apache 2.0 LLM extensions for Envoy Gateway from the Envoy Proxy community, with CEL-based routing, JWT authentication, token metrics, and MCP OAuth 2.1 support. Kubernetes Gateway API is the primary deployment target.
AI-native API gateway with built-in MCP hosting and OpenAPI-to-MCP conversion
Agent & MCP ToolingAI Infrastructure
Apache 2.0 AI-native API gateway from Alibaba, built on Envoy, with multi-provider LLM routing, built-in MCP hosting, OpenAPI-to-MCP conversion, and token rate limiting. Runs as a single Docker container or on Kubernetes via Helm.
Rust agent gateway covering MCP, A2A, and LLM under Linux Foundation
Agent & MCP ToolingAI Infrastructure
Apache 2.0 Rust agent gateway covering MCP, A2A, and LLM protocols, governed by the Linux Foundation with AWS, Cisco, IBM, Microsoft, Red Hat, and Solo.io participating. Kubernetes Gateway API-first with a standalone binary mode stabilizing through pre-1.0.
Rust LLM gateway with structured inference capture and experimentation workflows
AI Infrastructure
Apache 2.0 Rust LLM gateway from TensorZero Inc. that routes traffic across providers and stores every inference in ClickHouse with typed schemas, driving experimentation, evaluation, and fine-tuning workflows from captured data.
LLMOps platform with visual agent builder, RAG, and multi-provider gateway
Agent & MCP ToolingAI Infrastructure
Self-hostable LLMOps platform from LangGenius with a visual agent builder, RAG pipeline, multi-provider gateway, and prompt management. Uses the Dify Open Source License — modified Apache 2.0 with a non-compete clause prohibiting multi-tenant SaaS resale.
LLM engineering platform for tracing, prompts, evaluation, and datasets
AI Infrastructure
MIT LLM engineering platform from Langfuse for tracing, prompt management, evaluation, and datasets, captured via SDK instrumentation or gateway integration. Core including OIDC SSO is in OSS; advanced RBAC, audit retention, and SCIM sit in the Enterprise tier.
Python AI gateway with virtual keys, budgets, cost tracking, and MCP federation
Agent & MCP ToolingAI Infrastructure
Python-based AI gateway from BerriAI that proxies 100+ LLM providers behind one OpenAI-compatible API with virtual keys, hard-enforced budgets, cost dashboards, and an MCP federation endpoint. Core is MIT; SSO, JWT-to-key mapping, and audit retention sit in LiteLLM Enterprise.
Cross-platform inference and training accelerator for machine learning models.
AI InfrastructureML Platforms & MLOps
Open-source runtime for ONNX models with hardware-specific optimizations for CPUs, GPUs, and edge devices. Supports inferencing and training across multiple ML frameworks.
Open-source machine learning framework for building and deploying ML models
AI InfrastructureML Platforms & MLOps
An end-to-end open-source platform for machine learning with comprehensive tools, libraries, and community resources for building and deploying ML-powered applications.
Open-source toolkit for optimizing and deploying AI inference on Intel hardware
AI Infrastructure
+2 more
Open-source toolkit that accelerates AI inference with lower latency and higher throughput while maintaining accuracy. Supports computer vision, LLMs, and generative AI models from TensorFlow, PyTorch, and ONNX.
