Higress

Higress is an Apache 2.0 AI-native API gateway from Alibaba, built on Envoy and Istio, with multi-provider LLM routing, MCP hosting, OpenAPI-to-MCP conversion, token rate limiting, and semantic caching in one binary. It ships a standalone single-container mode as well as Kubernetes and Istio integration paths.

What it does

Higress exposes OpenAI-compatible endpoints for OpenAI, Anthropic, Azure, AWS Bedrock, DeepSeek, Moonshot, Alibaba Qwen, Google Gemini, and OpenAI-compatible upstreams, with automatic format translation. The MCP subsystem hosts MCP servers directly inside the gateway and converts OpenAPI specs into MCP servers — pointing Higress at a REST API yields an MCP server without writing a separate wrapper. Token rate limiting and consumer-based quotas use Redis for distributed state. Semantic caching uses embeddings to deduplicate similar prompts.

Wasm plugins written in Go, Rust, or JavaScript extend the gateway with custom logic. An OIDC authentication plugin handles end-user auth against generic OIDC providers.

Where it fits

Deployments that want a single gateway covering both LLM and conventional API traffic, with MCP hosting as a first-class feature. Alibaba positions Higress as the AI gateway inside the Alibaba Cloud ecosystem, but the open-source project is a complete gateway independent of any cloud provider.

Licensing

Apache 2.0. Alibaba Cloud offers a managed MSE service that runs Higress alongside other middleware; the open-source project is functionally complete on its own.

Deployment

Single Docker container plus optional Redis for the default standalone mode. Helm charts and Istio integration provide Kubernetes-native deployment alternatives. Docker Compose references exist for local development and smaller production use.

Limitations

• Primary documentation is Chinese; English documentation depth varies by subsystem.
• Generic OIDC integration with Western IdPs such as Keycloak, Zitadel, or Authentik works but has less English-language documentation than Chinese-ecosystem examples.
• Wasm plugin authoring requires setting up a Wasm toolchain for the chosen language.
• Community outside China is smaller, which can affect English-language issue turnaround.
• Envoy-based architecture carries some operational complexity when advanced tuning is required.