Quick facts
- Best for
- llm-d Kubernetes-native LLM inference serving stack
- Pricing
- Free
- Editor rating
- 4.5 / 5
- Community saves
- 0
About llm-d
Kubernetes-native distributed inference serving for large language models on modern accelerators and production infrastructure The listing is based on the official GitHub repository, README, stars, forks, open issues, license, and recent commit metadata available during the daily OpenTools pipeline run.
Pros
- Provides distributed orchestration above model servers
- Adds intelligent routing and KV-cache management patterns
- Targets production LLM serving on Kubernetes and modern accelerators
Cons
Pricing
Open source
$0
- • Use the source repository
- • Review the README and license
- • Run with your own AI tools and infrastructure