Azure AI Gateway

Azure AI Gateway

Configure Azure API Management (APIM) as an AI Gateway for governing AI models, MCP tools, and agents.

To deploy APIM, use the azure-prepare skill. See APIM deployment guide.

When to Use This Skill

---

Quick Reference

---

Get Gateway Details

# Get gateway URL
az apim show --name <apim-name> --resource-group <rg> --query "gatewayUrl" -o tsv

# List backends (AI models)
az apim backend list --service-name <apim-name> --resource-group <rg> \
  --query "[].{id:name, url:url}" -o table

# Get subscription key
az apim subscription keys list \
  --service-name <apim-name> --resource-group <rg> --subscription-id <sub-id>

---

Test AI Endpoint

GATEWAY_URL=$(az apim show --name <apim-name> --resource-group <rg> --query "gatewayUrl" -o tsv)

curl -X POST "${GATEWAY_URL}/​openai/​deployments/<deployment>/​chat/​completions?api-version=2024-02-01" \
  -H "Content-Type: application/​json" \
  -H "Ocp-Apim-Subscription-Key: <key>" \
  -d '{"messages": [{"role": "user", "content": "Hello"}], "max_tokens": 100}'

---

Common Tasks

Add AI Backend

See references/​patterns.md for full steps.

# Discover AI resources
az cognitiveservices account list --query "[?kind=='OpenAI']" -o table

# Create backend
az apim backend create --service-name <apim> --resource-group <rg> \
  --backend-id openai-backend --protocol http --url "https://<aoai>.openai.azure.com/​openai"

# Grant access (managed identity)
az role assignment create --assignee <apim-principal-id> \
  --role "Cognitive Services User" --scope <aoai-resource-id>

Apply AI Governance Policy

Recommended policy order in <inbound>:

1. Authentication - Managed identity to backend
2. Semantic Cache Lookup - Check cache before calling AI
3. Token Limits - Cost control
4. Content Safety - Filter harmful content
5. Backend Selection - Load balancing
6. Metrics - Token usage tracking

See references/​policies.md for complete example.

---

Troubleshooting

See references/​troubleshooting.md for details.

---

References

• Detailed Policies - Full policy examples
• Configuration Patterns - Step-by-step patterns
• Troubleshooting - Common issues
• AI-Gateway Samples
• GenAI Gateway Docs

SDK Quick References

• Content Safety: Python | TypeScript
• API Management: Python | .NET