vlyseyko/litellm-config
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
No description
14 commits 1 branch 0 tags 109 KiB
  • Python 81.2%
  • Shell 18.8%
Find a file
Exact
Lumi c293473194 qwen3.6-27b-vision: add coding-optimized sampling params, set as default fallback 
- temperature: 0.6, top_p: 0.95, top_k: 20 (Qwen official coding recommendations)
- Changed default_fallbacks from deepseek-v4-flash to qwen3.6-27b-vision
- No vLLM restart needed — LiteLLM passes these as defaults in each request
2026-07-09 14:36:38 +00:00
.gitignore feat: full audit and rewrite of litellm config 2026-06-18 21:16:24 +00:00
config.yaml qwen3.6-27b-vision: add coding-optimized sampling params, set as default fallback 2026-07-09 14:36:38 +00:00
deploy.sh feat: full audit and rewrite of litellm config 2026-06-18 21:16:24 +00:00
README.md feat: add explicit reasoning/thinking variants for all models 2026-06-19 18:48:08 +00:00
verify_pricing.py Consolidate Anthropic models: remove max_tokens, thinking, output_config params 2026-07-08 18:33:04 +00:00

README.md

LiteLLM Proxy Config Manager

Managed config for the LiteLLM proxy at 10.10.30.204:4000.

Location

  • TrueNAS source: /mnt/RAIDZ2/docker/data/open-webui/litellm/config.yaml
  • Git repo: ~/coding_agent_repos/litellm-config/config.yaml
  • Remote: https://git.novelbit.com/vlyseyko/litellm-config

Change Log

2026-06-18 - Full Audit & Rewrite

Verified against official pricing pages (see sources below).

Pricing Fixes:

Model Field Old ($/MTok) New ($/MTok) Issue
gemini-embedding-v2 input/output 0.02 0.20 10x underpriced
gpt-5-nano input 0.10 0.25 60% underpriced
gpt-5-nano output 0.30 1.50 80% underpriced
gemini-3.1-flash-lite-fast input 0.10 0.25 60% underpriced
gemini-3.1-flash-lite-fast output 0.30 1.50 80% underpriced
gemini-3.5-flash-medium input 0.50 1.50 67% underpriced
gemini-3.5-flash-medium output 3.00 9.00 67% underpriced

Model Name Fixes:

Old New Reason
claude-opus-4.7 claude-opus-4.8 Model "opus 4.7" doesn't exist
anthropic/claude-sonnet-4-6 anthropic/claude-sonnet-4-20250514 Canonical API name
gemini/gemini-3.5-flash-medium gemini/gemini-3.5-flash "medium" suffix doesn't exist

Context Window Fixes:

Model Old max_input_tokens New max_input_tokens
grok-4.3 131,072 1,048,576 (1M)
grok-4.3-* variants 131,072 1,048,576 (1M)

2026-06-18 - Added Explicit Reasoning/Thinking Variants

Each model with reasoning/thinking now has explicit entries for each level, exposed directly to Open WebUI:

Provider Parameter Values
OpenAI reasoning_effort none, low, high
Anthropic extended_thinking + budget_tokens standard vs. thinking (16000 tokens)
Google thinking_level minimal, medium, high
xAI reasoning_effort none, low, medium, high

Reasoning/Thinking Model Variants

OpenAI GPT-5.5 ($5/$30/MTok)

Alias Reasoning
gpt-5.5 none
gpt-5.5-low low
gpt-5.5-high high

OpenAI GPT-5.4 ($2.50/$15/MTok)

Alias Reasoning
gpt-5.4 none
gpt-5.4-low low
gpt-5.4-high high

Anthropic Claude Opus 4.8 ($5/$25/MTok)

Alias Thinking
claude-opus-4.8 none (max_tokens: 16000)
claude-opus-4.8-thinking enabled (16000 budget, max_tokens: 32000)

Anthropic Claude Sonnet 4.6 ($3/$15/MTok)

Alias Thinking
claude-sonnet-4.6 none (max_tokens: 16000)
claude-sonnet-4.6-thinking enabled (16000 budget, max_tokens: 32000)

Google Gemini 3.5 Flash ($1.50/$9/MTok)

Alias Thinking
gemini-3.5-flash minimal
gemini-3.5-flash-medium medium
gemini-3.5-flash-high high

Google Gemini 3.1 Pro ($2/$12/MTok)

Alias Thinking
gemini-3.1-pro minimal
gemini-3.1-pro-medium medium
gemini-3.1-pro-high high

Google Gemini 3.1 Flash-Lite ($0.25/$1.50/MTok)

Alias Thinking
gemini-3.1-flash-lite minimal
gemini-3.1-flash-lite-medium medium
gemini-3.1-flash-lite-high high

xAI Grok 4.3 ($1.25/$2.50/MTok)

Alias Reasoning
grok-4.3 none
grok-4.3-low low
grok-4.3-medium medium
grok-4.3-high high

Sources

Platform URL
OpenAI https://developers.openai.com/api/docs/pricing
Anthropic https://claude.com/pricing (API tab)
Google Gemini https://ai.google.dev/gemini-api/docs/pricing
xAI Grok https://docs.x.ai/developers/models

Verification

Run python3 verify_pricing.py to check all pricing and context window values.

Deploy

cd ~/coding_agent_repos/litellm-config
./deploy.sh
# or dry run:
./deploy.sh --dry-run

Current Model Summary

Alias Provider Model Input Output
qwen3.6-27b Local vLLM qwen3.6-27b-256k-text $0.35 $0.35
qwen3.6-27b-vision Local vLLM qwen3.6-27b-256k-vision $0.35 $0.35
gemma-4-31B Local vLLM gemma-4-31b-256k-text $0.35 $0.35
gemini-3.1-flash-lite Google gemini-3.1-flash-lite $0.25 $1.50
gemini-3.1-flash-lite-medium Google gemini-3.1-flash-lite (medium thinking) $0.25 $1.50
gemini-3.1-flash-lite-high Google gemini-3.1-flash-lite (high thinking) $0.25 $1.50
gemini-3.5-flash Google gemini-3.5-flash $1.50 $9.00
gemini-3.5-flash-medium Google gemini-3.5-flash (medium thinking) $1.50 $9.00
gemini-3.5-flash-high Google gemini-3.5-flash (high thinking) $1.50 $9.00
gemini-3.1-pro Google gemini-3.1-pro-preview $2.00 $12.00
gemini-3.1-pro-medium Google gemini-3.1-pro-preview (medium thinking) $2.00 $12.00
gemini-3.1-pro-high Google gemini-3.1-pro-preview (high thinking) $2.00 $12.00
gemini-deep-research Google deep-research-preview-04-2026 $2.00 $18.00
gemini-embedding-v2 Google gemini-embedding-2 $0.20 $0.20
gpt-5.4-nano OpenAI gpt-5.4-nano $0.20 $1.25
gpt-5.4-mini OpenAI gpt-5.4-mini $0.75 $4.50
gpt-5.4 OpenAI gpt-5.4 $2.50 $15.00
gpt-5.4-low OpenAI gpt-5.4 (low reasoning) $2.50 $15.00
gpt-5.4-high OpenAI gpt-5.4 (high reasoning) $2.50 $15.00
gpt-5.5 OpenAI gpt-5.5 $5.00 $30.00
gpt-5.5-low OpenAI gpt-5.5 (low reasoning) $5.00 $30.00
gpt-5.5-high OpenAI gpt-5.5 (high reasoning) $5.00 $30.00
claude-opus-4.8 Anthropic claude-opus-4-20250514 $5.00 $25.00
claude-opus-4.8-thinking Anthropic claude-opus-4-20250514 (extended thinking) $5.00 $25.00
claude-sonnet-4.6 Anthropic claude-sonnet-4-20250514 $3.00 $15.00
claude-sonnet-4.6-thinking Anthropic claude-sonnet-4-20250514 (extended thinking) $3.00 $15.00
claude-haiku-4.5 Anthropic claude-haiku-4-5-20251001 $1.00 $5.00
grok-build-0.1 xAI grok-build-0.1 $1.00 $2.00
grok-4.3 xAI grok-4.3 $1.25 $2.50
grok-4.3-low xAI grok-4.3 (low reasoning) $1.25 $2.50
grok-4.3-medium xAI grok-4.3 (med reasoning) $1.25 $2.50
grok-4.3-high xAI grok-4.3 (high reasoning) $1.25 $2.50