lmstudiofixed/services/litellm-lmstudio-adapter

LiteLLM LM Studio Adapter

This folder contains a standalone FastAPI adapter that lets LiteLLM or any other OpenAI-compatible caller use LM Studio native APIs while preserving LM Studio-specific reasoning controls.

What it does

• Exposes OpenAI-compatible endpoints:
  • GET /v1/models
  • POST /v1/chat/completions
  • POST /v1/responses
  • POST /v1/embeddings
  • POST /v1/completions
• Exposes LM Studio native passthrough endpoints without modification:
  • /api/v1/*
  • /api/v0/*
• Supports SSE streaming for:
  • POST /v1/chat/completions
  • POST /v1/responses
• Preserves LM Studio reasoning toggle by translating:
  • reasoning: "on" | "off"
  • enable_thinking: true | false

What is intentionally stubbed for now

• /v1/audio/*
• /v1/images/*
• /v1/moderations
• /v1/files

These return OpenAI-shaped 501 not_implemented errors for now.

Run

python services\litellm-lmstudio-adapter\litellm_lmstudio_adapter.py --host 127.0.0.1 --port 8010 --upstream http://127.0.0.1:7860

LiteLLM usage

Use this adapter as an OpenAI-compatible backend behind LiteLLM.

Example LiteLLM config idea:

model_list:
  - model_name: qwen36-local
    litellm_params:
      model: openai/qwen/qwen3.6-35b-a3b
      api_base: http://127.0.0.1:8010
      api_key: dummy

Then call LiteLLM normally with:

{
  "model": "qwen36-local",
  "messages": [
    {"role": "user", "content": "Compute 317 * 29. Give the final answer only."}
  ],
  "reasoning": "off"
}

Tests

python -m unittest discover -s services/litellm-lmstudio-adapter/tests -p "test_*.py"

Verified locally

• POST /v1/chat/completions non-streaming:
  • reasoning: "off" => direct answer, reasoning_tokens = 0
  • reasoning: "on" => reasoning content returned
• POST /v1/chat/completions streaming:
  • emits OpenAI-style chat completion chunks
  • ends with [DONE]
• POST /v1/responses streaming:
  • emits response.created
  • emits response.output_text.delta
  • emits response.completed