# LiteLLM LM Studio Adapter

This folder contains a standalone FastAPI adapter that lets LiteLLM or any other OpenAI-compatible caller use LM Studio native APIs while preserving LM Studio-specific reasoning controls.

## What it does

- Exposes OpenAI-compatible endpoints:
  - `GET /v1/models`
  - `POST /v1/chat/completions`
  - `POST /v1/responses`
  - `POST /v1/embeddings`
  - `POST /v1/completions`
- Exposes LM Studio native passthrough endpoints without modification:
  - `/api/v1/*`
  - `/api/v0/*`
- Supports SSE streaming for:
  - `POST /v1/chat/completions`
  - `POST /v1/responses`
- Preserves LM Studio reasoning toggle by translating:
  - `reasoning: "on" | "off"`
  - `enable_thinking: true | false`

## What is intentionally stubbed for now

- `/v1/audio/*`
- `/v1/images/*`
- `/v1/moderations`
- `/v1/files`

These return OpenAI-shaped `501 not_implemented` errors for now.

## Run

```bash
python services\litellm-lmstudio-adapter\litellm_lmstudio_adapter.py --host 127.0.0.1 --port 8010 --upstream http://127.0.0.1:7860
```

## LiteLLM usage

Use this adapter as an OpenAI-compatible backend behind LiteLLM.

Example LiteLLM config idea:

```yaml
model_list:
  - model_name: qwen36-local
    litellm_params:
      model: openai/qwen/qwen3.6-35b-a3b
      api_base: http://127.0.0.1:8010
      api_key: dummy
```

Then call LiteLLM normally with:

```json
{
  "model": "qwen36-local",
  "messages": [
    {"role": "user", "content": "Compute 317 * 29. Give the final answer only."}
  ],
  "reasoning": "off"
}
```

## Tests

```bash
python -m unittest discover -s services/litellm-lmstudio-adapter/tests -p "test_*.py"
```

## Verified locally

- `POST /v1/chat/completions` non-streaming:
  - `reasoning: "off"` => direct answer, `reasoning_tokens = 0`
  - `reasoning: "on"` => reasoning content returned
- `POST /v1/chat/completions` streaming:
  - emits OpenAI-style chat completion chunks
  - ends with `[DONE]`
- `POST /v1/responses` streaming:
  - emits `response.created`
  - emits `response.output_text.delta`
  - emits `response.completed`