llm-session-proxy

The problem it solves

OpenCode Go wants a session identifier on every request, and most clients cannot set custom headers.

The upstream expects a header like this:

x-opencode-session : stable per conversation (used for prompt caching and routing)

If the client does not send it, the upstream refuses the call outright:

400 Request is missing x-opencode-session and cannot be routed efficiently

This tool moves that job onto your machine: point the client's Base URL at the local proxy and the proxy handles the rest. Nothing in it is tied to one provider — swap upstream and inject.headers and it fronts any OpenAI- or Anthropic-compatible endpoint.

What it does

Every item is adjustable through a config file, an environment variable, or a CLI flag.

Quick start

Requires Node.js ≥ 18.

# The default config already targets OpenCode Go — just start it
npx llm-session-proxy

# Client Base URL:  http://127.0.0.1:9355/zen/go/v1
# Model ID:         proxy- + the real model name, e.g. proxy-glm-5.3-flash
# API Key:          your OpenCode API key

npx llm-session-proxy \
  --upstream https://api.deepseek.com \
  --port 8788 \
  --inject "x-session-id={{session.id}}" \
  --inject "x-trace-id={{uuid}}" \
  --model-map fast=deepseek-chat \
  --model-map smart=deepseek-reasoner \
  --model-prefix ""

# Client Base URL: http://127.0.0.1:8788/v1
# Model names can be short aliases like fast / smart

# Generate an annotated sample config
npx llm-session-proxy --init

# Edit it, then start
npx llm-session-proxy -c llm-session-proxy.config.json

{
  // supports // and /* */ comments, and trailing commas
  "listen": { "host": "127.0.0.1", "port": 9355 },
  "upstream": { "host": "opencode.ai" },
  "inject": {
    "headers": {
      "x-opencode-session": "{{session.id}}",
      "x-opencode-request": "{{session.requestId}}"
    }
  },
  "model": { "stripPrefixes": ["proxy-"] }
}

What to put in the client's Base URL

OpenCode Go model cheat sheet

The upstream splits models across three endpoints — whether a model is usable depends on the protocol your client speaks, which matters more than the model name. Every client model ID below resolves out of the box: the aliases are built into the proxy. Curious what any alias turns into? npx llm-session-proxy --dry-run --model proxy-glm.

① /zen/go/v1/chat/completions · OpenAI-compatible, what most clients use

② /zen/go/v1/responses · OpenAI Responses API, client must support that protocol

③ /zen/go/v1/messages · Anthropic protocol, client must be able to speak it

Configuration reference

Precedence: defaults < config file < environment variables < CLI flags.

upstream — where requests are forwarded

session — session IDs

inject — parameter injection

model — model name rewriting

transformers — body transforms, always on

Names only: the registry is a fixed list compiled into the proxy, so a config file can never make it execute code.

Order matters, because each transformer mutates the body in place: rename-fields before drop-fields is not the same as the reverse. A name that is not in the table is a startup error, not a silent no-op.

router — buckets and rules

A bucket decides two things: which model to use and which transforms to attach. enabled is false by default, so an existing config behaves exactly as before until you turn it on.

"router": {
  "enabled": true,
  "defaultBucket": "default",
  "buckets": {
    "think":       { "model": "glm-5.3-think" },
    "longContext": { "model": "glm-5.3-long" },
    "background":  { "model": "glm-5.3-flash", "transformers": ["clamp-max-tokens"] }
  },
  "rules": [
    { "bucket": "think",       "path": "/zen/go/v1/messages" },
    { "bucket": "longContext", "minBytes": 60000 },
    { "bucket": "background",  "modelPrefix": "proxy-haiku", "maxBytes": 4096 }
  ]
}

A bucket's transformers are appended to the global transformers.enabled list — a bucket can add transforms but cannot cancel a global one. A bucket's model is applied after alias rewriting and replaces whatever was there, so it should be a real upstream ID rather than a proxy- alias. {{model}} in an injected header sees the bucket's value, because injection runs last.

protocol — convert between protocols

The proxy detects the client's protocol from the request path (/chat/completions → chat, /messages → messages, /responses → responses) and a route decides what the upstream speaks. When the two differ, request bodies are converted before forwarding and responses converted back — JSON as a whole, SSE event by event. enabled is false by default.

"protocol": {
  "enabled": true,
  "paths": {
    "chat": "/zen/go/v1/chat/completions",
    "messages": "/zen/go/v1/messages",
    "responses": "/zen/go/v1/responses"
  },
  "routes": [
    { "model": "minimax", "target": "messages" },
    { "model": "gpt", "target": "responses" }
  ]
}

Field normalisation is built in: max_tokens ↔ max_completion_tokens ↔ max_output_tokens; reasoning_effort ↔ thinking.budget_tokens (fixed table 1024 / 8192 / 16384, back by threshold); tool declarations and tool calls reshaped between the three formats; stop ↔ stop_sequences; chat's system message ↔ Anthropic's top-level system ↔ Responses' instructions. Fields with no equivalent are dropped by name into a logged dropped list. Anthropic requires max_tokens, so a chat request without either name gets a conservative, logged 4096 instead of a 400. Upstream 4xx/5xx error bodies are never converted — they belong to the upstream's protocol and pass through verbatim.

Template variables

CLI reference

Every flag mixes with a config file; CLI flags win.

Logs

Logging to disk is on by default, because the failures that are hardest to diagnose — an uncaught exception, a silent crash — leave nothing behind on stderr once the terminal is gone. The file is $LSP_HOME/logs/llm-session-proxy.log, or ~/.lsp/logs/llm-session-proxy.log when LSP_HOME is unset. Override it with --log-file <path> / --log-dir <dir>, or switch it off with --no-log-file.

Archival. On startup, and at most once every six hours while running, rotated files (app.log, app.log.N, app-YYYY-MM-DD.log) older than log.keepDays — 30 days by default — are deleted. The file currently being written is never touched, and files that do not match that naming pattern, including anything else in the same directory, are left alone.

# Where will logs actually land?
llm-session-proxy --print-config | grep resolvedFile

Preflight: --dry-run and --doctor

Both flags validate the config and print what the proxy would actually do, then exit — no server, no log file, no request to the upstream. The injection table is not a description of the templates; it is the rendered result, so a misspelled template shows up here instead of in a 400 from the upstream.

$ llm-session-proxy --dry-run
llm-session-proxy v0.2.2 — dry run

Model
  sample              proxy-glm
  strip               prefix "proxy-" -> glm
  mapped              glm -> glm-5.3
  result              glm-5.3  (mapped)
  map                 27 built-in aliases, 0 overrides

Router
  enabled             no
  default bucket      default
  bucket default      (none)
  bucket background   (none)
  bucket think        (none)
  bucket longContext  (none)
  rules               (no rules)
  sample route        default (router disabled)

Transformers
  global              (none)
  effective           (none)
  available           noop, drop-fields, drop-empty-fields, rename-fields, clamp-max-tokens

Protocol
  enabled             no (pass-through)
  paths               chat=/zen/go/v1/chat/completions, messages=/zen/go/v1/messages, responses=/zen/go/v1/responses
  rules               (no rules)
  sample conversion   no conversion for the sample request

Result
  OK — the configuration is valid.

The Router and Transformers blocks answer "which bucket would this request land in, and what would be done to it". For the config shown under router above:

Router
  enabled             yes
  default bucket      default
  bucket default      (none)
  bucket background   model=glm-5.3-flash transformers=clamp-max-tokens
  bucket think        model=glm-5.3-think
  bucket longContext  model=glm-5.3-long
  #0                  path^=/zen/go/v1/messages -> think
  #1                  bytes>=60000 -> longContext
  #2                  model~=proxy-haiku* AND bytes<=4096 -> background
  sample route        default  (default bucket, no rule matched /v1/chat/completions)

Transformers
  global              drop-empty-fields
  effective           drop-empty-fields
  available           noop, drop-fields, drop-empty-fields, rename-fields, clamp-max-tokens

#0/#1/#2 are the rules in order, with path^= meaning "path starts with" and model~= meaning "model prefix". The sample route row is the result of actually running the matcher over a sample POST /v1/chat/completions, so it lands on the fallback: nothing matched and the request goes to the default bucket. --router think overrides the whole thing: sample route think (forced by --router). effective is what will really run — the global list first, then whatever the winning bucket appends. In Chinese output these two blocks are titled 路由分桶 and 变换, deliberately distinct from the pre-existing 路由 (path rewriting) section.

The Protocol block answers "would this request be converted, and to what". With --protocol messages it shows the forced target and, on the sample conversion row, the direction plus the upstream path the converted request takes (chat -> messages /zen/go/v1/messages). In Chinese output the section is titled 协议互转.

--doctor runs the same report and adds DNS resolution, a TCP connect with handshake time, a TLS handshake when the protocol is https (an untrusted certificate is reported, not treated as a hard failure), a listen-port availability check, and a reminder that the proxy never injects credentials. It sends no HTTP request and no credentials — reachability is answered at the transport layer — so a doctor run cannot burn rate-limit quota. It exits 0 when everything is fine and 1 when something needs fixing, which makes it usable as a startup gate:

llm-session-proxy --doctor && llm-session-proxy

Local status endpoints

# Overview: session count, hit rate, injected headers, upstream target
curl http://127.0.0.1:9355/__llm_session_proxy__/status

# Per-session detail: ID, request count, last used
curl http://127.0.0.1:9355/__llm_session_proxy__/sessions

Handy when debugging: if sessions.active stays at 0, requests are not reaching the proxy at all — usually the model ID is missing its proxy- prefix and the client intercepted it.

How it works

Each request goes through a fixed sequence; the three-tier session strategy is the key part.

The order of 03–06 is deliberate. Rules see the model name the client sent and the resolved one; the bucket's model override lands after alias resolution, so it can only be a real upstream ID; injection comes last-but-one, so {{model}} in a header reflects the final decision; and protocol conversion runs last of all — transforms and injection work on the client's field names, and only the finished body is translated.

The three-tier session strategy

Explicit identifier — the client sent x-opencode-session itself, or the body carries session_id. Reused as-is; this is the most accurate path.

Content fingerprint — SHA-256 over system plus the first user message. Later turns in the same conversation only append messages, so the first one never changes, the fingerprint stays stable, and the turn lands back on the same ID.

One-off random — when neither is available (a non-JSON body, say), a random ID is sent so the upstream does not return 400. These requests are kept out of the session table so it cannot grow without bound.

Stability

A single malformed request cannot take the process down. An invalid Host header, a malformed request line, or an upstream status line / header with illegal characters all become ordinary 4xx / 5xx responses, and the proxy keeps serving.

Uncaught errors land in the log file. A last-resort handler writes the full stack of uncaughtException and unhandledRejection into the log file. Node's default prints to stderr and terminates immediately, leaving the log file empty — which looks exactly like "logs are fine, the process just vanished". That is why file logging is on by default: see Logs.

Limitations

Loopback only. The proxy forwards your API key, so do not bind the listen address to 0.0.0.0.

response.stream: false breaks SSE. Keep the default true unless you genuinely want whole-response buffering.

With bufferBody: false the body cannot be rewritten — model rewriting and parameter injection stop working, and only headers can be injected.

The built-in model map is a snapshot, not a live catalogue. Upstream model IDs change without notice, and this proxy deliberately makes no network call to discover them. If an alias stops resolving, it is a one-line override in your config file — not a bug — and --dry-run shows which alias resolved to what before you send a real request.

Protocol conversion is deliberately lossy at the edges. Anthropic thinking / signature stream deltas have no chat equivalent and are dropped (by name, in the log); messages ↔ responses composes through chat rather than a direct converter. What survives is text, tool calls, finish reasons and usage — the parts clients actually act on. The transform registry remains a fixed list: the proxy never loads a transform from a path.

Roadmap

Zero dependencies, local, single process — every item has to fit that shape. Full reasoning, the comparison with comparable gateways and the explicit non-goals live in ROADMAP.md.

The v0.2 performance targets are measured by a harness that ships inside the repo: added latency of ≤ 1 ms p50 and ≤ 5 ms p99 against a local mock upstream, ≥ 1,200 RPS at 100 concurrent connections, zero process exits under the malformed-input corpus, and byte-for-byte SSE pass-through. Numbers get published together with their methodology — without it a benchmark says nothing.

Deliberately not on the roadmap: virtual keys and billing, running models locally, a hosted version, a plugin marketplace, a Kubernetes operator. Each of them pulls against “zero dependencies, local, single process”.