Chat Completions

POST /v1/chat/completions is the broadest OpenAI-compatible generation endpoint.

curl https://api.koscompute.com/v1/chat/completions \
  -H "Authorization: Bearer $KOSCOMPUTE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen/qwen3.8-27b",
    "messages": [{"role": "user", "content": "Give three names for a database proxy."}],
    "temperature": 0.4,
    "max_completion_tokens": 180
  }'

Messages and roles¶

messages must be a non-empty ordered conversation. Common roles are system, developer, user, assistant, and tool. Tool results must include the tool_call_id returned by the assistant. Multimodal user content is an array of typed text and image_url parts.

Completion budget¶

Prefer max_completion_tokens. The legacy max_tokens field is an alias for the same total completion-token limit. If both are sent, their values must match. Reasoning and visible output share this budget; a reasoning sub-budget never increases it.

The configured model maximum is published as max_output_length. A request above that maximum is normally rejected. finish_reason:"length" means the total completion budget was exhausted, including the possibility that reasoning consumed it before much visible text was produced.

Sampling¶

Common supported controls include temperature, top_p, top_k, penalties, stop, seed, logprobs, and top_logprobs. The router forwards only parameters advertised by the selected model. Do not set both temperature and top_p merely to make output "more deterministic"; tune one first.

Optional capabilities¶

• Set stream:true for SSE; stream_options.include_usage requests the final usage chunk.
• Send tools and tool_choice for local functions.
• Set response_format.type to json_object or json_schema on capable models.
• Use reasoning, reasoning_effort, or compatible thinking controls only where model metadata advertises reasoning.
• Send image_url content only to models with image input.

Response and headers¶

Non-streaming responses contain id, model, choices, and usage. A choice contains an assistant message and finish_reason. Successful generated responses include X-Request-ID and Inference-Id; public probes such as /health do not necessarily include them. X-Zero-Data-Retention: true states the body-retention contract.

See the generated endpoint reference for the schema and errors for retry decisions.