LLM Gatewaydocs

Chat Completions

POST/v1/chat/completions

Generate a model response for a conversation. The request and response follow the OpenAI Chat Completions API; the gateway adds an optional routing extension on the request and provider / usage.x_llm_cost_micro on the response.

Request

bash
curl "https://api.smartapihub.com/v1/chat/completions" \
  -H "Authorization: Bearer $LLM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4o-mini",
    "messages": [
      { "role": "system", "content": "You are a terse assistant." },
      { "role": "user", "content": "What is the capital of France?" }
    ],
    "temperature": 0.2,
    "max_tokens": 64
  }'

Fields

FieldTypeNotes
modelstring, requiredvendor/name id from /v1/models.
messagesarray, requiredAt least one message. role ∈ system, user, assistant, tool, developer. content is a string, null, or an array of parts (text, image_url, input_audio). Assistant messages may carry tool_calls; tool messages need tool_call_id.
streambooleanServer-Sent Events. See Streaming.
stream_options{ include_usage? }Accepted for compatibility. The gateway always emits a final usage chunk on streams.
temperaturenumber 0–2
top_pnumber 0–1
max_tokensinteger ≥ 1Also used for the pre-flight cost estimate; if omitted, the model's default max is assumed for the estimate.
max_completion_tokensinteger ≥ 1Newer alias for max_tokens.
stopstring or up to 4 strings
n1Only one choice is supported. Any other value is 400 validation_failed.
presence_penalty / frequency_penaltynumber −2–2
logit_biasobjecttoken id → bias.
seedintegerBest-effort determinism where the provider supports it.
userstring ≤ 256End-user identifier, forwarded to the provider.
toolsarrayFunction tools: { type: "function", function: { name, description?, parameters?, strict? } }.
tool_choicenone | auto | required | { type:"function", function:{ name } }
parallel_tool_callsboolean
response_format{ type: "text" } | { type: "json_object" } | { type: "json_schema", json_schema }
routingobjectPlatform extension, see below. Stripped before the request is forwarded.

Unknown fields are ignored. Parameters the selected model does not support (check supported_parameters on the model) may be dropped by the provider.

The routing extension

json
{
  "model": "meta/llama-3.1-70b-instruct",
  "messages": [{ "role": "user", "content": "…" }],
  "routing": {
    "strategy": "fastest",
    "providers": ["groq", "together"]
  }
}
FieldTypeEffect
strategypriority_cheapest | cheapest | fastest | priority | highest_availability | smartOverrides the organization's routing strategy for this request. smart is not yet available and falls back to priority_cheapest.
providersstring[] ≤ 10Provider slugs to restrict candidates to, in preference order. Providers not serving the model are ignored; if none remain the request fails with 503 no_provider_available.

Details in Routing.

Response

json
{
  "id": "chatcmpl-8f1c…",
  "object": "chat.completion",
  "created": 1725446400,
  "model": "openai/gpt-4o-mini",
  "provider": "openai",
  "choices": [
    {
      "index": 0,
      "message": { "role": "assistant", "content": "Paris." },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 24,
    "completion_tokens": 2,
    "total_tokens": 26,
    "prompt_tokens_details": { "cached_tokens": 0 },
    "x_llm_cost_micro": 5
  }
}
FieldNotes
modelThe gateway model id you requested (not the provider's internal model name).
providerSlug of the provider that produced this response.
choices[].finish_reasonstop, length, tool_calls, content_filter, or null on stream chunks.
usageAlways present. x_llm_cost_micro is the amount charged to your organization in micro-USD after discounts. x_llm_usage_estimated: true appears when the provider did not report token counts and the gateway estimated them.

Tool calls

Tool calling works exactly as in the OpenAI API: the model returns finish_reason: "tool_calls" with message.tool_calls[], you append a tool message with tool_call_id and the result, then call again.

json
{
  "role": "assistant",
  "content": null,
  "tool_calls": [
    { "id": "call_1", "type": "function", "function": { "name": "get_weather", "arguments": "{\"city\":\"Paris\"}" } }
  ]
}

Response headers

Every inference response — success or error, streaming or not — carries these headers:

HeaderExampleMeaning
X-Request-Idreq_01J6ZK3M9PQR7S8T9VUnique id for this request. Quote it in support tickets; find it in the dashboard Requests explorer.
X-LLM-ProvideropenaiProvider that served the request (final attempt after any fallback).
X-LLM-Modelopenai/gpt-4o-miniResolved gateway model id.
X-LLM-Routing-Reasonstrategy=priority_cheapest selected=openai priority=10 cost_micro=5 healthy=true fallbacks=azure-openaiCompact description of the routing decision.
X-LLM-Cost-Micro5Same value as usage.x_llm_cost_micro. On streams the headers are sent with the first chunk, before the final usage is known — rely on the final usage chunk for the charged amount.

Errors

Common failures for this endpoint (full list in Errors):

HTTPcodeCause
400validation_failedSchema violation; param names the field (e.g. messages.0.role).
403model_not_allowedKey scope excludes the model.
404model_not_foundUnknown or disabled model id.
402insufficient_credits / budget_exceeded / spending_limit_exceededEstimated cost exceeds balance or a budget.
413request_too_largeBody over the size limit.
429rate_limited / tokens_per_minute_exceeded / concurrency_exceededBack off per Retry-After.
503no_provider_availableNo healthy provider serves this model right now.
502 / 503 / 504provider_error / provider_unavailable / provider_timeoutAll providers failed after retries and fallbacks.

Legacy completions

POST/v1/completionsprompt-based legacy API

Accepts the OpenAI legacy body (model, prompt, max_tokens, temperature, top_p, stop, stream, n: 1, penalties, seed, user, plus routing). Internally the prompt is mapped to a single user message for providers that lack a native completions endpoint, and the response is returned as text_completion with choices[].text. Prefer chat completions for new code.

Migrating from OpenAIResponses