Streaming

Streaming requests

Set "stream": true and consume server-sent events exactly like the OpenAI API:

stream = client.chat.completions.create(
    base_url="https://api.runaii.cloud/v1",
    model="runaii/glm-5.3-flash",
    messages=[{"role": "user", "content": "Write a haiku about GPUs."}],
    stream=True,
)

for chunk in stream:
    delta = chunk.choices[0].delta.content
    if delta:
        print(delta, end="")

The terminal usage chunk

After the final content chunk — and before [DONE] — runaii emits one extra chunk carrying the full usage and cost for the request:

{
  "id": "rq-...",
  "object": "chat.completion.chunk",
  "choices": [{ "index": 0, "delta": {}, "finish_reason": "stop" }],
  "usage": {
    "prompt_tokens": 14,
    "completion_tokens": 32,
    "total_tokens": 46,
    "prompt_tokens_details": { "cached_tokens": 0 },
    "cost": 0.000017
  }
}

This means streaming clients can display spend in real time without a second API call — the same field set as non-streaming responses.

Billing on streams

Streaming requests are billed identically to non-streaming: estimate reserved at start, settled from the provider-reported token counts at stream end, refund of any unused reservation — all in one transaction. If your client disconnects mid-stream, the reservation is reversed; partial delivery is never charged.

Errors mid-stream

If the upstream cell dies mid-stream, the stream ends with a retryable error chunk and the request is fully refunded (usage.cost will not appear — nothing was charged). Safe pattern: catch the error and re-issue; the ledger guarantees you were not double-charged.