Streaming responses

Confirm a short non-streaming request before enabling streaming. The client should keep reading the response until the stream finishes and should tolerate empty delta content.

Before streaming

Use the current key and a model ID returned by the model list endpoint. Do not use a long production request as the first connection test.

Python

stream = client.chat.completions.create(
    model="your-model-id",
    messages=[{"role": "user", "content": "Explain this briefly."}],
    stream=True,
)

for chunk in stream:
    text = chunk.choices[0].delta.content
    if text:
        print(text, end="", flush=True)

Node.js

const stream = await client.chat.completions.create({
  model: "your-model-id",
  messages: [{ role: "user", content: "Explain this briefly." }],
  stream: true,
});

for await (const chunk of stream) {
  const text = chunk.choices[0]?.delta?.content;
  if (text) process.stdout.write(text);
}

Client disconnects

Stop rendering when the user leaves the page or cancels the request. Avoid automatically replaying the same long streaming request, because a retry may create a new request and new usage.