Skip to content

Streaming responses

Set stream to true to receive server-sent events. Read content deltas until the stream completes. A chunk can contain an assistant role or empty content while the service is working; that is not the answer or a reason to submit another request. Keep the connection open and show a clear working state in your UI.

stream = client.chat.completions.create(
model="Gaia",
messages=[{"role": "user", "content": "Explain plate tectonics."}],
stream=True,
stream_options={"include_usage": True},
)
for chunk in stream:
if chunk.choices:
text = chunk.choices[0].delta.content
if text:
print(text, end="", flush=True)
if chunk.usage is not None:
print("\nUsage:", chunk.usage)

The snippet uses the client configured in the quickstart. Handle a usage-only chunk with an empty choices list. At the wire level the stream ends with data: [DONE]. An interrupted stream may not contain final usage; do not treat missing usage as zero.

Authoritative usage reporting is still undergoing integration in the preview. The requested include_usage contract must pass release checks before paid API activation. Internal library and orchestration tool calls are not exposed as application tools to execute.