Streaming

Overview

Streaming lets you receive partial output as it is generated, which improves perceived latency and user experience. For new OpenAI-style integrations whose selected model exposes a native Responses endpoint, prefer Responses streaming. Responses availability requires the model details to advertise that request format and a same-protocol route to be currently available. If your framework or model uses Chat Completions streaming, AI Sonar supports that compatibility path too.

Recommended: Responses Streaming

curl https://api.aisonar.dev/v1/responses \
  -H "Authorization: Bearer sk-your-api-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5.4",
    "input": "Write a short poem.",
    "stream": true
  }'

from openai import OpenAI

client = OpenAI(
    api_key="sk-your-api-key",
    base_url="https://api.aisonar.dev/v1"
)

stream = client.responses.create(
    model="gpt-5.4",
    input="Write a short poem.",
    stream=True
)

for event in stream:
    if event.type == "response.output_text.delta":
        print(event.delta, end="", flush=True)

import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: 'sk-your-api-key',
  baseURL: 'https://api.aisonar.dev/v1'
});

const stream = await client.responses.create({
  model: 'gpt-5.4',
  input: 'Write a short poem.',
  stream: true
});

for await (const event of stream) {
  if (event.type === 'response.output_text.delta') {
    process.stdout.write(event.delta);
  }
}

AI Sonar preserves native Responses SSE event names, order, and fields. It parses usage and terminal state out of band but does not rewrite the wire events. Once the first event has been delivered, the gateway does not retry the request.

Responses WebSocket

Connect to wss://api.aisonar.dev/v1/responses and send official response.create events. The WebSocket surface does not expose response.cancel; stream is implicit and background is unsupported on this transport. A response.create with generate: false creates a warmup response with no model output or charge and returns an ID that can be continued. Each connection processes one active response at a time, does not multiplex responses, and has a 60-minute maximum lifetime. Gateway-created response events use a response-scoped monotonic sequence_number. Stream events named error use the flat event shape; connection and protocol errors retain their nested error object.

Chat Completions Streaming

If your framework still expects SSE chunks from /v1/chat/completions, that also works:

stream = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Write a short poem"}],
    stream=True
)

for chunk in stream:
    content = chunk.choices[0].delta.content
    if content:
        print(content, end="", flush=True)

Gemini Streaming

POST /v1beta/models/{model}:streamGenerateContent?alt=sse preserves Gemini-native chunks. Metadata-only events, intermediate chunks without finishReason, and natural EOF are valid. AI Sonar does not append a Chat Completions [DONE] marker.

Stream End Conditions

Typical completion conditions:

response.completed for Responses API streams
finish_reason: "stop" for Chat Completions streams
finish_reason: "length" when a token limit is hit
tool/function call events when the model wants to use tools

Web App Pattern

async function streamChat(message) {
  const response = await fetch('https://api.aisonar.dev/v1/chat/completions', {
    method: 'POST',
    headers: {
      'Authorization': 'Bearer sk-your-api-key',
      'Content-Type': 'application/json'
    },
    body: JSON.stringify({
      model: 'gpt-4o',
      messages: [{ role: 'user', content: message }],
      stream: true
    })
  });

  const reader = response.body.getReader();
  const decoder = new TextDecoder();

  while (true) {
    const { done, value } = await reader.read();
    if (done) break;

    const chunk = decoder.decode(value);
    const lines = chunk.split('\\n').filter(line => line.startsWith('data: '));

    for (const line of lines) {
      const data = line.slice(6);
      if (data === '[DONE]') return;
      const parsed = JSON.parse(data);
      const content = parsed.choices?.[0]?.delta?.content;
      if (content) {
        document.getElementById('output').textContent += content;
      }
    }
  }
}

Best Practices

Prefer Responses streaming for new builds

Use /v1/responses if your SDK or app already supports it. Keep /v1/chat/completions streaming for compatibility-driven integrations.

Flush output incrementally

Append delta chunks to the UI or terminal as they arrive rather than waiting for the full response.

Handle disconnects and retries

Treat network and service disconnects as normal failure modes and reconnect carefully for long-running sessions.

Getting Started

Core Guides

Media Guides

Overview

Recommended: Responses Streaming

Responses WebSocket

Chat Completions Streaming

Gemini Streaming

Stream End Conditions

Web App Pattern

Best Practices

​Overview

​Recommended: Responses Streaming

​Responses WebSocket

​Chat Completions Streaming

​Gemini Streaming

​Stream End Conditions

​Web App Pattern

​Best Practices

Overview

Recommended: Responses Streaming

Responses WebSocket

Chat Completions Streaming

Gemini Streaming

Stream End Conditions

Web App Pattern

Best Practices