# Streaming

> Parsing SSE, the usage chunk, cancellation, streaming errors, the 180s ceiling, and when stream=true is required

来源：https://zhonkemodel.dflop.top/en/docs/guides/streaming

The platform supports SSE (Server-Sent Events) streaming. **The first token arrives sooner**, so your UI can render as generation happens.

## Basic usage

Client setup (base_url and API key) is covered in the [Quickstart](../quickstart.md); each snippet below repeats only the minimum.

### OpenAI SDK

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://zhonkezhonkeapi.dflop.top/v1",
    api_key="sk-gpushare-xxx",
)

stream = client.chat.completions.create(
    model="claude-sonnet-4-6",
    messages=[{"role": "user", "content": "Stream a haiku"}],
    stream=True,
)

for chunk in stream:
    delta = chunk.choices[0].delta.content or ""
    print(delta, end="", flush=True)
```

### Anthropic SDK

```python
from anthropic import Anthropic

client = Anthropic(
    base_url="https://zhonkezhonkeapi.dflop.top",  # ⚠️ no /v1
    api_key="sk-gpushare-xxx",
)

with client.messages.stream(
    model="claude-sonnet-4-6",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Stream a haiku"}],
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)
```

### Gemini SDK

```python
from google import genai

client = genai.Client(
    api_key="sk-gpushare-xxx",
    http_options={"base_url": "https://zhonkezhonkeapi.dflop.top"},
)

stream = client.models.generate_content_stream(
    model="gemini-2.5-pro",
    contents="Stream a haiku",
)
for chunk in stream:
    print(chunk.text, end="", flush=True)
```

## When stream=true is required

| Tool | Streaming required | Why |
|---|---|---|
| `image_generation` | ✅ | the upstream WebSocket channel has no synchronous response — it only streams |
| `web_search` | ✅ | same, and the search progress comes back live as stream chunks |
| `function` tools | either | they use the HTTP channel, streaming or not |
| Plain text chat | either | streaming or not |

Otherwise the gateway returns **400 `invalid_request_error`** with the message `` Tools `web_search` and `image_generation` require `stream: true` ``.

## SSE wire format

The gateway forwards the upstream SSE as-is: **each `data:` line is one chunk**.

### OpenAI Chat stream chunks

```
data: {"id":"chatcmpl-xxx","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"Hello"}}]}

data: {"id":"chatcmpl-xxx","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":" world"}}]}

data: {"id":"chatcmpl-xxx","object":"chat.completion.chunk","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: {"id":"chatcmpl-xxx","object":"chat.completion.chunk","choices":[],"usage":{"prompt_tokens":10,"completion_tokens":2,"total_tokens":12}}

data: [DONE]
```

Two things hand-written parsers get wrong most often:

- The `finish_reason` chunk **carries no usage** — usage arrives as a **separate trailing chunk** right after it, which is the standard `stream_options.include_usage` convention
- The usage chunk's `choices` is an **empty array**, so `chunk.choices[0].delta` goes out of bounds — check `chunk.choices` is non-empty before reading `delta`

### Anthropic stream events

```
event: message_start
data: {"type":"message_start","message":{...}}

event: content_block_delta
data: {"type":"content_block_delta","delta":{"type":"text_delta","text":"Hello"}}

event: message_stop
data: {"type":"message_stop"}
```

### Gemini streaming (SSE)

```
data: {"candidates":[{"content":{"parts":[{"text":"Hello"}]}}]}

data: {"candidates":[{"content":{"parts":[{"text":" world"}]},"finishReason":"STOP"}]}
```

The Gemini protocol has **no `[DONE]` sentinel** — the stream closing is the end signal, so don't wait for `[DONE]`.

## Parsing by hand (curl, or a client with no SDK)

```bash
curl https://zhonkezhonkeapi.dflop.top/v1/chat/completions \
  -H "Authorization: Bearer $PLATFORM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5.5",
    "stream": true,
    "messages": [{"role":"user","content":"Hello"}]
  }' \
  --no-buffer | while read line; do
    echo "$line"
  done
```

Parsing with JavaScript fetch:

```javascript
const resp = await fetch("https://zhonkezhonkeapi.dflop.top/v1/chat/completions", {
  method: "POST",
  headers: {
    "Authorization": `Bearer ${process.env.PLATFORM_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    model: "gpt-5.5",
    stream: true,
    messages: [{ role: "user", content: "Hello" }],
  }),
});

const reader = resp.body.getReader();
const decoder = new TextDecoder();
let buffer = "";

while (true) {
  const { done, value } = await reader.read();
  if (done) break;
  buffer += decoder.decode(value, { stream: true });

  const lines = buffer.split("\n");
  buffer = lines.pop() ?? "";

  for (const line of lines) {
    if (!line.startsWith("data: ")) continue;
    const data = line.slice(6);
    if (data === "[DONE]") return;
    const chunk = JSON.parse(data);
    process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
  }
}
```

## Getting token usage

OpenAI Chat streaming does **not** return a trailing usage chunk by default. To get real token counts, add `stream_options`:

```python
stream = client.chat.completions.create(
    model="gpt-5.5",
    messages=[...],
    stream=True,
    stream_options={"include_usage": True},  # the important bit
)

last_chunk = None
for chunk in stream:
    if chunk.choices:
        print(chunk.choices[0].delta.content or "", end="")
    last_chunk = chunk

# the final chunk carries usage
print(f"\nTokens: {last_chunk.usage}")
```

Internally the gateway **auto-injects** `stream_options.include_usage=true` for most upstream channels (except GLM, which sends a usage chunk anyway). So you'll get token counts without setting it — but setting it explicitly is more robust.

## Interrupting and cancelling

### OpenAI SDK (Python)

```python
stream = client.chat.completions.create(..., stream=True)
try:
    for chunk in stream:
        if some_user_canceled():
            stream.close()  # close explicitly; the gateway sees a client disconnect
            break
        print(chunk.choices[0].delta.content or "", end="")
except KeyboardInterrupt:
    stream.close()
```

### fetch (JavaScript)

```javascript
const controller = new AbortController();

const resp = await fetch("https://zhonkezhonkeapi.dflop.top/v1/chat/completions", {
  signal: controller.signal,
  // ...
});

// then, to interrupt:
controller.abort();
```

When the client disconnects, the gateway's upstream connection **closes immediately too** (on both HTTP and WebSocket channels) — the mechanism is a dropped connection rather than an explicit cancel instruction, and the effect is that the upstream stops generating. **Tokens already generated are still billed**; if the disconnect means the trailing usage chunk never arrived, the gateway books output tokens estimated from the characters already streamed.

## Error handling

### Before the stream opens: an ordinary HTTP error, not SSE

Auth failure (401), exhausted balance (402), a nonexistent model (400 `model_not_found`) and no available channel (503) all happen **before the stream opens** and come back as an ordinary HTTP JSON error body — there's no SSE yet, so handle them like non-streaming errors. Full list: [Error handling](../reference/errors.md).

### After the stream opens (HTTP 200): it depends on the path

**Tool requests (`web_search` / `image_generation`, on the WS V2 channel)**: a mid-stream error is synthesised into an error chunk followed by `[DONE]`:

```
data: {"error":{"message":"<the specific reason>","type":"api_error","code":"upstream_error"}}

data: [DONE]
```

`type` and `code` are **fixed values** (`api_error` / `upstream_error`), so don't branch on `code` to classify the error — the actual reason is only in the `message` text. When parsing, **check for an `error` field before handling `delta.content`**, and treat `error` as the end-of-turn signal.

**Plain text chat and every `/v1/messages`, `/v1/responses` and `/v1beta` path**: when the upstream fails mid-stream the gateway **synthesises nothing** and the stream simply **ends early** — there's no guarantee of `[DONE]`, `finish_reason`, `message_stop` or any error frame. A robust client must treat "stream ended without a terminator" as an interruption, not a normal completion. (Error events native to the upstream protocol, such as Anthropic's `event: error`, are forwarded verbatim — but those come from the upstream; the gateway never generates them.)

On either path, whatever content streamed before the error or interruption is partially valid and can be kept on screen.

### The ~180-second ceiling

A single request — streaming included, across `/v1/chat/completions`, `/v1/messages`, `/v1/responses` and every `/v1beta` chat endpoint — has a **180-second total ceiling**. Streaming just delivers the first token sooner; it **doesn't exempt you from the total**. A timeout looks exactly like the "stream ends early" case above: if long answers always get cut off around the three-minute mark, this is almost certainly why. Long generations with a large `max_tokens`, slow models and long `image_generation` turns can all hit it. What to do:

- Lower `max_tokens`, or split long work across several turns
- Set your SDK timeout to **≥ 200s** (slightly above the gateway ceiling, so the gateway is the one that decides it timed out)
- Tokens already produced are billed as usual

Per-endpoint timeouts: [API reference](../reference/api-reference.md).

## Reverse proxies and Nginx

The gateway sets these response headers:
```http
Content-Type: text/event-stream
Cache-Control: no-cache, no-store, no-transform
X-Accel-Buffering: no
```

If you proxy this gateway behind your own reverse proxy, make sure **buffering is off** (`proxy_buffering off;` in Nginx) — otherwise the whole streamed response is buffered until it completes and delivered in one go, which defeats the point.

## Debugging

Watching the raw SSE with curl is the most direct approach:

```bash
curl https://zhonkezhonkeapi.dflop.top/v1/chat/completions \
  -H "Authorization: Bearer $PLATFORM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-5.5","stream":true,"messages":[{"role":"user","content":"hi"}]}' \
  --no-buffer 2>&1 | head -20
```

If you see the response arrive **all at once** instead of chunk by chunk, something in between is buffering.