Streaming

The proxy supports streaming responses. Set stream: true (OpenAI) or use the streaming Messages call (Anthropic) and KINDI relays the provider's Server-Sent Events, unmasking each frame as it passes through.

stream must be the JSON boolean true. KINDI tests the field by identity, so the string "true", 1, and "1" do not enable streaming; the request is served as a batch call and you get a single JSON body instead of an SSE stream. This bites hand-rolled clients that assemble the body from form values, query strings, or environment variables; the official SDKs always send a real boolean.

How streaming unmask works

When the provider streams text, a single <MASKED_…> token can be split across two or three SSE deltas. KINDI runs a small per-stream state machine that buffers just enough to detect a token straddling a delta boundary, substitutes the original PII once the full token has arrived, and flushes any residual buffer at end-of-stream. You receive the provider's native SSE shape with the tokens already restored.

1234567891011121314from openai import OpenAI
from kindi import proxy_config

cfg = proxy_config("openai", "mk_live_...", "sk-...")
client = OpenAI(base_url=cfg["base_url"], default_headers=cfg["default_headers"], api_key="unused")

stream = client.chat.completions.create(
    model="gpt-5.4-mini",
    messages=[{"role": "user", "content": "Summarize: patient John Doe, ID 1012345672."}],
    stream=True,
)
for chunk in stream:
    delta = chunk.choices[0].delta.content or ""
    print(delta, end="", flush=True)  # PII already restored per frame

What to watch for

Mid-stream provider errors are billed. Once KINDI has masked your prompt and started forwarding to the provider, the KINDI masking fee has already been charged. If the provider fails partway through the stream (rate limit, content filter, network drop), that is the provider's event; and on your own key, the provider may still bill the partial completion. KINDI does not refund the masking fee for a provider-side stream failure (the masking work was done).
Token preservation matters more when streaming. Long streamed completions are where models are most likely to paraphrase or mangle a <MASKED_…> token. If exact token survival is important, opt into preserve tags.
Managed demo length is bounded. On the managed-demo path, output is clamped by a max-output cap, so a streamed demo response is shorter than an own-key response. See Limitations.