Streaming failures
Detect incomplete SSE streams and terminal errors after HTTP headers are sent.
An SSE request can receive HTTP 200 and still fail later. Once headers are sent, HTTP does not allow the server to replace the status line with 502 or 504.
Detect completion¶
- Chat Completions should reach a terminal choice and
[DONE]. - Responses should reach
response.completed;response.failedor an error event is terminal failure. - Anthropic Messages should reach
message_stop.
If the socket closes without the protocol's normal terminal event, mark the result incomplete. Do not silently treat accumulated text as a complete answer.
Retry decisions¶
Retrying a pure text request is usually safe but can return different content. For tool workflows, track whether a complete tool call was received and whether the tool executed. Never execute partial JSON arguments.
Use an application deadline long enough for reasoning pauses, while keeping a separate idle timeout to detect dead connections. Preserve the response request ID when available.