Chat - Streaming

How the streaming protocol works end-to-end, from adapter response to live UI updates.

The Chat component streams assistant responses token-by-token. The adapter's sendMessage() method returns a ReadableStream<ChatMessageChunk | ChatStreamEnvelope>. The runtime reads this stream, processes each chunk, and updates the normalized store so that UI components re-render incrementally.

Stream lifecycle

Every stream must follow this lifecycle:

1. start — Begin a new assistant message. The runtime creates the message shell and sets its status to 'streaming'.
2. Content chunks — Text, reasoning, tool, source, file, or data chunks populate the message parts.
3. finish or abort — Terminal chunk. finish marks the message as 'sent'; abort marks it as 'cancelled'.

If the stream closes without a terminal chunk, the runtime treats it as a disconnect (see Error and disconnect handling below).

Chunk types

Lifecycle chunks

Text chunks

Text chunks create a ChatTextMessagePart in the message. Multiple text-delta chunks are batched according to streamFlushInterval before being applied to the store.

Reasoning chunks

Reasoning chunks create a ChatReasoningMessagePart, useful for chain-of-thought or thinking trace displays.

Tool chunks

The toolInvocation.state tracks the tool lifecycle: 'input-streaming' -> 'input-available' -> 'approval-requested' -> 'output-available' (or 'output-error' / 'output-denied').

Source, file, and data chunks

Data chunks use a type that matches the data-* pattern (for example, data-citations or data-summary). They create ChatDataMessagePart entries and also fire the onData callback. If transient is true, the part is not persisted in the message.

Step boundary chunks

Step chunks create ChatStepStartMessagePart entries, useful for showing multi-step agentic processing in the UI.

Building a stream

When writing an adapter, construct a ReadableStream from an array of chunks:

const adapter: ChatAdapter = {
  async sendMessage({ message }) {
    return new ReadableStream({
      start(controller) {
        controller.enqueue({ type: 'start', messageId: 'msg-1' });
        controller.enqueue({ type: 'text-start', id: 'text-1' });
        controller.enqueue({ type: 'text-delta', id: 'text-1', delta: 'Hello!' });
        controller.enqueue({ type: 'text-end', id: 'text-1' });
        controller.enqueue({ type: 'finish', messageId: 'msg-1' });
        controller.close();
      },
    });
  },
};

Or convert a server-sent event stream into chunks:

async function fromSSE(
  url: string,
  signal: AbortSignal,
): Promise<ReadableStream<ChatMessageChunk>> {
  const response = await fetch(url, { signal });
  const reader = response.body!.getReader();
  const decoder = new TextDecoder();
  let buffer = '';

  return new ReadableStream({
    async pull(controller) {
      const { done, value } = await reader.read();
      if (done) {
        controller.close();
        return;
      }
      buffer += decoder.decode(value, { stream: true });
      const lines = buffer.split('\n');
      buffer = lines.pop() ?? '';
      for (const line of lines) {
        if (line.startsWith('data: ')) {
          const data = line.slice(6).trim();
          if (data && data !== '[DONE]') {
            controller.enqueue(JSON.parse(data));
          }
        }
      }
    },
  });
}

Stream envelopes

For deduplication and ordering, wrap chunks in a ChatStreamEnvelope:

interface ChatStreamEnvelope {
  eventId?: string; // unique event identifier for deduplication
  sequence?: number; // monotonic ordering number
  chunk: ChatMessageChunk;
}

The runtime accepts both raw ChatMessageChunk objects and enveloped chunks in the same stream. Envelopes are useful for SSE-based transports where chunks might arrive out of order or be replayed.

Flush interval

Rapid text and reasoning deltas are batched before being applied to the store. The streamFlushInterval prop on ChatBox (or ChatProvider) controls the batching window. The default is 16 ms (approximately one frame at 60 fps).

<ChatBox adapter={adapter} streamFlushInterval={32} />

Stopping and cancelling streams

The sendMessage input includes an AbortSignal that fires when the user clicks the stop button. Pass it to your fetch call so the HTTP request is cancelled automatically:

async sendMessage({ message, signal }) {
  const res = await fetch('/api/chat', {
    method: 'POST',
    body: JSON.stringify({ message }),
    signal, // browser cancels the request when the user stops
  });
  return res.body!;
},

If your backend requires explicit cancellation (for example, sending a separate cancel request), implement the optional stop() method on the adapter alongside the signal:

stop() {
  fetch('/api/chat/cancel', { method: 'POST' });
},

You can also stop streaming programmatically using the useChat() hook:

const { stopStreaming } = useChat();

// Cancel the active stream
stopStreaming();

Reconnecting to streams

Implement the adapter's reconnectToStream() method to resume an interrupted stream — for example, when an SSE connection drops mid-response:

async reconnectToStream({ conversationId, messageId, signal }) {
  const res = await fetch('/api/chat/reconnect', {
    method: 'POST',
    body: JSON.stringify({ conversationId, messageId }),
    signal,
  });
  if (res.status === 404) return null; // message no longer resumable
  return res.body!;
},

Return null if the interrupted message cannot be resumed. The runtime calls reconnectToStream() automatically after detecting a disconnected stream.

Error and disconnect handling

If the stream closes without a terminal chunk (finish or abort), the runtime:

1. Records a recoverable stream error.
2. Sets the message status to 'error'.
3. Calls onError and onFinish with isDisconnect: true.
4. If reconnectToStream() is implemented, attempts to resume.

If the adapter's sendMessage() throws, the runtime records a send error and surfaces it through the error model. See Error handling for more details.

How chunks become message parts

The stream processor maps chunks to ChatMessagePart entries:

1. text-start creates a new ChatTextMessagePart with state: 'streaming'.
2. text-delta appends to the existing text part.
3. text-end sets state: 'done'.
4. Tool chunks follow a similar lifecycle, updating toolInvocation.state through each phase.
5. Source, file, and data chunks each create their corresponding part type immediately.
6. start-step creates a ChatStepStartMessagePart separator.

The message's status field also updates through the stream:

• 'sending' — set when the user message is optimistically added
• 'streaming' — set when start arrives
• 'sent' — set when finish arrives
• 'cancelled' — set when abort arrives
• 'error' — set when the stream fails

See also

• Adapter for the adapter interface that produces streams.
• Error handling for error model and recovery.
• Scrolling for how auto-scroll follows streaming content.

API

See the documentation below for a complete reference to all of the props and classes available to the components mentioned here.

• <ChatBox />