---
title: "Streaming"
description: "Stream server-side AI edits into the document in real time, so changes appear in the browser as the model generates them."
canonical_url: "https://tiptap.dev/docs/ai/ai-toolkit/agents/streaming"
---

# Streaming

Stream server-side AI edits into the document in real time, so changes appear in the browser as the model generates them.

> **Alpha:**
>
> Streaming is in alpha: it's not recommended for production deployments just yet, and the API may
> change. We expect it to reach beta in the coming weeks. If you need production-readiness before
> that, check out [streaming in the AI Toolkit client
> package](https://tiptap.dev/docs/ai/ai-toolkit/client/agents/streaming.md).

Stream AI edits from the server so they appear in the document in real time, as the model generates them. The edit runs on your server and is applied to the Tiptap Cloud document over a live connection, so every collaborator sees it happen.

> **Requires Collaboration:**
>
> Streaming applies the edit to a live Tiptap Cloud document and syncs it to the editor through
> [Collaboration](https://tiptap.dev/docs/collaboration/getting-started/overview.md), so both are required. Unlike the
> non-streaming tools, there is no inline-document mode. If you are not on Tiptap Cloud
> Collaboration yet, you will need it to stream edits from the server.

> **Interactive demo:** [server ai stream tool chatbot](https://ai-toolkit-demos.vercel.app/server-ai-stream-tool-chatbot)

See the [source code on GitHub](https://github.com/ueberdosis/ai-toolkit-demos).

## Tech stack

- A client-side [React](https://react.dev/) application
- A server application
- An AI agent framework like the [Vercel AI SDK](https://ai-sdk.dev/)
- [OpenAI](https://openai.com/) models
- The Server AI Toolkit

The Server AI Toolkit works with any AI agent framework and AI model. Use it with any frontend framework and backend programming language. This guide uses a simple Node.js server endpoint that takes a [Request](https://developer.mozilla.org/en-US/docs/Web/API/Request) object and returns a [Response](https://developer.mozilla.org/en-US/docs/Web/API/Response).

Before you continue, set up the Server AI Toolkit by following the [installation guide](https://tiptap.dev/docs/content-ai/capabilities/server-ai-toolkit/install.md).

## How it works

1. Your server fetches the `tiptapEdit` tool definition and reads the document, just as it does with a [non-streaming agent](https://tiptap.dev/docs/ai/ai-toolkit/agents/ai-agent-chatbot.md).
2. As the model generates the tool call, your server forwards the raw input deltas to `POST /v4/ai/toolkit/stream-tool` as [NDJSON](https://github.com/ndjson/ndjson-spec) `start`, `delta`, and `end` messages.
3. From there, the Server AI Toolkit applies the edit for you: as the forwarded deltas arrive, it incrementally updates the Tiptap Cloud document over a live connection. You never apply anything yourself; the document updates happen automatically on the server side as the deltas stream in.
4. The client renders the changes through [Collaboration](https://tiptap.dev/docs/collaboration/getting-started/overview.md). The editor never calls a streaming method itself, so the edits arrive as document updates.

## Stream an edit

> **Works with most AI backends:**
>
> This example uses the [AI SDK by Vercel](https://ai-sdk.vercel.app/), but you can drive
> `stream-tool` from most AI back-end setups. You just forward your model's tool-call input as
> NDJSON deltas.

On the server, open a [duplex](https://developer.mozilla.org/en-US/docs/Web/API/RequestInit#duplex) NDJSON request to `stream-tool` and drive it with your model's tool-call stream. A duplex request lets you stream the body up incrementally, so you can forward each tool-call delta as the model produces it instead of sending the whole input at once.

The `stream-tool` request and response are both NDJSON: one JSON object per line. You send three message types: `start` to open the stream, `delta` for each chunk of the AI-generated tool input, and `end` once the input is complete.

Install dependencies

Install these dependencies in your server app:

```bash
npm install ai zod
```

```ts
import { gateway, stepCountIs, streamText, tool, type UIMessage } from 'ai'
import { z } from 'zod'
import { getAuthHeaders } from './server-ai-toolkit/get-auth-headers'
import { streamToolFetch } from './server-ai-toolkit/stream-tool-fetch'

// The Server AI Toolkit API base URL
const apiBaseUrl = process.env.TIPTAP_CLOUD_AI_API_URL || 'https://api.tiptap.dev'

export async function handleStreamEditRequest(request: Request): Promise<Response> {
  const { messages, editorContext, documentId } = await request.json()
  // Single-shot edit: the task is the latest user message.
  const lastUser = (messages as UIMessage[]).filter((m) => m.role === 'user').at(-1)
  const task =
    lastUser?.parts
      .filter((p) => p.type === 'text')
      .map((p) => (p as { text: string }).text)
      .join(' ') ?? ''

  // 1. Fetch the tiptapEdit tool definition and read the document.
  //    These steps match the non-streaming flow. See the REST API reference.
  const { tools, systemPrompt } = await fetch(`${apiBaseUrl}/v4/ai/toolkit/fetch-tools`, {
    method: 'POST',
    headers: await getAuthHeaders(),
    body: JSON.stringify({ editorContext, tools: { tiptapEdit: true } }),
  }).then((res) => res.json())
  const tiptapEdit = tools.find((t) => t.name === 'tiptapEdit')

  const read = await fetch(`${apiBaseUrl}/v4/ai/toolkit/execute-tool`, {
    method: 'POST',
    headers: await getAuthHeaders(documentId),
    body: JSON.stringify({
      tool: { name: 'readDocument', input: {} },
      document: { type: 'cloud', id: documentId },
      editorContext,
    }),
  }).then((res) => res.json())

  // 2. Open a duplex NDJSON request to stream-tool. Each message is one line.
  const encoder = new TextEncoder()
  let controller: ReadableStreamDefaultController<Uint8Array>
  const body = new ReadableStream<Uint8Array>({ start: (c) => (controller = c) })
  const send = (message: object) =>
    controller.enqueue(encoder.encode(`${JSON.stringify(message)}\n`))

  // `streamToolFetch` sends the request over HTTP/2 so the duplex upload
  // survives a reverse proxy (an HTTP/1.1 proxy drains the request body once the
  // response starts, which truncates the stream). See the demo source for the
  // helper. `duplex: 'half'` is required whenever you stream a request body.
  const upstream = streamToolFetch(`${apiBaseUrl}/v4/ai/toolkit/stream-tool`, {
    method: 'POST',
    headers: { ...(await getAuthHeaders(documentId)), 'Content-Type': 'application/x-ndjson' },
    body,
    duplex: 'half',
  })

  // 3. Drive the model. Forward its tool input as start / delta / end messages.
  const result = streamText({
    model: gateway('openai/gpt-5.4-mini'),
    system: `${systemPrompt}\n\nMake the edit with tiptapEdit, then reply with one short sentence confirming what you changed.`,
    prompt: JSON.stringify({ content: read.tool.output, task }),
    tools: {
      tiptapEdit: tool({
        description: tiptapEdit.description,
        inputSchema: z.fromJSONSchema(tiptapEdit.inputSchema),
        // Constrain sampling to the tool's input schema.
        strict: true,
        onInputStart: () =>
          send({
            version: 1,
            type: 'start',
            editorContext,
            document: { type: 'cloud', id: documentId },
            tool: { name: 'tiptapEdit' },
            user: 'ai-assistant',
          }),
        onInputDelta: ({ inputTextDelta }) =>
          send({ version: 1, type: 'delta', argsTextDelta: inputTextDelta }),
        onInputAvailable: () => {
          send({ version: 1, type: 'end' })
          controller.close()
        },
        // Wait for the AI server to finish applying the edit, then report a
        // result so the chat turn completes.
        execute: async () => {
          const res = await upstream
          await res.text()
          return { ok: res.ok }
        },
      }),
    },
    // Step 0: force the edit. Step 1: forbid tools so the model writes the
    // confirmation sentence for the chat.
    stopWhen: stepCountIs(2),
    prepareStep: ({ stepNumber }) => ({
      toolChoice: stepNumber === 0 ? 'required' : 'none',
    }),
  })

  // The edits reach the editor via Y.Doc sync; this stream drives the chat.
  return result.toUIMessageStreamResponse()
}
```

See the [REST API reference](https://tiptap.dev/docs/ai/ai-toolkit/api-reference/rest-api.md#stream-a-tool) for the full request and response message shapes, and the [demo source](https://github.com/ueberdosis/ai-toolkit-demos) for a production-ready route with error handling.

## Client-side setup

Install dependencies

Install these dependencies in your client app:

```bash
npm install @ai-sdk/react @tiptap/extension-collaboration @tiptap/react @tiptap/starter-kit @tiptap-pro/provider @tiptap/ai-toolkit ai yjs
```

On the client, add the `Collaboration` and `ServerAiToolkit` extensions, connect the document with a `TiptapCollabProvider`, and drive the chat with the AI SDK `useChat` hook (the same as the other server demos). The streamed edits arrive in the editor through Collaboration; `useChat` shows the model's replies.

```tsx
import { useChat } from '@ai-sdk/react'
import { Collaboration } from '@tiptap/extension-collaboration'
import { EditorContent, useEditor } from '@tiptap/react'
import StarterKit from '@tiptap/starter-kit'
import { ServerAiToolkit, getEditorContext } from '@tiptap/ai-toolkit'
import { DefaultChatTransport } from 'ai'
import { useRef } from 'react'

const editor = useEditor({
  immediatelyRender: false,
  extensions: [
    StarterKit.configure({ undoRedo: false }),
    Collaboration.configure({ document: doc }),
    ServerAiToolkit,
  ],
})

// editorContext in a ref to avoid an AI SDK stale-closure (vercel/ai#7819).
const editorContextRef = useRef(editor ? getEditorContext(editor) : null)
editorContextRef.current = editor ? getEditorContext(editor) : null

const { messages, sendMessage, status } = useChat({
  transport: new DefaultChatTransport({
    api: '/api/stream-edit',
    body: () => ({
      editorContext: editorContextRef.current,
      documentId: 'your-document-id',
    }),
  }),
})
```

## Stream as tracked changes

To let users review streamed edits instead of applying them directly, add `reviewOptions` to the `start` message. The edit then appears as a tracked-change suggestion that's typed into the document in real time and that users can accept or reject.

```ts
send({
  version: 1,
  type: 'start',
  editorContext,
  document: { type: 'cloud', id: documentId },
  tool: { name: 'tiptapEdit' },
  user: 'ai-assistant',
  reviewOptions: { mode: 'trackedChanges' },
})
```

Add the [Tracked Changes](https://tiptap.dev/docs/tracked-changes/getting-started/overview.md) extension on the client to render and accept or reject the suggestions. See the [Use with Tracked Changes guide](https://tiptap.dev/docs/ai/ai-toolkit/agents/tracked-changes.md) and the [Tracked Changes streaming demo](https://ai-toolkit-demos.vercel.app/server-ai-stream-tool-chatbot-tracked-changes).

## End result

The result is a simple but polished AI chatbot application:

> **Interactive demo:** [server ai stream tool chatbot](https://ai-toolkit-demos.vercel.app/server-ai-stream-tool-chatbot)

See the [source code on GitHub](https://github.com/ueberdosis/ai-toolkit-demos).

## Next steps

- [REST API reference](https://tiptap.dev/docs/ai/ai-toolkit/api-reference/rest-api.md#stream-a-tool): the full `stream-tool` request and response protocol.
- [Review options](https://tiptap.dev/docs/ai/ai-toolkit/api-reference/review-options.md): all `reviewOptions` settings.
- [Use with Tracked Changes](https://tiptap.dev/docs/ai/ai-toolkit/agents/tracked-changes.md): review server-side edits before they're accepted.
- [Streaming with the client-side AI Toolkit](https://tiptap.dev/docs/ai/ai-toolkit/client/agents/streaming.md): stream into the Tiptap Editor client-side, with the client variant of the AI Toolkit.