---
title: "Streaming"
description: "Add real-time streaming to your AI agent chatbot to see changes as they happen."
canonical_url: "https://tiptap.dev/docs/content-ai/capabilities/ai-toolkit/agents/streaming"
---

# Streaming

Add real-time streaming to your AI agent chatbot to see changes as they happen.

> **Continuation from the AI agent chatbot guide:**
>
> This guide continues the [AI agent chatbot
> guide](https://tiptap.dev/docs/content-ai/capabilities/ai-toolkit/agents/ai-agent-chatbot.md). Read it first.

Activate the AI Toolkit's tool streaming capabilities to update the document in real-time while the AI is generating content.

> **Interactive demo:** [tool streaming](https://ai-toolkit-demos.vercel.app/tool-streaming)

See the [source code on GitHub](https://github.com/ueberdosis/ai-toolkit-demos).

## Key changes

To add streaming to the AI agent chatbot we built in [the previous guide](https://tiptap.dev/docs/content-ai/capabilities/ai-toolkit/agents/ai-agent-chatbot.md), we need to replace the `executeTool` method with the `streamTool` method.

First, while the tool call is streaming, you can call `streamTool` repeatedly, every time a new streaming part is received. This will update the document incrementally.

```tsx
const aiToolkit = getAiToolkit(editor)

const result = aiToolkit.streamTool({
  toolCallId: 'call_123',
  toolName,
  // Content is still streaming, so we pass a partial JSON object.
  input,
  // This parameter indicates that the tool streaming has not finished yet
  hasFinished: false,
})
```

Then, when the tool call is complete, call the `streamTool` method again with `hasFinished: true` to indicate that the tool call has finished streaming. This will update the document with the final content.

```tsx
const result = aiToolkit.streamTool({
  toolCallId: 'call_123',
  toolName,
  // Streaming is complete, so we can pass the full JSON object
  input,
  // This parameter indicates that the tool streaming has finished
  hasFinished: true,
})
```

To implement this process in the AI agent chatbot we built in [the previous guide](https://tiptap.dev/docs/content-ai/capabilities/ai-toolkit/agents/ai-agent-chatbot.md), follow these steps:

### 1. Handle streaming updates

Add a `useEffect` hook to handle streaming updates while the tool call is in progress. Inside this hook, we call `streamTool` repeatedly, every time a new streaming part is received.

```tsx
// While the tool streaming is in progress, we need to update the document
// as the tool input changes
useEffect(() => {
  if (!editor) return

  // Find the last message
  const lastMessage = messages[messages.length - 1]
  if (!lastMessage) return

  // Find the last tool that the AI has just called
  const toolCallParts = lastMessage.parts.filter((p) => p.type.startsWith('tool-')) ?? []
  const lastToolCall = toolCallParts[toolCallParts.length - 1]
  if (!lastToolCall) return

  // Get the tool call data
  interface ToolStreamingPart {
    input: unknown
    state: string
    toolCallId: string
    type: string
  }
  const part = lastToolCall as ToolStreamingPart
  if (!(part.state === 'input-streaming')) return
  const toolName = part.type.replace('tool-', '')

  // Apply the tool call to the document, while it is streaming
  const toolkit = getAiToolkit(editor)
  toolkit.streamTool({
    toolCallId: part.toolCallId,
    toolName,
    input: part.input,
    // This parameter indicates that the tool streaming has not finished yet
    hasFinished: false,
  })
}, [addtoolOutput, editor, messages])
```

### 2. Handle streaming completion

In our demo, we use the `useChat` hook from the [AI SDK by Vercel](https://ai-sdk.vercel.app/) to implement the AI agent chatbot. This hook contains an `onToolCall` event handler that runs when the tool call finishes streaming.

Inside this handler, we call `streamTool` with `hasFinished: true` to indicate that the tool call has finished streaming.

```tsx
const { messages, sendMessage, addtoolOutput } = useChat({
  transport: new DefaultChatTransport({ api: '/api/chat' }),
  sendAutomaticallyWhen: lastAssistantMessageIsCompleteWithToolCalls,
  async onToolCall({ toolCall }) {
    if (!editor) return

    const { toolName, input, toolCallId } = toolCall

    // Use the AI Toolkit to stream the tool
    const toolkit = getAiToolkit(editor)
    const result = toolkit.streamTool({
      toolCallId,
      toolName,
      input,
      // This parameter indicates that the tool streaming is complete
      hasFinished: true,
    })

    addtoolOutput({ tool: toolName, toolCallId, output: result.output })
  },
})
```

> **Pass the same attributes in all streamTool calls:**
>
> You should pass the same attribute values in all `streamTool` calls. For example, in the first call to the `streamTool` method, if you pass `{mode: 'preview'}` to the `reviewOptions` parameter, you should pass the same value (`{mode: 'preview'}`) in all subsequent `streamTool` calls.

## Complete implementation

Here's the complete updated component with tool streaming:

```tsx
'use client'

import { DefaultChatTransport, lastAssistantMessageIsCompleteWithToolCalls } from 'ai'
import { useChat } from '@ai-sdk/react'
import { EditorContent, useEditor } from '@tiptap/react'
import StarterKit from '@tiptap/starter-kit'
import { useEffect, useState } from 'react'
import { AiToolkit, getAiToolkit } from '@tiptap-pro/ai-toolkit'

export default function Page() {
  const editor = useEditor({
    immediatelyRender: false,
    extensions: [StarterKit, AiToolkit],
    content: `<h1>AI Agent Demo</h1><p>Ask the AI to improve this.</p>`,
  })

  const { messages, sendMessage, addtoolOutput } = useChat({
    transport: new DefaultChatTransport({ api: '/api/chat' }),
    sendAutomaticallyWhen: lastAssistantMessageIsCompleteWithToolCalls,
    async onToolCall({ toolCall }) {
      if (!editor) return

      const { toolName, input, toolCallId } = toolCall

      // When the tool streaming is complete, we need to apply the tool call to the document
      // Use the AI Toolkit to execute the tool
      const toolkit = getAiToolkit(editor)
      const result = toolkit.streamTool({
        toolCallId,
        toolName,
        input,
        // This parameter indicates that the tool streaming is complete
        hasFinished: true,
      })

      addtoolOutput({ tool: toolName, toolCallId, output: result.output })
    },
  })

  const [input, setInput] = useState(
    'Insert, at the end of the document, a long story with 10 paragraphs about Tiptap',
  )

  // While the tool streaming is in progress, we need to update the document
  // as the tool input changes
  useEffect(() => {
    if (!editor) return

    // Find the last message
    const lastMessage = messages[messages.length - 1]
    if (!lastMessage) return

    // Find the last tool that the AI has just called
    const toolCallParts = lastMessage.parts.filter((p) => p.type.startsWith('tool-')) ?? []
    const lastToolCall = toolCallParts[toolCallParts.length - 1]
    if (!lastToolCall) return

    // Get the tool call data
    interface ToolStreamingPart {
      input: unknown
      state: string
      toolCallId: string
      type: string
    }
    const part = lastToolCall as ToolStreamingPart
    if (!(part.state === 'input-streaming')) return
    const toolName = part.type.replace('tool-', '')

    // Apply the tool call to the document, while it is streaming
    const toolkit = getAiToolkit(editor)
    toolkit.streamTool({
      toolCallId: part.toolCallId,
      toolName,
      input: part.input,
      // This parameter indicates that the tool streaming has not finished yet
      hasFinished: false,
    })
  }, [addtoolOutput, editor, messages])

  if (!editor) return null

  return (
    <div>
      <EditorContent editor={editor} />
      {messages?.map((message) => (
        <div key={message.id} style={{ whiteSpace: 'pre-wrap' }}>
          <strong>{message.role}</strong>
          <br />
          {message.parts
            .filter((p) => p.type === 'text')
            .map((p) => p.text)
            .join('\n')}
        </div>
      ))}
      <form
        onSubmit={(e) => {
          e.preventDefault()
          sendMessage({ text: input })
          setInput('')
        }}
      >
        <input value={input} onChange={(e) => setInput(e.target.value)} />
      </form>
    </div>
  )
}
```

## End result

With tool streaming, users can see changes happening in real-time as the AI generates them. Try it out:

> **Interactive demo:** [tool streaming](https://ai-toolkit-demos.vercel.app/tool-streaming)

See the [source code on GitHub](https://github.com/ueberdosis/ai-toolkit-demos).

## Show the AI Caret

You can add the `AiCaret` extension to display a cursor that indicates where the AI is inserting content during streaming. This gives users real-time visual feedback.

```tsx
import { AiCaret, AiToolkit, getAiToolkit } from '@tiptap-pro/ai-toolkit'

const editor = useEditor({
  extensions: [StarterKit, AiToolkit, AiCaret],
})
```

See the [AI Caret guide](https://tiptap.dev/docs/content-ai/capabilities/ai-toolkit/advanced-guides/ai-caret.md) for configuration options and CSS styles.

## Show a review UI

Display changes in a review UI, so users can accept or reject them.

There are two approaches:

- Use the [**Tracked Changes**](https://tiptap.dev/docs/tracked-changes/getting-started/overview.md) extension to render the review UI. Changes persist as part of the document and are visible to other users.
- **AI Toolkit suggestions**: a decoration-based UI that's ephemeral and only visible to the current user of the document.

Configure the review UI by setting the `reviewOptions` parameter. This parameter should have the same value every time the `streamTool` method is called.

```ts
toolkit.streamTool({
  // ... other options

  // Show the review UI with Tracked Changes
  reviewOptions: { mode: 'trackedChanges' },

  // Show the review UI with AI Toolkit suggestions
  reviewOptions: { mode: 'review' },
})
```

See the [review changes](https://tiptap.dev/docs/content-ai/capabilities/ai-toolkit/agents/review-changes.md) guide to learn more about the review UI.

See the [AI Toolkit demos](https://github.com/ueberdosis/ai-toolkit-demos) for examples on how to combine AI Toolkit streaming with the review UI.

## Next steps

- [Review changes guide](https://tiptap.dev/docs/content-ai/capabilities/ai-toolkit/agents/review-changes.md): Let users preview and approve changes before they're applied.
- [AI Caret](https://tiptap.dev/docs/content-ai/capabilities/ai-toolkit/advanced-guides/ai-caret.md): Show a cursor where the AI is inserting content.