Integrate your LLM

You can integrate the AI Agent extension with your own backend and LLM provider instead of using Tiptap Cloud. This gives you complete control over the AI model, tools, and conversation flow.

Custom LLM demo

We provide our Business and Enterprise customers with a detailed Custom LLM demo. It includes the client and server code, and instructions on how to run and deploy it. See plans and pricing.

Basic setup

To get started, follow these steps:

Create a backend service that communicates with your LLM provider
Configure the AI Agent extension to call your backend service

Create a backend service that calls your LLM

When you're building an AI Agent, you need to define a set of tools that it can call. The AIAgentToolkit class provides a list of text-editing tools that you can send to the LLM provider API. You can combine these tools with your own custom tools (like web search, agentic RAG, or orchestration) to build a custom AI Agent that can edit rich text as well as perform other actions.

In your server code, install the @tiptap-pro/extension-ai-agent and @tiptap-pro/extension-ai-agent-server libraries.

npm install @tiptap-pro/extension-ai-agent @tiptap-pro/extension-ai-agent-server

The AiAgentToolkit class provides methods for generating the system prompt and tool definitions in a format that can be sent to your LLM provider API or your AI Agent framework of choice.

import { openaiChatCompletionsAdapter } from '@tiptap-pro/extension-ai-agent'
import { AiAgentToolkit } from '@tiptap-pro/extension-ai-agent-server'

const toolkit = new AiAgentToolkit()

const response = await openai.chat.completions.create({
  model: 'gpt-4.1',
  messages: [
    {
      role: 'system',
      content: `You are Tiptap AI Agent, an AI agent that edits rich text documents.
// ... other system prompt instructions
${toolkit.getSystemPrompt()}`,
    },
    ...args.llmMessages,
  ],
  tools: [
    ...toolkit.getTools(openaiChatCompletionsAdapter),
    // ... combine with your custom tools
    customWebSearchTool,
    customWeatherReportTool,
  ],
})

Configure the AI Agent extension to call your backend

The AI Agent extension provides a resolver option that allows you to integrate with your custom backend service. This function is responsible for sending the messages to the LLM and returning the response.

import { AiAgentProvider } from '@tiptap-pro/extension-ai-agent'

const provider = new AiAgentProvider({
  resolver: async (options) => {
    // Your custom logic to send the chat messages to the LLM
    // and return the response
    const response = await yourCustomBackend.sendChatMessages(options.llmMessages)
    return response
  },
  // ... Other options
})

The resolver's response should contain these properties:

chatMessages: An array of regular chat messages generated by the AI, typically containing text responses. It does not include tool call messages.
toolCallChatMessages: An array of tool call messages generated by the AI, representing actions the AI wants to take. These messages contain instructions for tools to execute specific operations, such as editing the document or asking the user a question.

These properties allow you to separate the AI's textual responses from its requested actions, giving you more control over how to handle and display the AI's output in your application.

You can use the adapter's parseResponse function to convert the LLM response to the format expected by the AI Agent extension.

import { openaiChatCompletionsAdapter } from '@tiptap-pro/extension-ai-agent-server'

// In the server, call your LLM provider API
const response = await openai.chat.completions.create({
  // ...
})

// Parse the response to get the chat messages and tool calls in the format expected by the resolver
const result = openaiChatCompletionsAdapter.parseResponse(response)

Adapters

Adapters convert between the AI Agent's internal format and the format used by different LLM providers and AI Agent frameworks. The extension includes these built-in adapters:

Adapter	Provider
`openaiResponsesAdapter`	OpenAI API Responses API
`openaiChatCompletionsAdapter`	OpenAI API Chat Completions API
`anthropicMessagesAdapter`	Anthropic Claude Messages API

Need more adapters?

We are working on adding more adapters for different LLM providers and frameworks. If you need an adapter for a specific provider that is not yet available, contact us with your specific use case

If you want to use an LLM provider that is not yet supported by our backend library, you can create a custom adapter by implementing the AiAgentAdapter interface.

Configure the AI Agent's tools

You can customize the AiAgentToolkit instance by passing custom text-editing tools to its constructor.

import { AiAgentToolkit, toolsStarterKit } from '@tiptap-pro/extension-ai-agent-server'

const toolkit = new AiAgentToolkit({
  // The tools starter kit contains all the built-in tools
  ...toolsStarterKit(),
  // Add, for example, a tool to search the editor's content
  customSearchTool()
})

Built-in tools

The AI Agent extension includes these pre-built tools:

Tool Category	Tools
Reading	`read_first_chunk`, `read_next_chunk`, `read_previous_chunk`
Writing	`replace_chunk`, `replace_document`
Formatting	`run_command` (with various commands like `setBold`, `setItalic`, etc.)
Workflow	`plan`, `ask_user`, `finish_with_summary`

Each tool has two components:

Tool definition: Defines the tool's ID, description, and JSON schema. This data is sent to the LLM to generate the tool calls.
Tool handler: Implements the logic for executing the tool in the editor. If you define a tool in your backend, you must define a tool handler in the AI Agent provider, in the toolHandlers option.

Custom tools

You can create custom tools to extend the AI Agent's capabilities. First, define the tool (server-side):

import { AiAgentTool } from '@tiptap-pro/extension-ai-agent-server'

export const customSearchTool = (): AiAgentTool => ({
  id: 'search_document',
  toolDescription: 'Searches for text in the document',
  jsonSchema: {
    type: 'object',
    properties: {
      query: {
        type: 'string',
        description: 'The text to search for',
      },
    },
    required: ['query'],
  },
  // Optional additional rules for the system prompt
  systemPromptRules: [
    'Use this tool when you need to find specific content in the document',
    'The search is case-insensitive',
  ],
})

Then, implement the tool handler (client-side)

import { AiAgentToolCallHandler } from '@tiptap-pro/extension-ai-agent'
import { z } from 'zod'

// Schema for validating tool arguments
const SearchToolSchema = z.object({
  query: z.string(),
})

export const searchToolHandler = (): AiAgentToolCallHandler => ({
  id: 'search_document',
  modifiesEditor: false, // This tool doesn't modify the document
  handleToolCall: ({ editor, toolCall }) => {
    // Validate arguments
    const args = SearchToolSchema.parse(toolCall.arguments)

    // Implement search logic
    const html = editor.getHTML()
    const results = findTextInHtml(html, args.query)

    // Return results to the LLM
    return 'Found these results: ' + results.join(', ')
  },
})

Finally, add the tool to your toolkit and provider:

// Server-side
const toolkit = new AiAgentToolkit({
  tools: [
    // Built-in tools
    ...toolsStarterKit(),
    // Custom tools
    customSearchTool(),
  ],
})

// Client-side
const provider = new CustomAiAgentProvider({
  toolHandlers: [
    // Tool handlers for built-in tools
    ...toolHandlersStarterKit(),
    // Custom handlers
    searchToolHandler(),
  ],
})

Develop your backend in other programming languages than TypeScript

Although the @tiptap-pro/extension-ai-agent-server package is written in TypeScript, you can develop your backend in any other programming language.

First, extract the system prompt and tool definitions from the AiAgentToolkit instance by converting them to JSON. Then, use the JSON data to call the LLM provider in your code written in another programming language.

const toolkit = new AiAgentToolkit()

// Get the system prompt and tool definitions in JSON format
const systemPrompt: string = toolkit.getSystemPrompt()
const toolDefinitions: string = JSON.stringify(toolkit.getTools(openaiChatCompletionsAdapter))