Use with Diagrams
Let your AI agent work with the charts and diagrams of Diagrams. The document tools of the Server AI Toolkit read, add and change them like any other content. There are no extra tools to enable.
Alpha
Diagrams is in alpha and available through a pilot program. The API may change in future releases.
Continuation from the AI agent chatbot guide
This guide continues the AI agent chatbot guide. Read it first.
How it works
- When your editor has the
chartandflowDiagramnodes of Diagrams,getEditorContextadds a description of them to the editor context. The Server AI Toolkit gives it to the AI in thesystemPromptoffetch-tools. - A chart keeps its Chart Data in its
dataattribute, and its type, title and other settings, such as horizontal bars or a target line, in its Chart Spec, thespecattribute. A diagram keeps its Mermaid code insourceand its caption intitle. A chart'ssourceattribute is different: it links the chart to a Data Source. The AI reads and writes these attributes in theformatof your request. - The AI changes charts and diagrams with the document tools, like
tiptapQuery.
Set up the editor
Add Diagrams to the editor of your AI agent chatbot, see Install Diagrams. Then get the editor context from that editor, like in the AI agent chatbot guide:
import { getEditorContext, ServerAiToolkit } from '@tiptap/ai-toolkit'
import { useEditor } from '@tiptap/react'
import StarterKit from '@tiptap/starter-kit'
import { Diagrams } from '@tiptap-pro/extension-diagrams'
import { echartsRenderer } from '@tiptap-pro/extension-diagrams/echarts'
const editor = useEditor({
immediatelyRender: false,
extensions: [StarterKit, ServerAiToolkit, Diagrams.configure({ renderers: [echartsRenderer()] })],
})
const editorContext = editor ? getEditorContext(editor) : nullThis editor draws charts. To draw flow diagrams too, add a Diagram Renderer, see A first diagram. Without a Mermaid Renderer, a diagram that the AI writes shows as unavailable.
You don't need to describe charts and diagrams with customNodes. getEditorContext adds their description.
Enable the tools
Enable tiptapQuery when you fetch the tool definitions:
tools: {
tiptapQuery: true,
}Its setAttrs action changes only the attributes that the AI names. So a chart keeps its other settings, and a diagram keeps its size, when the AI changes their data or code. tiptapRead and tiptapEdit also read and write charts and diagrams, but tiptapEdit replaces whole nodes. See Limitations.
Try prompts like these:
- "Add a pie chart of the June revenue and costs after the first paragraph"
- "Change the June revenue in the bar chart to 230"
- "In the order flow diagram, add a Pack step between In stock? and Ship"
Check the result
The AI's changes reach your editor through collaboration, like the changes of any other user. To check a chart that the AI wrote, read it from the editor:
const charts = editor.getJSON().content?.filter((node) => node.type === 'chart') ?? []
console.log(charts.map((chart) => chart.attrs?.spec?.type))Describe charts and diagrams yourself
The description names the chart types of Diagrams. Your Renderers may draw only some of them, or types of their own. To tell the AI which types your editor draws, add an item for chart with addJsonSchemaAwareness or the customNodes option of getEditorContext, and list the types in it. An item for chart or flowDiagram replaces the built-in description. See Custom nodes.
Limitations
- The server doesn't check the data that the AI writes. A chart without rows, or with data that Diagrams can't read, shows as empty. A chart of a type that no registered Renderer draws shows as unavailable.
- Replacing a chart or a diagram resets the attributes that the AI doesn't repeat, like the size of a diagram.
setAttrsoftiptapQuerykeeps them. - Text search and
containsdon't look inside attributes, so they don't find the numbers of a chart or the title of a diagram. To find charts and diagrams, the AI selects them by their type. - The AI can't link a chart to a Data Source or to a table. The Tools for a model of Diagrams can link a chart to a Data Source with
insertLinkedChart. A CSV or JSON file at a URL needsallowSourceUrl, which is off by default. - The AI writes Mermaid diagrams. The server tells it to keep the code of draw.io and BPMN diagrams as it is, not to change the data of a chart that reads it from a Data Source, and to change the table that a chart or a diagram follows instead of the chart or the diagram. It doesn't enforce any of these.
Next steps
- Learn more about Diagrams
- See the Tiptap Query API reference
- Describe your own nodes with Custom nodes