Skip to main content

​
Overview

A MessageType consists of three main components:
​
role
RoleEnumType
The role of the message sender.Constraints: Must be one of the defined roles.
​
content
Array<ContentType>
Array of content items.Constraints: Must contain at least one content item.

​
Role Types

The role field uses a union of predefined roles that determine the sender of the message.

​
RoleEnumType

​
"system"
string
Messages from the system.Usage: Used for system prompts and instructions.
​
"user"
string
Messages from the user.Usage: User input and queries.
​
"assistant"
string
Messages from the assistant.Usage: LLM responses and outputs.
​
"tool"
string
Messages from a tool.Usage: Tool execution results.
// Examples of different roles
const systemMessage: MessageType = {
  role: "system",
  content: [{ modality: "text", value: "You are a helpful assistant."}]
};

const userMessage: MessageType = {
  role: "user",
  content: [{ modality: "text", value: "Hello, how are you?"}]
};

const assistantMessage: MessageType = {
  role: "assistant",
  content: [{ modality: "text", value: "I'm doing well, thank you!"}]
};

​
Content Types

The content field contains an array of ContentType items, which is a union based on the modality field. Each content type supports different data formats and use cases.

​
ContentType Discriminated Union

The ContentType is discriminated by the modality field:
​
"text"
TextContentType
Plain text content.
​
"image"
ImageContentType
Image content with various formats.
​
"tool-call"
ToolCallContentType
Function/tool call requests.
​
"tool-response"
ToolResponseContentType
Tool execution responses.
​
"reasoning"
ReasoningContentType
LLM reasoning and thinking content.

​
Text Content

​
TextContentType

Represents plain text content in messages.
​
modality
"text"
Content type discriminator.Constraints: Must be "text".
​
value
string
The text content.Constraints: Any valid string.
const textContent: TextContentType = {
  modality: "text",
  value: "Hello, how can I help you today?"
};

​
Image Content

​
ImageContentType

Represents image content with support for both base64-encoded images and URL references.
​
modality
"image"
Content type discriminator.Constraints: Must be "image".
​
detail
ImageContentDetailsLiteralType
Processing detail level.Constraints: Must be "low", "medium", "high", or "auto"
​
value
ImageContentValueType
Image data.Constraints: Must be either Base64 or URL type

​
ImageContentDetailsLiteralType

Controls how the image should be processed by the LLM:
​
"low"
string
Low detail processing (faster, less accurate).
​
"medium"
string
Medium detail processing.
​
"high"
string
High detail processing (slower, more accurate).
​
"auto"
string
Automatic detail selection.

​
ImageContentValueType Discriminated Union

The value field in ImageContentType is itself a discriminated union based on the type field:

​
Base64ImageContentValueType

For inline base64-encoded images:
​
type
"base64"
Value type discriminator.Constraints: Must be "base64"
​
base64
string
Base64-encoded image data.Constraints: Valid base64 string.
​
mediaType
MediaTypeLiteral
Image format.Constraints: Must be "png", "jpeg", "webp", or "gif"

​
UrlImageContentValueType

For images referenced by URL:
​
type
"url"
Value type discriminator.Constraints: Must be "url"
​
url
string
URL pointing to the image.Constraints: Valid URL string
// Base64 image example
const base64ImageContent: ImageContentType = {
  modality: "image",
  detail: "high",
  value: {
    type: "base64",
    base64: "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8z8BQDwAEhQGAhKmMIQAAAABJRU5ErkJggg==",
    mediaType: "png"
  }
};

// URL image example
const urlImageContent: ImageContentType = {
  modality: "image",
  detail: "medium",
  value: {
    type: "url",
    url: "https://example.com/image.jpg"
  }
};

​
Tool Call Content

​
ToolCallContentType

Represents a function or tool call request from the LLM.
​
modality
"tool-call"
Content type discriminator.Constraints: Must be "tool-call".
​
index
number
Tool call index.Constraints: Non-negative integer.
​
id
string
Unique identifier for the tool call.Constraints: Non-empty string.
​
name
string
Name of the tool being called.Constraints: Non-empty string.
​
arguments
string
JSON string of tool arguments.Constraints: Valid string (typically JSON).
const toolCallContent: ToolCallContentType = {
  modality: "tool-call",
  index: 0,
  id: "call_123456789",
  name: "search_database",
  arguments: '{"query": "user information", "limit": 10}'
};

​
Tool Response Content

​
ToolResponseContentType

Represents the response from a tool execution.
​
modality
"tool-response"
Content type discriminator.Constraints: Must be "tool-response".
​
index
number
Tool response index.Constraints: Non-negative integer.
​
id
string
Tool call identifier.Constraints: Non-empty string (matches tool call ID).
​
name
string
Name of the tool that was called.Constraints: Non-empty string.
​
data
string
Response data from the tool.Constraints: Any valid string.
const toolResponseContent: ToolResponseContentType = {
  modality: "tool-response",
  index: 0,
  id: "call_123456789",
  name: "search_database",
  data: '{"results": [{"id": 1, "name": "John Doe"}, {"id": 2, "name": "Jane Smith"}]}'
};

​
Reasoning Content

​
ReasoningContentType

Represents LLM reasoning and thinking processes, supporting both full reasoning and redacted content.
​
modality
"reasoning"
Content type discriminator.Constraints: Must be "reasoning".
​
value
ReasoningContentValueUnionType
Reasoning data.Constraints: Must be thinking or redacted type.

​
ReasoningContentValueUnionType Discriminated Union

The value field is discriminated by the type field:

​
ReasoningContentValueType

For full reasoning content:
​
type
"thinking"
Value type discriminator.Constraints: Must be "thinking".
​
thinking
string
The reasoning/thinking text.Constraints: Any valid string.
​
signature
string
Signature for the reasoning.Constraints: Any valid string.

​
RedactedReasoningContentValueType

For redacted reasoning content:
​
type
"redacted"
Value type discriminator.Constraints: Must be "redacted".
​
data
string
Redacted data placeholder.Constraints: Any valid string
// Full reasoning example
const reasoningContent: ReasoningContentType = {
  modality: "reasoning",
  value: {
    type: "thinking",
    thinking: "I need to analyze the user's request carefully to provide the most helpful response...",
    signature: "reasoning-signature-123",
  }
};

// Redacted reasoning example
const redactedReasoningContent: ReasoningContentType = {
  modality: "reasoning",
  value: {
    type: "redacted",
    data: "This reasoning has been redacted for privacy",
  }
};

​
Complete Example

const complexMessage: MessageType = {
  role: "assistant",
  content: [
    {
      modality: "text",
      value: "I've analyzed the image and found the following information:"
    },
    {
      modality: "reasoning",
      value: {
        type: "thinking",
        thinking: "The image appears to contain a chart. I should extract the data points.",
        signature: "reasoning-signature-456",
      },
    },
    {
      modality: "tool-call",
      index: 0,
      id: "call_987654321",
      name: "analyze_chart",
      arguments: '{"chart_type": "bar", "data_points": ["Q1", "Q2", "Q3", "Q4"]}'
    }
  ]
};