multimodal AI

AI models that process and generate multiple types of data — text, images, audio, and video — in a unified architecture.

Syntax

generative-ai

input: [text + image + audio] -> model -> output: [text | image | audio]

Example

generative-ai

// Multimodal models (2025):
// GPT-4o: text + vision + audio
// Claude 3: text + vision
// Gemini 1.5: text + vision + audio + video

// Example: image + text input to Claude:
const response = await client.messages.create({
  model: "claude-opus-4-5",
  messages: [{
    role: "user",
    content: [
      { type: "image", source: { type: "base64", data: imageData } },
      { type: "text", text: "What is in this image?" }
    ]
  }]
});