> ## Documentation Index
> Fetch the complete documentation index at: https://concentrate.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Multi-Modal Inputs

> Send image content alongside text to vision-capable models on the Concentrate AI API, including supported formats, size and resolution limits, and examples.

## Overview

The Concentrate AI API supports multi-modal inputs, allowing you to send images alongside text to vision-capable models. Images can be provided as base64 data URIs or public URLs, and the API normalizes the format across all providers automatically.

## Supported Models

The following models support image inputs:

| Provider          | Models                                                                                  |
| ----------------- | --------------------------------------------------------------------------------------- |
| **OpenAI**        | GPT-5.2, GPT-5.1, GPT-5, GPT-5 Mini, GPT-5 Nano, GPT-4.1, GPT-4o                        |
| **Anthropic**     | Claude Opus 4.6, Claude Opus 4.5, Claude Sonnet 4.5, Claude Sonnet 4, Claude Sonnet 3.7 |
| **Google Vertex** | Gemini 3 Pro, Gemini 3 Flash, Gemini 2.5 Pro, Gemini 2.5 Flash                          |
| **Mistral**       | Pixtral Large, Mistral Medium, Mistral Small, Magistral Medium                          |
| **Cohere**        | Command A Vision                                                                        |
| **AWS Bedrock**   | Claude models (via Bedrock), OpenAI models (via Bedrock)                                |
| **Azure**         | GPT-5, GPT-4o, Claude models (via Azure)                                                |
| **Z.AI**          | GLM-4.6V, GLM-4.5V                                                                      |

<Info>
  Use the [Get Model](/docs/api-reference/endpoint/get-model) endpoint to check if a specific model supports image inputs by looking for the `image_processing` field in the provider configuration.
</Info>

## Sending Images

Images are sent as content blocks within the `input` array. Use the `input_image` content type alongside `input_text` blocks.

### Image Input Format

```json theme={null}
{
  "type": "input_image",
  "image_url": "data:image/png;base64,iVBOR...",
  "detail": "auto"
}
```

**Properties:**

* `type` (required): `"input_image"`
* `image_url` (required): Base64 data URI or public HTTPS URL
* `detail` (optional): `"low"`, `"high"`, or `"auto"` (default: `"auto"`)
  * `low`: Faster processing, lower token cost, suitable for simple images
  * `high`: Full resolution analysis, higher token cost, better for detailed images
  * `auto`: Let the model decide based on the image

### `detail` Parameter Support

The `detail` parameter controls image resolution for token estimation, but provider support varies depending on the underlying API format:

| Provider          | Accepts Detail | Notes                                                                          |
| ----------------- | :------------: | ------------------------------------------------------------------------------ |
| **OpenAI**        |       Yes      | Passed through via the Responses API format                                    |
| **xAI**           |       Yes      | Passed through via the Responses API format                                    |
| **Cohere**        |       Yes      | Explicitly mapped to the Cohere API                                            |
| **Azure**         |     Depends    | Forwarded for OpenAI-format models; not applicable for Anthropic-format models |
| **Anthropic**     |       N/A      | Anthropic's API does not have a detail parameter                               |
| **Google Vertex** |       N/A      | Gemini's API does not have a detail parameter                                  |
| **AWS Bedrock**   |       N/A      | Bedrock's native image format does not have a detail parameter                 |
| **Mistral**       |       Yes      | Mistral's Conversations API does not have a detail parameter                   |
| **Z.AI**          |       N/A      | Z.AI does not have a detail parameter                                          |

For providers marked **N/A**, their native APIs have no equivalent concept — the provider determines image processing resolution automatically.

## Examples

### Base64 Image

<CodeGroup>
  ```bash cURL theme={null}
  curl https://api.concentrate.ai/v1/responses \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer YOUR_API_KEY" \
    -d '{
      "model": "gpt-5.2",
      "input": [
        {
          "role": "user",
          "content": [
            {
              "type": "input_text",
              "text": "What do you see in this image?"
            },
            {
              "type": "input_image",
              "image_url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUg..."
            }
          ]
        }
      ]
    }'
  ```

  ```python Python theme={null}
  import requests
  import base64

  # Read and encode the image
  with open("photo.png", "rb") as f:
      image_b64 = base64.b64encode(f.read()).decode("utf-8")

  response = requests.post(
      "https://api.concentrate.ai/v1/responses",
      headers={
          "Authorization": "Bearer YOUR_API_KEY",
          "Content-Type": "application/json"
      },
      json={
          "model": "gpt-5.2",
          "input": [
              {
                  "role": "user",
                  "content": [
                      {
                          "type": "input_text",
                          "text": "What do you see in this image?"
                      },
                      {
                          "type": "input_image",
                          "image_url": f"data:image/png;base64,{image_b64}"
                      }
                  ]
              }
          ]
      }
  )

  data = response.json()
  print(data["output"][0]["content"][0]["text"])
  ```

  ```typescript TypeScript theme={null}
  async function analyzeImage(imagePath: string) {

    const imageB64 = fs.readFileSync(imagePath).toString("base64");

    const response = await fetch("https://api.concentrate.ai/v1/responses", {
      method: "POST",
      headers: {
        "Authorization": "Bearer YOUR_API_KEY",
        "Content-Type": "application/json"
      },
      body: JSON.stringify({
        model: "gpt-5.2",
        input: [
          {
            role: "user",
            content: [
              {
                type: "input_text",
                text: "What do you see in this image?"
              },
              {
                type: "input_image",
                image_url: `data:image/png;base64,${imageB64}`
              }
            ]
          }
        ]
      })
    });

    const data = await response.json();
    console.log(data.output[0].content[0].text);
  }
  ```
</CodeGroup>

### URL Image

You can also pass a publicly accessible image URL:

```json theme={null}
{
  "model": "claude-sonnet-4-5",
  "input": [
    {
      "role": "user",
      "content": [
        {
          "type": "input_text",
          "text": "Describe this image in detail."
        },
        {
          "type": "input_image",
          "image_url": "https://example.com/photo.jpg"
        }
      ]
    }
  ]
}
```

<Warning>
  Image URLs must be publicly accessible. Private URLs and URLs requiring authentication may be blocked.
</Warning>

### Multiple Images

Send multiple images in a single request:

```json theme={null}
{
  "model": "gemini-2.5-pro",
  "input": [
    {
      "role": "user",
      "content": [
        {
          "type": "input_text",
          "text": "Compare these two images and describe the differences."
        },
        {
          "type": "input_image",
          "image_url": "https://example.com/image1.jpg"
        },
        {
          "type": "input_image",
          "image_url": "https://example.com/image2.jpg"
        }
      ]
    }
  ]
}
```

### With Streaming

Multi-modal requests work with streaming:

```json theme={null}
{
  "model": "gpt-5.2",
  "stream": true,
  "input": [
    {
      "role": "user",
      "content": [
        {
          "type": "input_text",
          "text": "Describe this image."
        },
        {
          "type": "input_image",
          "image_url": "data:image/jpeg;base64,/9j/4AAQ...",
          "detail": "high"
        }
      ]
    }
  ]
}
```

## Supported Formats

| Format | MIME Type    | Data URI Prefix           |
| ------ | ------------ | ------------------------- |
| PNG    | `image/png`  | `data:image/png;base64,`  |
| JPEG   | `image/jpeg` | `data:image/jpeg;base64,` |
| GIF    | `image/gif`  | `data:image/gif;base64,`  |
| WebP   | `image/webp` | `data:image/webp;base64,` |

<Warning>
  Some providers only support a subset of these image types. You can check the specific model info for specific information on supported model types.
</Warning>

## Limits

Image limits vary by provider and model. Exceeding these limits will return a `400` error.

| Provider                       | Max Images Per Request | Max Total Size |
| ------------------------------ | ---------------------- | -------------- |
| **OpenAI** (GPT-5, GPT-4o)     | 500                    | 50 MB          |
| **Anthropic** (Claude)         | 100                    | 32 MB          |
| **Google Vertex** (Gemini 3)   | 900                    | 7 MB           |
| **Google Vertex** (Gemini 2.5) | 3,000                  | 7 MB           |
| **Cohere** (Command A Vision)  | 20                     | 20 MB          |
| **Mistral** (Pixtral Large)    | 8                      | 10 MB          |

### Resolution limits

Some models also enforce per-image resolution limits, exposed on the `image_processing` capability object returned by [Get Model](/docs/api-reference/endpoint/get-model) and [List Models](/docs/api-reference/endpoint/list-models):

* `max_pixels` — Maximum total pixels (width × height) allowed for a single image.
* `max_dimension` — Maximum allowed value for either the width or the height of a single image, in pixels.

If a field is omitted for a given provider, that limit is not enforced. Images that exceed either limit are rejected with a `400` error before being sent to the upstream provider — resize or downscale before uploading.

<Info>
  Image tokens are calculated based on per provider algorithms. Higher resolution images consume more tokens. For providers that support it, consider setting `detail` to "low" to reduce costs.
</Info>

## Error Handling

Common errors when using image inputs:

| Error                                 | Cause                                                           |
| ------------------------------------- | --------------------------------------------------------------- |
| `Model does not support image inputs` | The selected model does not have vision capabilities            |
| `Too many images`                     | Request exceeds the model's `max_images_per_request` limit      |
| `Image size exceeds limit`            | Total image data exceeds the model's `max_total` size limit     |
| `Image resolution exceeds limit`      | Image exceeds the model's `max_pixels` or `max_dimension` limit |
| `Invalid image format`                | Image is not PNG, JPEG, GIF, or WebP                            |
| `Invalid image URL`                   | URL is not a valid HTTP/HTTPS URL or data URI                   |

## Related Documentation

<CardGroup cols={2}>
  <Card title="Request Parameters" icon="sliders" href="/docs/api-reference/endpoint/request-parameters">
    Complete parameter reference
  </Card>

  <Card title="Streaming" icon="water" href="/docs/api-reference/endpoint/streaming">
    Use multi-modal with streaming
  </Card>

  <Card title="Create Response" icon="message" href="/docs/api-reference/endpoint/create-response">
    Main endpoint documentation
  </Card>

  <Card title="List Models" icon="list" href="/docs/api-reference/endpoint/list-models">
    Check model capabilities
  </Card>
</CardGroup>