> ## Documentation Index
> Fetch the complete documentation index at: https://docs.openp.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Vision (image input)

> Pass in images so the model can read, describe and extract information

The major multimodal models (GPT-4o, Claude 4.x, Gemini 2.5) all support **image input**.
You can have the model describe an image, recognize objects, do OCR, generate alt text, or answer questions about an image.

## Supported models

| Model                                   |   OpenAI protocol  | Claude protocol | Gemini protocol |
| --------------------------------------- | :----------------: | :-------------: | :-------------: |
| `gpt-5.5`, `gpt-5.4-mini`               |          ✅         |        —        |        —        |
| `o3` series                             |          ✅         |        —        |        —        |
| `claude-haiku-4-5`, `sonnet`, `opus`    | ✅ (auto-converted) |        ✅        |        —        |
| `gemini-3.1-pro-preview` / `flash-lite` | ✅ (auto-converted) |        —        |        ✅        |
| `qwen-vl-*`                             |          ✅         |        —        |        —        |

## OpenAI protocol

### Via URL

```json theme={null}
{
  "model": "gpt-5.5",
  "messages": [{
    "role": "user",
    "content": [
      {"type": "text", "text": "What animal is in the image?"},
      {"type": "image_url", "image_url": {
        "url": "https://example.com/cat.jpg",
        "detail": "high"
      }}
    ]
  }]
}
```

`detail` values: `low` (fast, cheap) / `high` (high resolution, expensive) / `auto` (default).

### Via base64

```python theme={null}
import base64
with open("photo.jpg", "rb") as f:
    b64 = base64.b64encode(f.read()).decode()

resp = client.chat.completions.create(
    model="gpt-5.5",
    messages=[{"role": "user", "content": [
        {"type": "text", "text": "Describe this image"},
        {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{b64}"}}
    ]}]
)
```

Supported formats: `image/png` / `jpeg` / `gif` / `webp`.

## Claude protocol

```json theme={null}
{
  "model": "claude-opus-4-8",
  "max_tokens": 1024,
  "messages": [{
    "role": "user",
    "content": [
      {"type": "text", "text": "OCR this image"},
      {"type": "image", "source": {
        "type": "url",
        "url": "https://example.com/doc.png"
      }}
    ]
  }]
}
```

Or base64:

```json theme={null}
{"type": "image", "source": {
  "type": "base64",
  "media_type": "image/png",
  "data": "<base64>"
}}
```

## Gemini protocol

```json theme={null}
{
  "contents": [{
    "role": "user",
    "parts": [
      {"text": "What's happening in the image?"},
      {"inline_data": {"mime_type": "image/png", "data": "<base64>"}}
    ]
  }]
}
```

Gemini also supports `file_data` (an already-uploaded file) plus PDF / video parts:

```json theme={null}
{"file_data": {"mime_type": "application/pdf", "file_uri": "https://..."}}
```

## Multiple images

Put multiple image objects in the same message:

```json theme={null}
{
  "role": "user",
  "content": [
    {"type": "text", "text": "Compare the differences between the two images"},
    {"type": "image_url", "image_url": {"url": "https://.../a.jpg"}},
    {"type": "image_url", "image_url": {"url": "https://.../b.jpg"}}
  ]
}
```

* GPT-4o: up to \~50 images per call.
* Claude: up to \~20 images per call.
* Gemini: up to \~3000 images or 1 long video per call.

## Billing

Image input is converted to tokens by a **pixels → tokens** strategy and counted in `prompt_tokens`; the exact rules depend on the upstream:

* **OpenAI**: `detail: low` ≈ 85 tokens/image; `high` is estimated as 512×512 tiles × 170 tokens.
* **Claude**: `(width × height) / 750` tokens; 1024×1024 ≈ 1400 tokens.
* **Gemini**: a fixed 258 tokens per image.

The console log shows the image portion of the actual `prompt_tokens`.

## Best practices

* **Keep images clear**: blur or low resolution significantly lowers recognition accuracy.
* **Resize when necessary**: scaling large images to a 1024-2048 long edge saves tokens.
* **Ask specific questions**: don't just say "describe the image" — asking specific questions (objects / positions / numbers / text / colors) works better.
* **Use specialized models for OCR**: for document OCR, prefer `claude-opus-4-8` or `gemini-3.1-pro-preview`; for Chinese documents, consider `qwen-vl-max`.