Add images to chat completions - Writer AI Studio

Documentation Index

Fetch the complete documentation index at: /llms.txt

Use this file to discover all available pages before exploring further.

You can include images directly in your chat conversations with Palmyra X5 using mixed content support. This allows you to send both text and images in the same message, enabling rich visual conversations with Palmyra X5.

Image analysis in chat completions is only supported with the Palmyra X5 model. Other models don’t support the image_url content type.

You need an API key to access the Writer API. Get an API key by following the steps in the API quickstart. We recommend setting the API key as an environment variable in a .env file with the name WRITER_API_KEY.

Overview

Mixed content support allows you to send messages that contain both text and images in a chat with Palmyra X5. You can include images directly in your chat messages, making conversations more natural and contextual.

How it works

When sending a message to the chat completion endpoint, you can use the content field in two ways:

  1. Text-only message:
    "messages": [
        {
            "role": "user",
            "content": "Hello, how are you?"
        }
    ]
    ```

2. **Mixed content message**:

"messages": [
    {
        "role": "user",
        "content": [{"type": "text", "text": "What do you see in this image?"}, {"type": "image_url", "image_url": {"url": "<IMAGE_URL>"}}]
    }
]
```

Endpoint

URL: POST https://api.writer.com/v1/chat

Content fragment types

Text fragment

    {
      "type": "text",
      "text": "Your text content here"
    }
    ```

#### Image fragment

{
  "type": "image_url",
  "image_url": {
    "url": "<IMAGE_URL>"
  }
}
```

Examples

Single image analysis

This example shows how to send a message with a single image. The content field is an array of content fragments, where each fragment can be either text or image. The text fragment includes the message for the model to analyze. The image fragment includes the image URL.

cURL

    curl --location 'https://api.writer.com/v1/chat' \
    --header 'Content-Type: application/json' \
    --header "Authorization: Bearer $WRITER_API_KEY" \
    --data '{
      "model": "palmyra-x5",
      "messages": [
        {
          "role": "user",
          "content": [
            {
              "type": "text",
              "text": "What do you see in this image?"
            },
            {
              "type": "image_url",
              "image_url": {
                "url": "<IMAGE_URL>"
              }
            }
          ]
        }
      ]
    }'
    ```

#### Examples in Python

from writerai import Writer

Initialize the client. If you don't pass the apiKey parameter,

# the client looks for the `WRITER_API_KEY` environment variable.
client = Writer()

response = client.chat.chat( model="palmyra-x5", messages=[ { "role": "user", "content": [ { "type": "text", "text": "What do you see in this image?" }, { "type": "image_url", "image_url": { "url": "" } } ] } ] )

print(response.choices[0].message.content) ```

Examples in JavaScript

    import { Writer } from "writer-sdk";

// Initialize the client. If you don't pass the `apiKey` parameter,
    // the client looks for the `WRITER_API_KEY` environment variable.
    const client = new Writer();

const response = await client.chat.chat({
      model: "palmyra-x5",
      messages: [
        {
          role: "user",
          content: [
            {
              type: "text",
              text: "What do you see in this image?"
            },
            {
              type: "image_url",
              image_url: {
                url: "<IMAGE_URL>"
              }
            }
          ]
        }
      ]
    });

console.log(response.choices[0].message.content);
    ```

### Using local images with data URLs

You can also use [data URLs](https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/Data_URIs) to send local images without uploading them to a remote server. This is useful for testing or when working with local files:

#### cURL

# Convert local image to base64 data URL
base64_image=$(base64 -i "<IMAGE_PATH>")
data_url="data:image/jpeg;base64,$base64_image"

curl --location 'https://api.writer.com/v1/chat'
--header 'Content-Type: application/json'
--header "Authorization: Bearer $WRITER_API_KEY"
--data "{ "model": "palmyra-x5", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Analyze this local image:" }, { "type": "image_url", "image_url": { "url": "$data_url" } } ] } ] }" ```

Examples in Python

    import base64
    from pathlib import Path
    from writerai import Writer

# Initialize the client. If you don't pass the `apiKey` parameter,
    # the client looks for the `WRITER_API_KEY` environment variable.
    client = Writer()

# Read local image and convert to base64 data URL
    def image_to_data_url(image_path):
        with open(image_path, "rb") as image_file:
            encoded_string = base64.b64encode(image_file.read()).decode('utf-8')
            file_extension = Path(image_path).suffix.lstrip('.');
            mime_type = f"image/{file_extension}" if file_extension in ['jpg', 'jpeg', 'png', 'gif'] else "image/jpeg";
            return f"data:{mime_type};base64,{encoded_string}"

# Convert local image to data URL
    data_url = image_to_data_url("<IMAGE_PATH>");

response = client.chat.chat(
      model="palmyra-x5",
      messages=[
        {
          "role": "user",
          "content": [
            {
              "type": "text",
              "text": "Analyze this local image:" 
            },
            {
              "type": "image_url",
              "image_url": {
                "url": data_url
              }
            }
          ]
        }
      ]
    )

print(response.choices[0].message.content)
    ```

#### Examples in JavaScript

import { Writer } from "writer-sdk";
import fs from 'fs';

// Convert local image to base64 data URL function imageToDataUrl(imagePath) { const imageBuffer = fs.readFileSync(imagePath); const base64String = imageBuffer.toString('base64'); const fileExtension = imagePath.split('.').pop(); const mimeType = ['jpg', 'jpeg', 'png', 'gif'].includes(fileExtension) ? image/${fileExtension} : 'image/jpeg'; return data:${mimeType};base64,${base64String}; }

// Convert local image to data URL const dataUrl = imageToDataUrl("");

const response = await client.chat.chat({ model: "palmyra-x5", messages: [ { role: "user", content: [ { type: "text", text: "Analyze this local image:" }, { type: "image_url", image_url: { url: dataUrl } } ] } ] });

console.log(response.choices[0].message.content); ```

Multiple images with text

You can include multiple images in a single message:

cURL

    curl --location 'https://api.writer.com/v1/chat' \
    --header 'Content-Type: application/json' \
    --header "Authorization: Bearer $WRITER_API_KEY" \
    --data '{
      "model": "palmyra-x5",
      "messages": [
        {
          "role": "user",
          "content": [
            {
              "type": "text",
              "text": "Compare these two images and tell me the differences:"
            },
            {
              "type": "image_url",
              "image_url": {
                "url": "<IMAGE_URL_1>"
              }
            },
            {
              "type": "image_url",
              "image_url": {
                "url": "<IMAGE_URL_2>"
              }
            }
          ]
        }
      ]
    }'
    ```

#### Examples in Python

from writerai import Writer

Initialize the client. If you don't pass the apiKey parameter,

# the client looks for the `WRITER_API_KEY` environment variable.
client = Writer()

response = client.chat.chat( model="palmyra-x5", messages=[ { "role": "user", "content": [ { "type": "text", "text": "Compare these two images and tell me the differences:" }, { "type": "image_url", "image_url": { "url": "" } }, { "type": "image_url", "image_url": { "url": "" } } ] } ] )

print(response.choices[0].message.content) ```

Examples in JavaScript

    import { Writer } from "writer-sdk";

const response = await client.chat.chat({
      model: "palmyra-x5",
      messages: [
        {
          role: "user",
          content: [
            {
              type: "text",
              text: "Compare these two images and tell me the differences:"
            },
            {
              type: "image_url",
              image_url: {
                url: "<IMAGE_URL_1>"
              }
            },
            {
              type: "image_url",
              image_url: {
                url: "<IMAGE_URL_2>"
              }
            }
          ]
        }
      ]
    });

console.log(response.choices[0].message.content);
    ```

## Next steps

- Learn about [chat completions](https://dev.writer.com/home/chat-completion) for text-based conversations
- Check out [Palmyra X5](https://dev.writer.com/home/models#palmyra-x5) model capabilities