Autobatcher

What is autobatcher?

autobatcher is a Python and TypeScript client that runs your normal OpenAI-style calls on Doubleword's cheaper tiers. You write normal async code using the familiar OpenAI interface, and autobatcher handles submission and polling behind the scenes.

By default, every text call (chat.completions and responses) runs on the async (flex) tier: it is sent as a background Responses request with service_tier="flex" and polled until it finishes. Pass completion_window="24h" (completionWindow: "24h" in JavaScript) to collect calls into 24h batch jobs instead, for the deepest discount. Embeddings always run as 24h batch jobs.

autobatcher exports two clients, AsyncOpenAI and BatchOpenAI. Both follow the rule above, so completion_window is what picks the tier. Both are drop-in replacements for the OpenAI client. They pass isinstance checks and provide full access to non-batched endpoints (models, files, etc.) out of the box.

Installation

pip install autobatcher

npm install autobatcher

Quick Start

from autobatcher import AsyncOpenAI

client = AsyncOpenAI(
    api_key="{{apiKey}}",
    base_url="https://api.doubleword.ai/v1"
)

response = await client.chat.completions.create(
    model="{{selectedModel.id}}",
    messages=[{"role": "user", "content": "Explain quantum computing"}],
)

print(response.choices[0].message.content)

import { AsyncOpenAI } from "autobatcher";

const client = new AsyncOpenAI({
  apiKey: "{{apiKey}}",
  baseURL: "https://api.doubleword.ai/v1"
});

const response = await client.chat.completions.create({
  model: "{{selectedModel.id}}",
  messages: [{ role: "user", content: "Explain quantum computing" }],
});

console.log(response.choices[0].message.content);

For bulk workloads with less time pressure, pass completion_window="24h" to submit 24h batch jobs:

from autobatcher import BatchOpenAI

client = BatchOpenAI(
    api_key="{{apiKey}}",
    base_url="https://api.doubleword.ai/v1",
    completion_window="24h",
)

response = await client.chat.completions.create(
    model="{{selectedModel.id}}",
    messages=[{"role": "user", "content": "Summarize this document..."}],
)

import { BatchOpenAI } from "autobatcher";

const client = new BatchOpenAI({
  apiKey: "{{apiKey}}",
  baseURL: "https://api.doubleword.ai/v1",
  completionWindow: "24h",
});

const response = await client.chat.completions.create({
  model: "{{selectedModel.id}}",
  messages: [{ role: "user", content: "Summarize this document..." }],
});

How It Works

In the default flex mode, each call is submitted straight away as a background request and polled until it completes, and your code gets a normal response back. Flex sends only the text of each message, so cache_control markers are dropped.

With completion_window="24h":

1. Requests are collected over a configurable time window or batch size
2. When the window closes or batch size is reached, requests are submitted as a batch
3. Results are polled and returned to waiting callers as they complete
4. Your code sees normal ChatCompletion responses

Per-request options such as extra_headers cannot be carried in a batch file, so 24h mode rejects them. Some agent frameworks set them on every call.

Configuration

Both AsyncOpenAI and BatchOpenAI accept the same configuration options:

Supported Endpoints

• client.chat.completions.create() — Chat completions
• client.embeddings.create() — Embeddings

All other OpenAI client methods (e.g., client.models.list(), client.files.create()) pass through unchanged to the underlying API.