## What is autobatcher?

[autobatcher](https://github.com/doublewordai/autobatcher) is a Python and TypeScript client that runs your normal OpenAI-style calls on Doubleword's cheaper tiers. You write normal async code using the familiar OpenAI interface, and autobatcher handles submission and polling behind the scenes.

By default, every text call (`chat.completions` and `responses`) runs on the [async (flex) tier](/inference-api/async-inference): it is sent as a background Responses request with `service_tier="flex"` and polled until it finishes. Pass `completion_window="24h"` (`completionWindow: "24h"` in JavaScript) to collect calls into [24h batch jobs](/inference-api/batch-inference) instead, for the deepest discount. Embeddings always run as 24h batch jobs.

autobatcher exports two clients, `AsyncOpenAI` and `BatchOpenAI`. Both follow the rule above, so `completion_window` is what picks the tier. Both are drop-in replacements for the OpenAI client. They pass `isinstance` checks and provide full access to non-batched endpoints (models, files, etc.) out of the box.

## Installation

```bash tabs=install name=Python sync=lang
pip install autobatcher
```
```bash tabs=install name=JavaScript sync=lang
npm install autobatcher
```

## Quick Start

```python tabs=quickstart name=Python (async) sync=lang
from autobatcher import AsyncOpenAI

client = AsyncOpenAI(
    api_key="{{apiKey}}",
    base_url="https://api.doubleword.ai/v1"
)

response = await client.chat.completions.create(
    model="{{selectedModel.id}}",
    messages=[{"role": "user", "content": "Explain quantum computing"}],
)

print(response.choices[0].message.content)
```
```javascript tabs=quickstart name=JavaScript (async) sync=lang
import { AsyncOpenAI } from "autobatcher";

const client = new AsyncOpenAI({
  apiKey: "{{apiKey}}",
  baseURL: "https://api.doubleword.ai/v1"
});

const response = await client.chat.completions.create({
  model: "{{selectedModel.id}}",
  messages: [{ role: "user", content: "Explain quantum computing" }],
});

console.log(response.choices[0].message.content);
```

For bulk workloads with less time pressure, pass `completion_window="24h"` to submit 24h batch jobs:

```python tabs=batch name=Python (batch) sync=lang
from autobatcher import BatchOpenAI

client = BatchOpenAI(
    api_key="{{apiKey}}",
    base_url="https://api.doubleword.ai/v1",
    completion_window="24h",
)

response = await client.chat.completions.create(
    model="{{selectedModel.id}}",
    messages=[{"role": "user", "content": "Summarize this document..."}],
)
```
```javascript tabs=batch name=JavaScript (batch) sync=lang
import { BatchOpenAI } from "autobatcher";

const client = new BatchOpenAI({
  apiKey: "{{apiKey}}",
  baseURL: "https://api.doubleword.ai/v1",
  completionWindow: "24h",
});

const response = await client.chat.completions.create({
  model: "{{selectedModel.id}}",
  messages: [{ role: "user", content: "Summarize this document..." }],
});
```

## How It Works

In the default flex mode, each call is submitted straight away as a background request and polled until it completes, and your code gets a normal response back. Flex sends only the text of each message, so `cache_control` markers are dropped.

With `completion_window="24h"`:

1. Requests are collected over a configurable time window or batch size
2. When the window closes or batch size is reached, requests are submitted as a batch
3. Results are polled and returned to waiting callers as they complete
4. Your code sees normal ChatCompletion responses

Per-request options such as `extra_headers` cannot be carried in a batch file, so 24h mode rejects them. Some agent frameworks set them on every call.

## Configuration

Both `AsyncOpenAI` and `BatchOpenAI` accept the same configuration options:

| Option | Default | Description |
|---|---|---|
| `completion_window` / `completionWindow` | unset (flex) | Set to `"24h"` for batch jobs |
| `batch_size` / `batchSize` | 1000 | Maximum requests per batch before auto-flush (24h mode) |
| `batch_window_seconds` / `batchWindowSeconds` | 10 (Python) / 10 (JS) | Seconds to wait before flushing a partial batch (24h mode) |
| `poll_interval_seconds` / `pollIntervalSeconds` | 5 | Seconds between polls for flex responses or batch completion |

## Supported Endpoints

- `client.chat.completions.create()` — Chat completions
- `client.embeddings.create()` — Embeddings

All other OpenAI client methods (e.g., `client.models.list()`, `client.files.create()`) pass through unchanged to the underlying API.