autobatcher

What is autobatcher?

autobatcher is a Python client that automatically converts your individual API calls into batched requests. You write normal async code using the familiar OpenAI interface, and autobatcher handles the batching complexity behind the scenes based on timing and volume to get you cost savings!

Installation

pip install autobatcher

How It Works

1. Requests are collected over a configurable time window or batch size (default: 1 second)
2. When the window closes or batch size is reached, requests are submitted as a batch
3. Results are polled and returned to waiting callers as they complete
4. Your code sees normal ChatCompletion responses