Create reader

POST 
/reader

Create reader

Takes in a ReaderConfig payload and loads up new python reader

Request

application/json

Body

required

access_tokenstringnullable
constrained_decoding_backendstringnullable
consumer_groupstringrequired
cuda_graph_cache_capacityint32nullable
cuda_visible_devicesstringnullable
devicestringrequired
disable_cuda_graphint32nullable
enable_document_processingbooleannullable
internal_gateway_ipstringnullable
lmfe_max_consecutive_whitespacesint32nullable
log_levelstringnullable
lorasstringnullable
max_batch_sizeint64nullable
max_sequence_lengthint32nullable
model_namestringrequired
nvlink_unavailableint32nullable
page_cache_sizestringnullable
prefill_chunk_sizeint64nullable
quant_typestringnullable
quantize_cache_bitsint32nullable
reader_log_as_jsonstringnullable
ssd_cache_sizeintegernullable
tensor_parallelint32nullable

Responses

201

Takes in a JSON payload and loads up new model and/or backend

application/json

Schema

Schema

_reader_config

object

required

access_tokenstringnullable

constrained_decoding_backendstringnullable

consumer_groupstringrequired

cuda_graph_cache_capacityint32nullable

cuda_visible_devicesstringnullable

devicestringrequired

disable_cuda_graphint32nullable

enable_document_processingbooleannullable

internal_gateway_ipstringnullable

lmfe_max_consecutive_whitespacesint32nullable

log_levelstringnullable

lorasstringnullable

max_batch_sizeint64nullable

max_sequence_lengthint32nullable

model_namestringrequired

nvlink_unavailableint32nullable

page_cache_sizestringnullable

prefill_chunk_sizeint64nullable

quant_typestringnullable

quantize_cache_bitsint32nullable

reader_log_as_jsonstringnullable

ssd_cache_sizeintegernullable

tensor_parallelint32nullable
_reader_idstringrequired

Example (from schema)

{
  "_reader_config": {
    "access_token": "string",
    "constrained_decoding_backend": "string",
    "consumer_group": "string",
    "cuda_graph_cache_capacity": 0,
    "cuda_visible_devices": "string",
    "device": "string",
    "disable_cuda_graph": 0,
    "enable_document_processing": true,
    "internal_gateway_ip": "string",
    "lmfe_max_consecutive_whitespaces": 0,
    "log_level": "string",
    "loras": "string",
    "max_batch_size": 0,
    "max_sequence_length": 0,
    "model_name": "string",
    "nvlink_unavailable": 0,
    "page_cache_size": "string",
    "prefill_chunk_size": 0,
    "quant_type": "string",
    "quantize_cache_bits": 0,
    "reader_log_as_json": "string",
    "ssd_cache_size": 0,
    "tensor_parallel": 0
  },
  "_reader_id": "string"
}

422

Malformed request body