Create reader
POST/reader
Create reader
Takes in a ReaderConfig payload and loads up new python reader
Request​
- application/json
Body
required
access_token stringnullable
assistant_model_name stringnullable
backend stringnullable
consumer_group stringrequired
cuda_visible_devices stringnullable
device stringrequired
disable_continuous_generation booleannullable
disable_cuda_graph int32nullable
disable_kernel_injection int32nullable
disable_static int32nullable
log_level stringnullable
max_batch_size int64nullable
max_sequence_length int32nullable
model_name stringrequired
nvlink_unavailable int32nullable
quant_type stringnullable
redis_host stringnullable
tensor_parallel int32nullable
Responses​
- 201
- 422
Takes in a JSON payload and loads up new model and/or backend
- application/json
- Schema
- Example (from schema)
Schema
_reader_config
object
required
access_token stringnullable
assistant_model_name stringnullable
backend stringnullable
consumer_group stringrequired
cuda_visible_devices stringnullable
device stringrequired
disable_continuous_generation booleannullable
disable_cuda_graph int32nullable
disable_kernel_injection int32nullable
disable_static int32nullable
log_level stringnullable
max_batch_size int64nullable
max_sequence_length int32nullable
model_name stringrequired
nvlink_unavailable int32nullable
quant_type stringnullable
redis_host stringnullable
tensor_parallel int32nullable
_reader_id stringrequired
{
"_reader_config": {
"access_token": "string",
"assistant_model_name": "string",
"backend": "string",
"consumer_group": "string",
"cuda_visible_devices": "string",
"device": "string",
"disable_continuous_generation": true,
"disable_cuda_graph": 0,
"disable_kernel_injection": 0,
"disable_static": 0,
"log_level": "string",
"max_batch_size": 0,
"max_sequence_length": 0,
"model_name": "string",
"nvlink_unavailable": 0,
"quant_type": "string",
"redis_host": "string",
"tensor_parallel": 0
},
"_reader_id": "string"
}
Malformed request body
Loading...