Skip to main content
Version: 0.12.x

Create reader

POST 

/reader

Create reader

Takes in a ReaderConfig payload and loads up new python reader

Request​

Body

required

    access_token stringnullable
    assistant_model_name stringnullable
    backend stringnullable
    consumer_group stringrequired
    cuda_graph_cache_capacity int32nullable
    cuda_visible_devices stringnullable
    device stringrequired
    disable_continuous_generation booleannullable
    disable_cuda_graph int32nullable
    disable_static int32nullable
    log_level stringnullable
    max_batch_size int64nullable
    max_sequence_length int32nullable
    model_name stringrequired
    nvlink_unavailable int32nullable
    quant_type stringnullable
    redis_host stringnullable
    tensor_parallel int32nullable

Responses​

Takes in a JSON payload and loads up new model and/or backend

Schema

    _reader_config

    object

    required

    access_token stringnullable
    assistant_model_name stringnullable
    backend stringnullable
    consumer_group stringrequired
    cuda_graph_cache_capacity int32nullable
    cuda_visible_devices stringnullable
    device stringrequired
    disable_continuous_generation booleannullable
    disable_cuda_graph int32nullable
    disable_static int32nullable
    log_level stringnullable
    max_batch_size int64nullable
    max_sequence_length int32nullable
    model_name stringrequired
    nvlink_unavailable int32nullable
    quant_type stringnullable
    redis_host stringnullable
    tensor_parallel int32nullable
    _reader_id stringrequired
Loading...