CLI Reference

https://static.scarf.sh/a.png?x-pxid=0beb35eb-7742-4dfb-b183-2228e8caf04c

bentoml

BentoML CLI tool

bentoml [OPTIONS] COMMAND [ARGS]...

Options

--version

Show the version and exit.

azure-functions

Commands for Azure Functions BentoService deployment

bentoml azure-functions [OPTIONS] COMMAND [ARGS]...

delete

Delete Azure Functions deployment

bentoml azure-functions delete [OPTIONS] NAME

Options

-n, --namespace <namespace>

Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace

--force

force delete the deployment record in database and ignore errors when deleting cloud resources

-q, --quiet

Hide all warnings and info logs

--verbose, --debug

Show debug logs when running the command

Arguments

NAME

Required argument

deploy

Deploy BentoService to Azure Functions

bentoml azure-functions deploy [OPTIONS] NAME

Options

-n, --namespace <namespace>

Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace

-b, --bento, --bento-service-bundle <bento>

Required Target BentoService to be deployed, referenced by its name and version in the format of name:version. For example: “iris_classifier:v1.2.0”

--location <location>

Required The Azure location name for the deployment

--min-instances <min_instances>

The minimum number of workers for the deployment. The default value is 1

--max-burst <max_burst>

The maximum number of elastic workers for the deployment. The default value is 20

--premium-plan-sku <premium_plan_sku>

The Azure Functions premium SKU for the deployment. The default value is EP1

Options

EP1|EP2|EP3

-l, --labels <labels>

Key:value pairs that are attached to deployments and intended to be used to specify identifying attributes of the deployments that are meaningful to users. Multiple labels are separated with ,

--function-auth-level <function_auth_level>

The authorization level for the deployed Azure Functions. The default value is anonymous

Options

anonymous|function|admin

-o, --output <output>
Options

json|yaml

--wait, --no-wait

Wait for apply action to complete or encounter an error.If set to no-wait, CLI will exit without waiting until it has verified cloud resource allocation

-q, --quiet

Hide all warnings and info logs

--verbose, --debug

Show debug logs when running the command

Arguments

NAME

Required argument

get

Get Azure Functions deployment information

bentoml azure-functions get [OPTIONS] NAME

Options

-n, --namespace <namespace>

Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace

-o, --output <output>
Options

json|yaml|table

-q, --quiet

Hide all warnings and info logs

--verbose, --debug

Show debug logs when running the command

Arguments

NAME

Required argument

list

List Azure Functions deployments

bentoml azure-functions list [OPTIONS]

Options

-n, --namespace <namespace>

Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace

--limit <limit>

The maximum amount of Azure Functions deployments to be listed at once

-l, --labels <labels>

Label query to filter Azure Functions deployments, supports ‘=’, ‘!=’, ‘IN’, ‘NotIn’, ‘Exists’, and ‘DoesNotExist’. (e.g. key1=value1, key2!=value2, key3 In (value3, value3a), key4 DoesNotExist)

--order-by <order_by>
Options

created_at|name

--asc, --desc

Ascending or descending order for list deployments

-o, --output <output>
Options

json|yaml|table

-q, --quiet

Hide all warnings and info logs

--verbose, --debug

Show debug logs when running the command

update

Update existing Azure Functions deployment

bentoml azure-functions update [OPTIONS] NAME

Options

-b, --bento, --bento-service-bundle <bento>

Target BentoService to be deployed, referenced by its name and version in the format of name:version. For example: “iris_classifier:v1.2.0”

-n, --namespace <namespace>

Deployment namespace managed by BentoML, the default value is “dev” which can be changed in BentoML configuration file

--min-instances <min_instances>

The minimum number of workers for the deployment.

--max-burst <max_burst>

The maximum number of elastic workers for the deployment.

--premium-plan-sku <premium_plan_sku>

The Azure Functions premium SKU for the deployment.

Options

EP1|EP2|EP3

-o, --output <output>
Options

json|yaml

--wait, --no-wait

Wait for apply action to complete or encounter an error.If set to no-wait, CLI will exit without waiting until it has verified cloud resource allocation

-q, --quiet

Hide all warnings and info logs

--verbose, --debug

Show debug logs when running the command

Arguments

NAME

Required argument

config

Configure BentoML configurations and settings

bentoml config [OPTIONS] COMMAND [ARGS]...

reset

Reset all local BentoML configs to default

bentoml config reset [OPTIONS]

Options

-q, --quiet

Hide all warnings and info logs

--verbose, --debug

Show debug logs when running the command

set

Set config value in local BentoML configuration file

bentoml config set [OPTIONS] [UPDATES]...

Options

-q, --quiet

Hide all warnings and info logs

--verbose, --debug

Show debug logs when running the command

Arguments

UPDATES

Optional argument(s)

unset

Unset config in local BentoML configuration file

bentoml config unset [OPTIONS] [UPDATES]...

Options

-q, --quiet

Hide all warnings and info logs

--verbose, --debug

Show debug logs when running the command

Arguments

UPDATES

Optional argument(s)

view

View local BentoML configurations

bentoml config view [OPTIONS]

Options

-q, --quiet

Hide all warnings and info logs

--verbose, --debug

Show debug logs when running the command

view-effective

View effective BentoML configs, including default config values and local config overrides

bentoml config view-effective [OPTIONS]

Options

-q, --quiet

Hide all warnings and info logs

--verbose, --debug

Show debug logs when running the command

containerize

Containerizes given Bento into a ready-to-use Docker image.

bentoml containerize [OPTIONS] BENTO

Options

--push
-t, --tag <tag>

Optional image tag. If not specified, Bento will generate one from the name of the Bento.

--build-arg <build_arg>

pass through docker image build arguments

-u, --username <username>
-p, --password <password>
-q, --quiet

Hide all warnings and info logs

--verbose, --debug

Show debug logs when running the command

Arguments

BENTO

Required argument

delete

Delete saved BentoService.

BENTO is the target BentoService to be deleted, referenced by its name and version in format of name:version. For example: “iris_classifier:v1.2.0”

bentoml delete command also supports deleting multiple saved BentoService at once, by providing name version tag separated by “,”, for example:

bentoml delete iris_classifier:v1.2.0,my_svc:v1,my_svc2:v3

bentoml delete [OPTIONS] BENTOS

Options

-y, --yes, --assume-yes

Automatic yes to prompts

-q, --quiet

Hide all warnings and info logs

--verbose, --debug

Show debug logs when running the command

Arguments

BENTOS

Required argument

deployment

Commands for managing and operating BentoService deployments

bentoml deployment [OPTIONS] COMMAND [ARGS]...

apply

Apply BentoService deployment from yaml file

bentoml deployment apply [OPTIONS]

Options

-f, --file <deployment_yaml>

Required

-o, --output <output>
Options

json|yaml

--wait, --no-wait

Wait for apply action to complete or encounter an error.If set to no-wait, CLI will return immediately. The default value is wait

-q, --quiet

Hide all warnings and info logs

--verbose, --debug

Show debug logs when running the command

create

Create BentoService deployment from yaml file

bentoml deployment create [OPTIONS]

Options

-f, --file <deployment_yaml>

Required

-o, --output <output>
Options

json|yaml

--wait, --no-wait

Wait for apply action to complete or encounter an error.If set to no-wait, CLI will return immediately. The default value is wait

-q, --quiet

Hide all warnings and info logs

--verbose, --debug

Show debug logs when running the command

delete

Delete deployment

bentoml deployment delete [OPTIONS] NAME

Options

-n, --namespace <namespace>

Deployment namespace managed by BentoML, default value is “default” which can be changed in BentoML configuration file

--force

force delete the deployment record in database and ignore errors when deleting cloud resources

-q, --quiet

Hide all warnings and info logs

--verbose, --debug

Show debug logs when running the command

Arguments

NAME

Required argument

get

Get deployment information

bentoml deployment get [OPTIONS] NAME

Options

-n, --namespace <namespace>

Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration file

-o, --output <output>
Options

json|yaml

-q, --quiet

Hide all warnings and info logs

--verbose, --debug

Show debug logs when running the command

Arguments

NAME

Required argument

list

List deployments

bentoml deployment list [OPTIONS]

Options

-n, --namespace <namespace>

Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration file

-p, --platform <platform>

platform

Options

sagemaker|lambda

--limit <limit>

The maximum amount of deployments to be listed at once

--labels <labels>

Label query to filter deployments, supports ‘=’, ‘!=’, ‘IN’, ‘NotIn’, ‘Exists’, and ‘DoesNotExist’. (e.g. key1=value1, key2!=value2, key3 In (value3, value3a), key4 DoesNotExist)

--order-by <order_by>
Options

created_at|name

--asc, --desc

Ascending or descending order for list deployments

-o, --output <output>
Options

json|yaml|table|wide

-q, --quiet

Hide all warnings and info logs

--verbose, --debug

Show debug logs when running the command

get

Get BentoService information

bentoml get [OPTIONS] BENTO

Options

--limit <limit>

Limit how many resources will be retrieved

--ascending-order
--print-location
--labels <labels>

Label query to filter BentoServices, supports ‘=’, ‘!=’, ‘IN’, ‘NotIn’, ‘Exists’, and ‘DoesNotExist’. (e.g. key1=value1, key2!=value2, key3 In (value3, value3a), key4 DoesNotExist)

-o, --output <output>
Options

json|yaml|table|wide

-q, --quiet

Hide all warnings and info logs

--verbose, --debug

Show debug logs when running the command

Arguments

BENTO

Required argument

info

List all APIs defined in the BentoService loaded from saved bundle

bentoml info [OPTIONS] BENTO

Options

-q, --quiet

Hide all warnings and info logs

--verbose, --debug

Show debug logs when running the command

Arguments

BENTO

Required argument

install-completion

Install shell command completion

bentoml install-completion [OPTIONS] [[bash|zsh|fish|powershell]] [PATH]

Options

--append, --overwrite

Append the completion code to the file

-q, --quiet

Hide all warnings and info logs

--verbose, --debug

Show debug logs when running the command

Arguments

SHELL

Optional argument

PATH

Optional argument

lambda

Commands for AWS Lambda BentoService deployments

bentoml lambda [OPTIONS] COMMAND [ARGS]...

delete

Delete AWS Lambda deployment

bentoml lambda delete [OPTIONS] NAME

Options

-n, --namespace <namespace>

Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace

--force

force delete the deployment record in database and ignore errors when deleting cloud resources

-q, --quiet

Hide all warnings and info logs

--verbose, --debug

Show debug logs when running the command

Arguments

NAME

Required argument

deploy

Deploy BentoService to AWS Lambda

bentoml lambda deploy [OPTIONS] NAME

Options

-b, --bento, --bento-service-bundle <bento>

Required Target BentoService to be deployed, referenced by its name and version in format of name:version. For example: “iris_classifier:v1.2.0”

-n, --namespace <namespace>

Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace

-l, --labels <labels>

Key:value pairs that are attached to deployments and intended to be used to specify identifying attributes of the deployments that are meaningful to users. Multiple labels are separated with ,

--region <region>

AWS region name for deployment

--api-name <api_name>

User defined API function will be used for inference

--memory-size <memory_size>

Maximum Memory Capacity for AWS Lambda function, you can set the memory size in 64MB increments from 128MB to 3008MB. The default value is 1024 MB.

--timeout <timeout>

The amount of time that AWS Lambda allows a function to run before stopping it. The default is 3 seconds. The maximum allowed value is 900 seconds

-o, --output <output>
Options

json|yaml

--wait, --no-wait

Wait for apply action to complete or encounter an error.If set to no-wait, CLI will return immediately. The default value is wait

-q, --quiet

Hide all warnings and info logs

--verbose, --debug

Show debug logs when running the command

Arguments

NAME

Required argument

get

Get AWS Lambda deployment information

bentoml lambda get [OPTIONS] NAME

Options

-n, --namespace <namespace>

Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace

-o, --output <output>
Options

json|yaml|table

-q, --quiet

Hide all warnings and info logs

--verbose, --debug

Show debug logs when running the command

Arguments

NAME

Required argument

list

List AWS Lambda deployments

bentoml lambda list [OPTIONS]

Options

-n, --namespace <namespace>

Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace

--limit <limit>

The maximum amount of AWS Lambda deployments to be listed at once

-l, --labels <labels>

Label query to filter Lambda deployments, supports ‘=’, ‘!=’, ‘IN’, ‘NotIn’, ‘Exists’, and ‘DoesNotExist’. (e.g. key1=value1, key2!=value2, key3 In (value3, value3a), key4 DoesNotExist)

--order-by <order_by>
Options

created_at|name

--asc, --desc

Ascending or descending order for list deployments

-o, --output <output>
Options

json|yaml|table|wide

-q, --quiet

Hide all warnings and info logs

--verbose, --debug

Show debug logs when running the command

update

Update existing AWS Lambda deployment

bentoml lambda update [OPTIONS] NAME

Options

-b, --bento, --bento-service-bundle <bento>

Target BentoService to be deployed, referenced by its name and version in format of name:version. For example: “iris_classifier:v1.2.0”

-n, --namespace <namespace>

Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace

--memory-size <memory_size>

Maximum memory capacity for AWS Lambda function in MB, you can set the memory size in 64MB increments from 128 to 3008. The default value is 1024

--timeout <timeout>

The amount of time that AWS Lambda allows a function to run before stopping it. The default is 3 seconds. The maximum allowed value is 900 seconds

-o, --output <output>
Options

json|yaml

--wait, --no-wait

Wait for apply action to complete or encounter an error.If set to no-wait, CLI will return immediately. The default value is wait

-q, --quiet

Hide all warnings and info logs

--verbose, --debug

Show debug logs when running the command

Arguments

NAME

Required argument

list

List BentoServices information

bentoml list [OPTIONS]

Options

--limit <limit>

Limit how many BentoServices will be retrieved

--offset <offset>

How many BentoServices will be skipped

--labels <labels>

Label query to filter BentoServices, supports ‘=’, ‘!=’, ‘IN’, ‘NotIn’, ‘Exists’, and ‘DoesNotExist’. (e.g. key1=value1, key2!=value2, key3 In (value3, value3a), key4 DoesNotExist)

--order-by <order_by>
Options

created_at|name

--ascending-order
-o, --output <output>
Options

json|yaml|table|wide

-q, --quiet

Hide all warnings and info logs

--verbose, --debug

Show debug logs when running the command

open-api-spec

Display API specification JSON in Open-API format

bentoml open-api-spec [OPTIONS] BENTO

Options

-q, --quiet

Hide all warnings and info logs

--verbose, --debug

Show debug logs when running the command

Arguments

BENTO

Required argument

retrieve

Retrieves BentoService artifacts into a target directory

bentoml retrieve [OPTIONS] BENTO

Options

--target_dir <target_dir>

Directory to put artifacts into. Defaults to pwd.

-q, --quiet

Hide all warnings and info logs

--verbose, --debug

Show debug logs when running the command

Arguments

BENTO

Required argument

run

Run a API defined in saved BentoService bundle from command line

bentoml run [OPTIONS] BENTO API_NAME [RUN_ARGS]...

Options

-q, --quiet

Hide all warnings and info logs

--verbose, --debug

Show debug logs when running the command

Arguments

BENTO

Required argument

API_NAME

Required argument

RUN_ARGS

Optional argument(s)

sagemaker

Commands for AWS Sagemaker BentoService deployments

bentoml sagemaker [OPTIONS] COMMAND [ARGS]...

delete

Delete AWS Sagemaker deployment

bentoml sagemaker delete [OPTIONS] NAME

Options

-n, --namespace <namespace>

Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace

--force

force delete the deployment record in database and ignore errors when deleting cloud resources

-q, --quiet

Hide all warnings and info logs

--verbose, --debug

Show debug logs when running the command

Arguments

NAME

Required argument

deploy

Deploy BentoService to AWS Sagemaker

bentoml sagemaker deploy [OPTIONS] NAME

Options

-b, --bento, --bento-service-bundle <bento>

Required Target BentoService to be deployed, referenced by its name and version in format of name:version. For example: “iris_classifier:v1.2.0”

-n, --namespace <namespace>

Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace

-l, --labels <labels>

Key:value pairs that are attached to deployments and intended to be used to specify identifying attributes of the deployments that are meaningful to users. Multiple labels are separated with ,

--region <region>

AWS region name for deployment

--api-name <api_name>

Required User defined API function will be used for inference.

--instance-type <instance_type>

Type of instance will be used for inference. Default to “m1.m4.xlarge”

--instance-count <instance_count>

Number of instance will be used. Default value is 1

--num-of-gunicorn-workers-per-instance <num_of_gunicorn_workers_per_instance>

Number of gunicorn worker will be used per instance. Default value for gunicorn worker is based on the instance’ cpu core counts. The formula is num_of_cpu/2 + 1

--timeout <timeout>

The amount of time Sagemaker will wait before return response

-o, --output <output>
Options

json|yaml

--wait, --no-wait

Wait for apply action to complete or encounter an error.If set to no-wait, CLI will return immediately. The default value is wait

-q, --quiet

Hide all warnings and info logs

--verbose, --debug

Show debug logs when running the command

Arguments

NAME

Required argument

get

Get AWS Sagemaker deployment information

bentoml sagemaker get [OPTIONS] NAME

Options

-n, --namespace <namespace>

Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace

-o, --output <output>
Options

json|yaml|table

-q, --quiet

Hide all warnings and info logs

--verbose, --debug

Show debug logs when running the command

Arguments

NAME

Required argument

list

List AWS Sagemaker deployment information

bentoml sagemaker list [OPTIONS]

Options

-n, --namespace <namespace>

Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace

--limit <limit>

The maximum amount of AWS Sagemaker deployments to be listed at once

-l, --labels <labels>

Label query to filter Sagemaker deployments, supports ‘=’, ‘!=’, ‘IN’, ‘NotIn’, ‘Exists’, and ‘DoesNotExist’. (e.g. key1=value1, key2!=value2, key3 In (value3, value3a), key4 DoesNotExist)

--order-by <order_by>
Options

created_at|name

--asc, --desc

Ascending or descending order for list deployments

-o, --output <output>
Options

json|yaml|table|wide

-q, --quiet

Hide all warnings and info logs

--verbose, --debug

Show debug logs when running the command

update

Update existing AWS Sagemaker deployment

bentoml sagemaker update [OPTIONS] NAME

Options

-b, --bento, --bento-service-bundle <bento>

Target BentoService to be deployed, referenced by its name and version in format of name:version. For example: “iris_classifier:v1.2.0”

-n, --namespace <namespace>

Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration file

--instance-type <instance_type>

Type of instance will be used for inference. Default to “m1.m4.xlarge”

--instance-count <instance_count>

Number of instance will be used. Default value is 1

--num-of-gunicorn-workers-per-instance <num_of_gunicorn_workers_per_instance>

Number of gunicorn worker will be used per instance. Default value for gunicorn worker is based on the instance’ cpu core counts. The formula is num_of_cpu/2 + 1

--api-name <api_name>

User defined API function will be used for inference.

--timeout <timeout>

The amount of time Sagemaker will wait before return response

-o, --output <output>
Options

json|yaml

--wait, --no-wait

Wait for apply action to complete or encounter an error.If set to no-wait, CLI will return immediately. The default value is wait

-q, --quiet

Hide all warnings and info logs

--verbose, --debug

Show debug logs when running the command

Arguments

NAME

Required argument

serve

Start a dev API server serving specified BentoService

bentoml serve [OPTIONS] BENTO

Options

--port <port>

The port to listen on for the REST api server, default is 5000

--enable-microbatch, --disable-microbatch

Run API server with micro-batch enabled

--run-with-ngrok

Use ngrok to relay traffic on a public endpoint to this API server on localhost

-q, --quiet

Hide all warnings and info logs

--verbose, --debug

Show debug logs when running the command

Arguments

BENTO

Required argument

Environment variables

BENTOML_PORT

Provide a default for --port

BENTOML_ENABLE_MICROBATCH

Provide a default for --enable-microbatch

BENTOML_ENABLE_NGROK

Provide a default for --run-with-ngrok

serve-gunicorn

Start a production API server serving specified BentoService

bentoml serve-gunicorn [OPTIONS] BENTO

Options

-p, --port <port>

The port to listen on for the REST api server, default is 5000

-w, --workers <workers>

Number of workers will start for the gunicorn server

--timeout <timeout>
--enable-microbatch, --disable-microbatch

Run API server with micro batch enabled

--microbatch-workers <microbatch_workers>

Number of micro-batch request dispatcher workers

-q, --quiet

Hide all warnings and info logs

--verbose, --debug

Show debug logs when running the command

Arguments

BENTO

Required argument

Environment variables

BENTOML_PORT

Provide a default for -p

BENTOML_GUNICORN_WORKERS

Provide a default for -w

BENTOML_ENABLE_MICROBATCH

Provide a default for --enable-microbatch

BENTOML_MICROBATCH_WORKERS

Provide a default for --microbatch-workers

yatai-service-start

Start BentoML YataiService for model management and deployment

bentoml yatai-service-start [OPTIONS]

Options

--db-url <db_url>

Database URL following RFC-1738, and usually can include username, password, hostname, database name as well as optional keyword arguments for additional configuration

--repo-base-url <repo_base_url>

Base URL for storing BentoML saved bundle files, this can be a file system path(POSIX/Windows), or a S3 URL, usually starting with “s3://”

--grpc-port <grpc_port>

Port to run YataiService gRPC server

--ui-port <ui_port>

Port to run YataiService Web UI server

--ui, --no-ui

Run YataiService with or without Web UI, when running with –no-ui, it will only run the gRPC server

--s3-endpoint-url <s3_endpoint_url>

S3 Endpoint URL is used for deploying with storage services that are compatible with Amazon S3, such as MinIO

-q, --quiet

Hide all warnings and info logs

--verbose, --debug

Show debug logs when running the command

Environment variables

BENTOML_DB_URL

Provide a default for --db-url

BENTOML_REPO_BASE_URL

Provide a default for --repo-base-url

BENTOML_GRPC_PORT

Provide a default for --grpc-port

BENTOML_WEB_UI_PORT

Provide a default for --ui-port

BENTOML_ENABLE_WEB_UI

Provide a default for --ui

BENTOML_S3_ENDPOINT_URL

Provide a default for --s3-endpoint-url