CLI Reference¶

bentoml¶

BentoML CLI tool

bentoml [OPTIONS] COMMAND [ARGS]...

Options

--version¶: Show the version and exit.

azure-functions¶

Commands for Azure Functions BentoService deployment

bentoml azure-functions [OPTIONS] COMMAND [ARGS]...

delete¶

Delete Azure Functions deployment

bentoml azure-functions delete [OPTIONS] NAME

Options

-n, --namespace <namespace>¶: Deployment namespace managed by BentoML, default value is “dev” whichcan be changed in BentoML configuration yatai_service/default_namespace

--force¶: force delete the deployment record in database and ignore errors when deleting cloud resources

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

NAME¶: Required argument

deploy¶

Deploy BentoService to Azure Functions

bentoml azure-functions deploy [OPTIONS] NAME

Options

-n, --namespace <namespace>¶: Deployment namespace managed by BentoML, default value is “dev” whichcan be changed in BentoML configuration yatai_service/default_namespace

-b, --bento, --bento-service-bundle <bento>¶: Target BentoService to be deployed, referenced by its name and version in the format of name:version. For example: “iris_classifier:v1.2.0” [required]

--location <location>¶: The Azure location name for the deployment [required]

--min-instances <min_instances>¶: The minimum number of workers for the deployment. The default value is 1

--max-burst <max_burst>¶: The maximum number of elastic workers for the deployment. The default value is 20

--premium-plan-sku <premium_plan_sku>¶

The Azure Functions premium SKU for the deployment. The default value is EP1

Options: EP1|EP2|EP3

-l, --labels <labels>¶: Key:value pairs that are attached to deployments and intended to be used to specify identifying attributes of the deployments that are meaningful to users. Multiple labels are separated with ,

--function-auth-level <function_auth_level>¶

The authorization level for the deployed Azure Functions. The default value is anonymous

Options: anonymous|function|admin

-o, --output <output>¶

Options: json|yaml

--wait, --no-wait¶: Wait for apply action to complete or encounter an error.If set to no-wait, CLI will exit without waiting until it has verified cloud resource allocation

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

NAME¶: Required argument

get¶

Get Azure Functions deployment information

bentoml azure-functions get [OPTIONS] NAME

Options

-n, --namespace <namespace>¶: Deployment namespace managed by BentoML, default value is “dev” whichcan be changed in BentoML configuration yatai_service/default_namespace

-o, --output <output>¶

Options: json|yaml|table

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

NAME¶: Required argument

list¶

List Azure Functions deployments

bentoml azure-functions list [OPTIONS]

Options

-n, --namespace <namespace>¶: Deployment namespace managed by BentoML, default value is “dev” whichcan be changed in BentoML configuration yatai_service/default_namespace

--limit <limit>¶: The maximum amount of Azure Functions deployments to be listed at once

-l, --labels <labels>¶: List deployments matching the giving labels

--order-by <order_by>¶

Options: created_at|name

--asc, --desc¶: Ascending or descending order for list deployments

-o, --output <output>¶

Options: json|yaml|table

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

update¶

Update existing Azure Functions deployment

bentoml azure-functions update [OPTIONS] NAME

Options

-b, --bento, --bento-service-bundle <bento>¶: Target BentoService to be deployed, referenced by its name and version in the format of name:version. For example: “iris_classifier:v1.2.0”

-n, --namespace <namespace>¶: Deployment namespace managed by BentoML, the default value is “dev” which can be changed in BentoML configuration file

--min-instances <min_instances>¶: The minimum number of workers for the deployment.

--max-burst <max_burst>¶: The maximum number of elastic workers for the deployment.

--premium-plan-sku <premium_plan_sku>¶

The Azure Functions premium SKU for the deployment.

Options: EP1|EP2|EP3

-o, --output <output>¶

Options: json|yaml

--wait, --no-wait¶: Wait for apply action to complete or encounter an error.If set to no-wait, CLI will exit without waiting until it has verified cloud resource allocation

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

NAME¶: Required argument

config¶

Configure BentoML configurations and settings

bentoml config [OPTIONS] COMMAND [ARGS]...

reset¶

Reset all local BentoML configs to default

bentoml config reset [OPTIONS]

Options

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

set¶

Set config value in local BentoML configuration file

bentoml config set [OPTIONS] [UPDATES]...

Options

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

UPDATES¶: Optional argument(s)

unset¶

Unset config in local BentoML configuration file

bentoml config unset [OPTIONS] [UPDATES]...

Options

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

UPDATES¶: Optional argument(s)

view¶

View local BentoML configurations

bentoml config view [OPTIONS]

Options

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

view-effective¶

View effective BentoML configs, including default config values and local config overrides

bentoml config view-effective [OPTIONS]

Options

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

delete¶

Delete saved BentoService.

BENTO is the target BentoService to be deleted, referenced by its name and version in format of name:version. For example: “iris_classifier:v1.2.0”

bentoml delete command also supports deleting multiple saved BentoService at once, by providing name version tag separated by “,”, for example:

bentoml delete iris_classifier:v1.2.0,my_svc:v1,my_svc2:v3

bentoml delete [OPTIONS] BENTOS

Options

-y, --yes, --assume-yes¶: Automatic yes to prompts

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

BENTOS¶: Required argument

deployment¶

Commands for manageing and operating BentoService deployments

bentoml deployment [OPTIONS] COMMAND [ARGS]...

apply¶

Apply BentoService deployment from yaml file

bentoml deployment apply [OPTIONS]

Options

-f, --file <deployment_yaml>¶: [required]

-o, --output <output>¶

Options: json|yaml

--wait, --no-wait¶: Wait for apply action to complete or encounter an error.If set to no-wait, CLI will return immediately. The default value is wait

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

create¶

Create BentoService deployment from yaml file

bentoml deployment create [OPTIONS]

Options

-f, --file <deployment_yaml>¶: [required]

-o, --output <output>¶

Options: json|yaml

--wait, --no-wait¶: Wait for apply action to complete or encounter an error.If set to no-wait, CLI will return immediately. The default value is wait

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

delete¶

Delete deployment

bentoml deployment delete [OPTIONS] NAME

Options

-n, --namespace <namespace>¶: Deployment namespace managed by BentoML, default value is “default” whichcan be changed in BentoML configuration file

--force¶: force delete the deployment record in database and ignore errors when deleting cloud resources

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

NAME¶: Required argument

get¶

Get deployment information

bentoml deployment get [OPTIONS] NAME

Options

-n, --namespace <namespace>¶: Deployment namespace managed by BentoML, default value is “dev” whichcan be changed in BentoML configuration file

-o, --output <output>¶

Options: json|yaml

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

NAME¶: Required argument

list¶

List deployments

bentoml deployment list [OPTIONS]

Options

-n, --namespace <namespace>¶: Deployment namespace managed by BentoML, default value is “dev” whichcan be changed in BentoML configuration file

-p, --platform <platform>¶

platform

Options: sagemaker|lambda

--limit <limit>¶: The maximum amount of deployments to be listed at once

-l, --labels <labels>¶: List deployments matching the giving labels

--order-by <order_by>¶

Options: created_at|name

--asc, --desc¶: Ascending or descending order for list deployments

-o, --output <output>¶

Options: json|yaml|table

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

get¶

Get BentoService information

bentoml get [OPTIONS] BENTO

Options

--limit <limit>¶: Limit how many resources will be retrieved

--ascending-order¶

--print-location¶

-o, --output <output>¶

Options: json|yaml|table|wide

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

BENTO¶: Required argument

info¶

List all APIs defined in the BentoService loaded from saved bundle

bentoml info [OPTIONS] BENTO

Options

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

BENTO¶: Required argument

install-completion¶

Install shell command completion

bentoml install-completion [OPTIONS] [[bash|zsh|fish|powershell]] [PATH]

Options

--append, --overwrite¶: Append the completion code to the file

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

SHELL¶: Optional argument

PATH¶: Optional argument

lambda¶

Commands for AWS Lambda BentoService deployments

bentoml lambda [OPTIONS] COMMAND [ARGS]...

delete¶

Delete AWS Lambda deployment

bentoml lambda delete [OPTIONS] NAME

Options

-n, --namespace <namespace>¶: Deployment namespace managed by BentoML, default value is “dev” whichcan be changed in BentoML configuration yatai_service/default_namespace

--force¶: force delete the deployment record in database and ignore errors when deleting cloud resources

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

NAME¶: Required argument

deploy¶

Deploy BentoService to AWS Lambda

bentoml lambda deploy [OPTIONS] NAME

Options

-b, --bento, --bento-service-bundle <bento>¶: Target BentoService to be deployed, referenced by its name and version in format of name:version. For example: “iris_classifier:v1.2.0” [required]

-n, --namespace <namespace>¶: Deployment namespace managed by BentoML, default value is “dev” whichcan be changed in BentoML configuration yatai_service/default_namespace

-l, --labels <labels>¶: Key:value pairs that are attached to deployments and intended to be usedto specify identifying attributes of the deployments that are meaningful to users. Multiple labels are separated with ,

--region <region>¶: AWS region name for deployment

--api-name <api_name>¶: User defined API function will be used for inference

--memory-size <memory_size>¶: Maximum Memory Capacity for AWS Lambda function, you can set the memory size in 64MB increments from 128MB to 3008MB. The default value is 1024 MB.

--timeout <timeout>¶: The amount of time that AWS Lambda allows a function to run before stopping it. The default is 3 seconds. The maximum allowed value is 900 seconds

-o, --output <output>¶

Options: json|yaml

--wait, --no-wait¶: Wait for apply action to complete or encounter an error.If set to no-wait, CLI will return immediately. The default value is wait

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

NAME¶: Required argument

get¶

Get AWS Lambda deployment information

bentoml lambda get [OPTIONS] NAME

Options

-n, --namespace <namespace>¶: Deployment namespace managed by BentoML, default value is “dev” whichcan be changed in BentoML configuration yatai_service/default_namespace

-o, --output <output>¶

Options: json|yaml|table

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

NAME¶: Required argument

list¶

List AWS Lambda deployments

bentoml lambda list [OPTIONS]

Options

-n, --namespace <namespace>¶: Deployment namespace managed by BentoML, default value is “dev” whichcan be changed in BentoML configuration yatai_service/default_namespace

--limit <limit>¶: The maximum amount of AWS Lambda deployments to be listed at once

-l, --labels <labels>¶: List deployments matching the giving labels

--order-by <order_by>¶

Options: created_at|name

--asc, --desc¶: Ascending or descending order for list deployments

-o, --output <output>¶

Options: json|yaml|table

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

update¶

Update existing AWS Lambda deployment

bentoml lambda update [OPTIONS] NAME

Options

-b, --bento, --bento-service-bundle <bento>¶: Target BentoService to be deployed, referenced by its name and version in format of name:version. For example: “iris_classifier:v1.2.0”

-n, --namespace <namespace>¶: Deployment namespace managed by BentoML, default value is “dev” whichcan be changed in BentoML configuration yatai_service/default_namespace

--memory-size <memory_size>¶: Maximum memory capacity for AWS Lambda function in MB, you can set the memory size in 64MB increments from 128 to 3008. The default value is 1024

--timeout <timeout>¶: The amount of time that AWS Lambda allows a function to run before stopping it. The default is 3 seconds. The maximum allowed value is 900 seconds

-o, --output <output>¶

Options: json|yaml

--wait, --no-wait¶: Wait for apply action to complete or encounter an error.If set to no-wait, CLI will return immediately. The default value is wait

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

NAME¶: Required argument

list¶

List BentoServices information

bentoml list [OPTIONS]

Options

--limit <limit>¶: Limit how many BentoServices will be retrieved

--offset <offset>¶: How many BentoServices will be skipped

--order-by <order_by>¶

Options: created_at|name

--ascending-order¶

-o, --output <output>¶

Options: json|yaml|table|wide

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

open-api-spec¶

Display API specification JSON in Open-API format

bentoml open-api-spec [OPTIONS] BENTO

Options

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

BENTO¶: Required argument

retrieve¶

Retrieves BentoService artifacts into a target directory

bentoml retrieve [OPTIONS] BENTO

Options

--target_dir <target_dir>¶: Directory to put artifacts into. Defaults to pwd.

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

BENTO¶: Required argument

run¶

Run a API defined in saved BentoService bundle from command line

bentoml run [OPTIONS] BENTO API_NAME [RUN_ARGS]...

Options

--with-conda¶: Run API server in a BentoML managed Conda environment

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

BENTO¶: Required argument

API_NAME¶: Required argument

RUN_ARGS¶: Optional argument(s)

sagemaker¶

Commands for AWS Sagemaker BentoService deployments

bentoml sagemaker [OPTIONS] COMMAND [ARGS]...

delete¶

Delete AWS Sagemaker deployment

bentoml sagemaker delete [OPTIONS] NAME

Options

-n, --namespace <namespace>¶: Deployment namespace managed by BentoML, default value is “dev” whichcan be changed in BentoML configuration yatai_service/default_namespace

--force¶: force delete the deployment record in database and ignore errors when deleting cloud resources

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

NAME¶: Required argument

deploy¶

Deploy BentoService to AWS Sagemaker

bentoml sagemaker deploy [OPTIONS] NAME

Options

-b, --bento, --bento-service-bundle <bento>¶: Target BentoService to be deployed, referenced by its name and version in format of name:version. For example: “iris_classifier:v1.2.0” [required]

-n, --namespace <namespace>¶: Deployment namespace managed by BentoML, default value is “dev” whichcan be changed in BentoML configuration yatai_service/default_namespace

-l, --labels <labels>¶: Key:value pairs that are attached to deployments and intended to be usedto specify identifying attributes of the deployments that are meaningful to users. Multiple labels are separated with ,

--region <region>¶: AWS region name for deployment

--api-name <api_name>¶: User defined API function will be used for inference. [required]

--instance-type <instance_type>¶: Type of instance will be used for inference. Default to “m1.m4.xlarge”

--instance-count <instance_count>¶: Number of instance will be used. Default value is 1

--num-of-gunicorn-workers-per-instance <num_of_gunicorn_workers_per_instance>¶: Number of gunicorn worker will be used per instance. Default value for gunicorn worker is based on the instance’ cpu core counts. The formula is num_of_cpu/2 + 1

--timeout <timeout>¶: The amount of time Sagemaker will wait before return response

-o, --output <output>¶

Options: json|yaml

--wait, --no-wait¶: Wait for apply action to complete or encounter an error.If set to no-wait, CLI will return immediately. The default value is wait

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

NAME¶: Required argument

get¶

Get AWS Sagemaker deployment information

bentoml sagemaker get [OPTIONS] NAME

Options

-n, --namespace <namespace>¶: Deployment namespace managed by BentoML, default value is “dev” whichcan be changed in BentoML configuration yatai_service/default_namespace

-o, --output <output>¶

Options: json|yaml|table

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

NAME¶: Required argument

list¶

List AWS Sagemaker deployment information

bentoml sagemaker list [OPTIONS]

Options

-n, --namespace <namespace>¶: Deployment namespace managed by BentoML, default value is “dev” whichcan be changed in BentoML configuration yatai_service/default_namespace

--limit <limit>¶: The maximum amount of AWS Sagemaker deployments to be listed at once

-l, --labels <labels>¶: List deployments matching the giving labels

--order-by <order_by>¶

Options: created_at|name

--asc, --desc¶: Ascending or descending order for list deployments

-o, --output <output>¶

Options: json|yaml|table

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

update¶

Update existing AWS Sagemaker deployment

bentoml sagemaker update [OPTIONS] NAME

Options

-b, --bento, --bento-service-bundle <bento>¶: Target BentoService to be deployed, referenced by its name and version in format of name:version. For example: “iris_classifier:v1.2.0”

-n, --namespace <namespace>¶: Deployment namespace managed by BentoML, default value is “dev” whichcan be changed in BentoML configuration file

--instance-type <instance_type>¶: Type of instance will be used for inference. Default to “m1.m4.xlarge”

--instance-count <instance_count>¶: Number of instance will be used. Default value is 1

--num-of-gunicorn-workers-per-instance <num_of_gunicorn_workers_per_instance>¶: Number of gunicorn worker will be used per instance. Default value for gunicorn worker is based on the instance’ cpu core counts. The formula is num_of_cpu/2 + 1

--api-name <api_name>¶: User defined API function will be used for inference.

--timeout <timeout>¶: The amount of time Sagemaker will wait before return response

-o, --output <output>¶

Options: json|yaml

--wait, --no-wait¶: Wait for apply action to complete or encounter an error.If set to no-wait, CLI will return immediately. The default value is wait

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

NAME¶: Required argument

serve¶

Start REST API server hosting BentoService loaded from saved bundle

bentoml serve [OPTIONS] BENTO

Options

--port <port>¶: The port to listen on for the REST api server, default is $5000

--with-conda¶: Run API server in a BentoML managed Conda environment

--enable-microbatch¶: (Beta) Run API server with micro-batch enabled

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

BENTO¶: Required argument

Environment variables

BENTOML_PORT: Provide a default for --port

BENTOML_ENABLE_MICROBATCH: Provide a default for --enable-microbatch

serve-gunicorn¶

Start REST API server from saved BentoService bundle with gunicorn

bentoml serve-gunicorn [OPTIONS] BENTO

Options

-p, --port <port>¶: The port to listen on for the REST api server, default is $5000

-w, --workers <workers>¶: Number of workers will start for the gunicorn server

--timeout <timeout>¶

--with-conda¶: Run API server in a BentoML managed Conda environment

--enable-microbatch¶: (Beta) Run API server with micro batch enabled

--microbatch-workers <microbatch_workers>¶: (Beta) Number of micro-batch request dispatcher workers

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

BENTO¶: Required argument

Environment variables

BENTOML_PORT: Provide a default for -p

BENTOML_GUNICORN_WORKERS: Provide a default for -w

BENTOML_ENABLE_MICROBATCH: Provide a default for --enable-microbatch

BENTOML_MICROBATCH_WORKERS: Provide a default for --microbatch-workers

yatai-service-start¶

Start BentoML YataiService for model management and deployment

bentoml yatai-service-start [OPTIONS]

Options

--db-url <db_url>¶: Database URL following RFC-1738, and usually can include username, password, hostname, database name as well as optional keyword arguments for additional configuration

--repo-base-url <repo_base_url>¶: Base URL for storing BentoML saved bundle files, this can be a filesystempath(POSIX/Windows), or a S3 URL, usually starting with “s3://”

--grpc-port <grpc_port>¶: Port to run YataiService gRPC server

--ui-port <ui_port>¶: Port to run YataiService Web UI server

--ui, --no-ui¶: Run YataiService with or without Web UI, when running with –no-ui, it will only run the gRPC server

--s3-endpoint-url <s3_endpoint_url>¶: S3 Endpoint URL is used for deploying with storage services that are compatible with Amazon S3, such as MinIO

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Environment variables

BENTOML_DB_URL: Provide a default for --db-url

BENTOML_REPO_BASE_URL: Provide a default for --repo-base-url

BENTOML_GRPC_PORT: Provide a default for --grpc-port

BENTOML_WEB_UI_PORT: Provide a default for --ui-port

BENTOML_ENABLE_WEB_UI: Provide a default for --ui

BENTOML_S3_ENDPOINT_URL: Provide a default for --s3-endpoint-url