CLI Reference¶

bentoml¶

BentoML CLI tool

bentoml [OPTIONS] COMMAND [ARGS]...

Options

--version¶: Show the version and exit.

azure-functions¶

Commands for Azure Functions BentoService deployment

bentoml azure-functions [OPTIONS] COMMAND [ARGS]...

delete¶

Delete Azure Functions deployment

bentoml azure-functions delete [OPTIONS] NAME

Options

-n, --namespace <namespace>¶: Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace

--force¶: force delete the deployment record in database and ignore errors when deleting cloud resources

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

NAME¶: Required argument

deploy¶

Deploy BentoService to Azure Functions

bentoml azure-functions deploy [OPTIONS] NAME

Options

-n, --namespace <namespace>¶: Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace

-b, --bento, --bento-service-bundle <bento>¶: Required Target BentoService to be deployed, referenced by its name and version in the format of name:version. For example: “iris_classifier:v1.2.0”

--location <location>¶: Required The Azure location name for the deployment

--min-instances <min_instances>¶: The minimum number of workers for the deployment. The default value is 1

--max-burst <max_burst>¶: The maximum number of elastic workers for the deployment. The default value is 20

--premium-plan-sku <premium_plan_sku>¶

The Azure Functions premium SKU for the deployment. The default value is EP1

Options: EP1 | EP2 | EP3

-l, --labels <labels>¶: Key:value pairs that are attached to deployments and intended to be used to specify identifying attributes of the deployments that are meaningful to users. Multiple labels are separated with ,

--function-auth-level <function_auth_level>¶

The authorization level for the deployed Azure Functions. The default value is anonymous

Options: anonymous | function | admin

-o, --output <output>¶

Options: json | yaml

--wait, --no-wait¶: Wait for apply action to complete or encounter an error.If set to no-wait, CLI will exit without waiting until it has verified cloud resource allocation

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

NAME¶: Required argument

get¶

Get Azure Functions deployment information

bentoml azure-functions get [OPTIONS] NAME

Options

-n, --namespace <namespace>¶: Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace

-o, --output <output>¶

Options: json | yaml | table

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

NAME¶: Required argument

list¶

List Azure Functions deployments

bentoml azure-functions list [OPTIONS]

Options

-n, --namespace <namespace>¶: Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace

--limit <limit>¶: The maximum amount of Azure Functions deployments to be listed at once

-l, --labels <labels>¶: Label query to filter Azure Functions deployments, supports ‘=’, ‘!=’, ‘IN’, ‘NotIn’, ‘Exists’, and ‘DoesNotExist’. (e.g. key1=value1, key2!=value2, key3 In (value3, value3a), key4 DoesNotExist)

--order-by <order_by>¶

Options: created_at | name

--asc, --desc¶: Ascending or descending order for list deployments

-o, --output <output>¶

Options: json | yaml | table

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

update¶

Update existing Azure Functions deployment

bentoml azure-functions update [OPTIONS] NAME

Options

-b, --bento, --bento-service-bundle <bento>¶: Target BentoService to be deployed, referenced by its name and version in the format of name:version. For example: “iris_classifier:v1.2.0”

-n, --namespace <namespace>¶: Deployment namespace managed by BentoML, the default value is “dev” which can be changed in BentoML configuration file

--min-instances <min_instances>¶: The minimum number of workers for the deployment.

--max-burst <max_burst>¶: The maximum number of elastic workers for the deployment.

--premium-plan-sku <premium_plan_sku>¶

The Azure Functions premium SKU for the deployment.

Options: EP1 | EP2 | EP3

-o, --output <output>¶

Options: json | yaml

--wait, --no-wait¶: Wait for apply action to complete or encounter an error.If set to no-wait, CLI will exit without waiting until it has verified cloud resource allocation

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

NAME¶: Required argument

config¶

Configure BentoML configurations and settings

bentoml config [OPTIONS] COMMAND [ARGS]...

reset¶

Reset all local BentoML configs to default

bentoml config reset [OPTIONS]

Options

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

set¶

Set config value in local BentoML configuration file

bentoml config set [OPTIONS] [UPDATES]...

Options

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

UPDATES¶: Optional argument(s)

unset¶

Unset config in local BentoML configuration file

bentoml config unset [OPTIONS] [UPDATES]...

Options

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

UPDATES¶: Optional argument(s)

view¶

View local BentoML configurations

bentoml config view [OPTIONS]

Options

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

view-effective¶

View effective BentoML configs, including default config values and local config overrides

bentoml config view-effective [OPTIONS]

Options

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

containerize¶

Containerizes given Bento into a ready-to-use Docker image.

bentoml containerize [OPTIONS] BENTO

Options

--push¶

-t, --tag <tag>¶: Optional image tag. If not specified, Bento will generate one from the name of the Bento.

--build-arg <build_arg>¶: pass through docker image build arguments

--yatai-url <yatai_url>¶: Specify the YataiService for running the containerization, default to the Local YataiService with local docker deamon. Example: “–yatai-url http://localhost:50050”

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

BENTO¶: Required argument

delete¶

Delete bento bundles in target YataiService. When the –yatai-url option is not specified, it will use local Yatai by default.

Specify target service bundles to remove:

Delete single bento bundle by “name:version”, e.g: bentoml delete IrisClassifier:v1
Bulk delete all bento bundles with a specific name, e.g.: bentoml delete IrisClassifier
Bulk delete multiple bento bundles by name and version, separated by “,”, e.g.: benotml delete Irisclassifier:v1,MyPredictService:v2
Bulk delete by tag, e.g.: bentoml delete –tag env=dev
Bulk delete all, e.g.: bentoml delete –all

bentoml delete [OPTIONS] [DELETE_TARGETS]

Options

--all¶: Set true to delete all bento bundles

--labels <labels>¶: Label query to filter bento bundles, supports ‘=’, ‘!=’, ‘IN’, ‘NotIn’, ‘Exists’, and ‘DoesNotExist’. (e.g. key1=value1, key2!=value2, key3 In (value3, value3a), key4 DoesNotExist)

--yatai-url <yatai_url>¶: Remote YataiService URL. Optional. Example: “–yatai-url http://localhost:50050”

-y, --yes, --assume-yes¶: Skip confirmation when deleting a specific bento bundle

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

DELETE_TARGETS¶: Optional argument

deployment¶

Commands for managing and operating BentoService deployments

bentoml deployment [OPTIONS] COMMAND [ARGS]...

apply¶

Apply BentoService deployment from yaml file

bentoml deployment apply [OPTIONS]

Options

-f, --file <deployment_yaml>¶: Required

-o, --output <output>¶

Options: json | yaml

--wait, --no-wait¶: Wait for apply action to complete or encounter an error.If set to no-wait, CLI will return immediately. The default value is wait

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

create¶

Create BentoService deployment from yaml file

bentoml deployment create [OPTIONS]

Options

-f, --file <deployment_yaml>¶: Required

-o, --output <output>¶

Options: json | yaml

--wait, --no-wait¶: Wait for apply action to complete or encounter an error.If set to no-wait, CLI will return immediately. The default value is wait

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

delete¶

Delete deployment

bentoml deployment delete [OPTIONS] NAME

Options

-n, --namespace <namespace>¶: Deployment namespace managed by BentoML, default value is “default” which can be changed in BentoML configuration file

--force¶: force delete the deployment record in database and ignore errors when deleting cloud resources

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

NAME¶: Required argument

get¶

Get deployment information

bentoml deployment get [OPTIONS] NAME

Options

-n, --namespace <namespace>¶: Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration file

-o, --output <output>¶

Options: json | yaml

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

NAME¶: Required argument

list¶

List deployments

bentoml deployment list [OPTIONS]

Options

-n, --namespace <namespace>¶: Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration file

-p, --platform <platform>¶

platform

Options: sagemaker | lambda

--limit <limit>¶: The maximum amount of deployments to be listed at once

--labels <labels>¶: Label query to filter deployments, supports ‘=’, ‘!=’, ‘IN’, ‘NotIn’, ‘Exists’, and ‘DoesNotExist’. (e.g. key1=value1, key2!=value2, key3 In (value3, value3a), key4 DoesNotExist)

--order-by <order_by>¶

Options: created_at | name

--asc, --desc¶: Ascending or descending order for list deployments

-o, --output <output>¶

Options: json | yaml | table | wide

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

ec2¶

commands for EC2

bentoml ec2 [OPTIONS] COMMAND [ARGS]...

delete¶

Delete AWS EC2 deployment

bentoml ec2 delete [OPTIONS] NAME

Options

-n, --namespace <namespace>¶: Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace

--force¶: force delete the deployment record in database and ignore errors when deleting cloud resources

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

NAME¶: Required argument

deploy¶

Deploy BentoService to EC2

bentoml ec2 deploy [OPTIONS] NAME

Options

-b, --bento <bento>¶: Required

-n, --namespace <namespace>¶

--region <region>¶: Region to deploy service in

--min-size <min_size>¶: The minimum limit helps ensure that you always have a certain number of instances running at all times.Default is 1

--desired-capacity <desired_capacity>¶: Desired number of instances size to run BentoService on.Should be between minimum and maximum capacities.Default is 1

--max-size <max_size>¶: The maximum limit lets Amazon EC2 Auto Scaling scale out the number of instances as needed to handle an increase in demand. Default is 1

--instance-type <instance_type>¶: Instance type of EC2 container.Default is t2 micro

--ami-id <ami_id>¶: AMI id.Default is Amazon Linux 2

-o, --output <output>¶

Options: json | yaml

--wait, --no-wait¶: Wait for apply action to complete or encounter an error.If set to no-wait, CLI will return immediately. The default value is wait

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

NAME¶: Required argument

get¶

Get EC2 deployment

bentoml ec2 get [OPTIONS] NAME

Options

-n, --namespace <namespace>¶: Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace

-o, --output <output>¶

Options: json | yaml | table

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

NAME¶: Required argument

list¶

List AWS Lambda deployments

bentoml ec2 list [OPTIONS]

Options

-n, --namespace <namespace>¶: Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace

--limit <limit>¶: The maximum amount of AWS Lambda deployments to be listed at once

--offset <offset>¶: The offset for list of AWS Lambda deployments

-l, --labels <labels>¶: Label query to filter Lambda deployments, supports ‘=’, ‘!=’, ‘IN’, ‘NotIn’, ‘Exists’, and ‘DoesNotExist’. (e.g. key1=value1, key2!=value2, key3 In (value3, value3a), key4 DoesNotExist)

--order-by <order_by>¶

Options: created_at | name

--asc, --desc¶: Ascending or descending order for list deployments

-o, --output <output>¶

Options: json | yaml | table | wide

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

update¶

Update existing AWS EC2 deployments

bentoml ec2 update [OPTIONS] NAME

Options

-b, --bento, --bento-service-bundle <bento>¶: Target BentoService to be deployed, referenced by its name and version in format of name:version. For example: “iris_classifier:v1.2.0”

-n, --namespace <namespace>¶: Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace

--min-size <min_size>¶: The minimum limit helps ensure that you always have a certain number of instances running at all times.Default is 1

--desired-capacity <desired_capacity>¶: Desired number of instances to run BentoService on.Should be between minimum and maximum capacities.Default is 1

--max-size <max_size>¶: The maximum limit lets Amazon EC2 Auto Scaling scale out the number of instances as needed to handle an increase in demand. Default is 1

--instance-type <instance_type>¶: Instance type of EC2 container.Default is t2 micro

--ami-id <ami_id>¶: AMI id.Default is Amazon Linux 2

-o, --output <output>¶

Options: json | yaml | table

--wait, --no-wait¶: Wait for apply action to complete or encounter an error.If set to no-wait, CLI will return immediately. The default value is wait

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

NAME¶: Required argument

get¶

Get BentoService information

bentoml get [OPTIONS] BENTO

Options

--limit <limit>¶: Limit how many resources will be retrieved

--ascending-order¶

--print-location¶

--labels <labels>¶: Label query to filter BentoServices, supports ‘=’, ‘!=’, ‘IN’, ‘NotIn’, ‘Exists’, and ‘DoesNotExist’. (e.g. key1=value1, key2!=value2, key3 In (value3, value3a), key4 DoesNotExist)

--yatai-url <yatai_url>¶: Remote YataiService URL. Optional. Example: “–yatai-url http://localhost:50050”

-o, --output <output>¶

Options: json | yaml | table | wide

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

BENTO¶: Required argument

info¶

List all APIs defined in the BentoService loaded from saved bundle

bentoml info [OPTIONS] BENTO

Options

--yatai-url <yatai_url>¶: Remote YataiService URL. Optional. Example: “–yatai-url http://localhost:50050”

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

BENTO¶: Required argument

install-completion¶

Install shell command completion

bentoml install-completion [OPTIONS] [[bash|zsh|fish|powershell]] [PATH]

Options

--append, --overwrite¶: Append the completion code to the file

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

SHELL¶: Optional argument

PATH¶: Optional argument

lambda¶

Commands for AWS Lambda BentoService deployments

bentoml lambda [OPTIONS] COMMAND [ARGS]...

delete¶

Delete AWS Lambda deployment

bentoml lambda delete [OPTIONS] NAME

Options

-n, --namespace <namespace>¶: Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace

--force¶: force delete the deployment record in database and ignore errors when deleting cloud resources

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

NAME¶: Required argument

deploy¶

Deploy BentoService to AWS Lambda

bentoml lambda deploy [OPTIONS] NAME

Options

-b, --bento, --bento-service-bundle <bento>¶: Required Target BentoService to be deployed, referenced by its name and version in format of name:version. For example: “iris_classifier:v1.2.0”

-n, --namespace <namespace>¶: Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace

-l, --labels <labels>¶: Key:value pairs that are attached to deployments and intended to be used to specify identifying attributes of the deployments that are meaningful to users. Multiple labels are separated with ,

--region <region>¶: AWS region name for deployment

--api-name <api_name>¶: User defined API function will be used for inference

--memory-size <memory_size>¶: Maximum Memory Capacity for AWS Lambda function, you can set the memory size in 64MB increments from 128MB to 3008MB. The default value is 1024 MB.

--timeout <timeout>¶: The amount of time that AWS Lambda allows a function to run before stopping it. The default is 3 seconds. The maximum allowed value is 900 seconds

-o, --output <output>¶

Options: json | yaml

--wait, --no-wait¶: Wait for apply action to complete or encounter an error.If set to no-wait, CLI will return immediately. The default value is wait

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

NAME¶: Required argument

get¶

Get AWS Lambda deployment information

bentoml lambda get [OPTIONS] NAME

Options

-n, --namespace <namespace>¶: Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace

-o, --output <output>¶

Options: json | yaml | table

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

NAME¶: Required argument

list¶

List AWS Lambda deployments

bentoml lambda list [OPTIONS]

Options

-n, --namespace <namespace>¶: Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace

--limit <limit>¶: The maximum amount of AWS Lambda deployments to be listed at once

-l, --labels <labels>¶: Label query to filter Lambda deployments, supports ‘=’, ‘!=’, ‘IN’, ‘NotIn’, ‘Exists’, and ‘DoesNotExist’. (e.g. key1=value1, key2!=value2, key3 In (value3, value3a), key4 DoesNotExist)

--order-by <order_by>¶

Options: created_at | name

--asc, --desc¶: Ascending or descending order for list deployments

-o, --output <output>¶

Options: json | yaml | table | wide

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

update¶

Update existing AWS Lambda deployment

bentoml lambda update [OPTIONS] NAME

Options

-b, --bento, --bento-service-bundle <bento>¶: Target BentoService to be deployed, referenced by its name and version in format of name:version. For example: “iris_classifier:v1.2.0”

-n, --namespace <namespace>¶: Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace

--memory-size <memory_size>¶: Maximum memory capacity for AWS Lambda function in MB, you can set the memory size in 64MB increments from 128 to 3008. The default value is 1024

--timeout <timeout>¶: The amount of time that AWS Lambda allows a function to run before stopping it. The default is 3 seconds. The maximum allowed value is 900 seconds

-o, --output <output>¶

Options: json | yaml

--wait, --no-wait¶: Wait for apply action to complete or encounter an error.If set to no-wait, CLI will return immediately. The default value is wait

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

NAME¶: Required argument

list¶

List BentoServices information

bentoml list [OPTIONS]

Options

--limit <limit>¶: Limit how many BentoServices will be retrieved

--offset <offset>¶: How many BentoServices will be skipped

--labels <labels>¶: Label query to filter BentoServices, supports ‘=’, ‘!=’, ‘IN’, ‘NotIn’, ‘Exists’, and ‘DoesNotExist’. (e.g. key1=value1, key2!=value2, key3 In (value3, value3a), key4 DoesNotExist)

--order-by <order_by>¶

Options: created_at | name

--ascending-order¶

--yatai-url <yatai_url>¶: Remote YataiService URL. Optional. Example: “–yatai-url http://localhost:50050”

-o, --output <output>¶

Options: json | yaml | table | wide

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

open-api-spec¶

Display API specification JSON in Open-API format

bentoml open-api-spec [OPTIONS] BENTO

Options

--yatai-url <yatai_url>¶: Remote YataiService URL. Optional. Example: “–yatai-url http://localhost:50050”

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

BENTO¶: Required argument

pull¶

Pull BentoService from remote yatai server

bentoml pull [OPTIONS] BENTO

Options

--yatai-url <yatai_url>¶: Required Remote YataiService URL. Example: “–yatai-url http://localhost:50050”

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

BENTO¶: Required argument

push¶

Push BentoService to remote yatai server

bentoml push [OPTIONS] BENTO

Options

--yatai-url <yatai_url>¶: Required Remote YataiService URL. Example: “–yatai-url http://localhost:50050”

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

BENTO¶: Required argument

retrieve¶

Retrieve

bentoml retrieve [OPTIONS] BENTO

Options

--yatai-url <yatai_url>¶: Remote YataiService URL. Example: “–yatai-url http://localhost:50050”

--target_dir <target_dir>¶: Target directory to save BentoService. Defaults to the current directory

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

BENTO¶: Required argument

run¶

Run a API defined in saved BentoService bundle from command line

bentoml run [OPTIONS] BENTO API_NAME [RUN_ARGS]...

Options

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

BENTO¶: Required argument

API_NAME¶: Required argument

RUN_ARGS¶: Optional argument(s)

sagemaker¶

Commands for AWS Sagemaker BentoService deployments

bentoml sagemaker [OPTIONS] COMMAND [ARGS]...

delete¶

Delete AWS Sagemaker deployment

bentoml sagemaker delete [OPTIONS] NAME

Options

-n, --namespace <namespace>¶: Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace

--force¶: force delete the deployment record in database and ignore errors when deleting cloud resources

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

NAME¶: Required argument

deploy¶

Deploy BentoService to AWS Sagemaker

bentoml sagemaker deploy [OPTIONS] NAME

Options

-b, --bento, --bento-service-bundle <bento>¶: Required Target BentoService to be deployed, referenced by its name and version in format of name:version. For example: “iris_classifier:v1.2.0”

-n, --namespace <namespace>¶: Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace

-l, --labels <labels>¶: Key:value pairs that are attached to deployments and intended to be used to specify identifying attributes of the deployments that are meaningful to users. Multiple labels are separated with ,

--region <region>¶: AWS region name for deployment

--api-name <api_name>¶: Required User defined API function will be used for inference.

--instance-type <instance_type>¶: Type of instance will be used for inference. Default to “m1.m4.xlarge”

--instance-count <instance_count>¶: Number of instance will be used. Default value is 1

--num-of-gunicorn-workers-per-instance <num_of_gunicorn_workers_per_instance>¶: Number of gunicorn worker will be used per instance. Default value for gunicorn worker is based on the instance’ cpu core counts. The formula is num_of_cpu/2 + 1

--timeout <timeout>¶: The amount of time Sagemaker will wait before return response

-o, --output <output>¶

Options: json | yaml

--wait, --no-wait¶: Wait for apply action to complete or encounter an error.If set to no-wait, CLI will return immediately. The default value is wait

--data-capture-s3-prefix <data_capture_s3_prefix>¶: To enable data capture (input and output), provide a destination s3 prefix (of the format s3://bucket-name/optional/prefix) for the captured data. To disable data capture, leave this blank.

--data-capture-sample-percent <data_capture_sample_percent>¶: When data capture is enabled, the sampling percentage. Default 100%. No effect without data-capture-s3-prefix.

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

NAME¶: Required argument

get¶

Get AWS Sagemaker deployment information

bentoml sagemaker get [OPTIONS] NAME

Options

-n, --namespace <namespace>¶: Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace

-o, --output <output>¶

Options: json | yaml | table

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

NAME¶: Required argument

list¶

List AWS Sagemaker deployment information

bentoml sagemaker list [OPTIONS]

Options

-n, --namespace <namespace>¶: Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace

--limit <limit>¶: The maximum amount of AWS Sagemaker deployments to be listed at once

-l, --labels <labels>¶: Label query to filter Sagemaker deployments, supports ‘=’, ‘!=’, ‘IN’, ‘NotIn’, ‘Exists’, and ‘DoesNotExist’. (e.g. key1=value1, key2!=value2, key3 In (value3, value3a), key4 DoesNotExist)

--order-by <order_by>¶

Options: created_at | name

--asc, --desc¶: Ascending or descending order for list deployments

-o, --output <output>¶

Options: json | yaml | table | wide

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

update¶

Update existing AWS Sagemaker deployment

bentoml sagemaker update [OPTIONS] NAME

Options

-b, --bento, --bento-service-bundle <bento>¶: Target BentoService to be deployed, referenced by its name and version in format of name:version. For example: “iris_classifier:v1.2.0”

-n, --namespace <namespace>¶: Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration file

--instance-type <instance_type>¶: Type of instance will be used for inference. Default to “m1.m4.xlarge”

--instance-count <instance_count>¶: Number of instance will be used. Default value is 1

--num-of-gunicorn-workers-per-instance <num_of_gunicorn_workers_per_instance>¶: Number of gunicorn worker will be used per instance. Default value for gunicorn worker is based on the instance’ cpu core counts. The formula is num_of_cpu/2 + 1

--api-name <api_name>¶: User defined API function will be used for inference.

--timeout <timeout>¶: The amount of time Sagemaker will wait before return response

-o, --output <output>¶

Options: json | yaml

--wait, --no-wait¶: Wait for apply action to complete or encounter an error.If set to no-wait, CLI will return immediately. The default value is wait

--data-capture-s3-prefix <data_capture_s3_prefix>¶: To enable data capture (input and output), provide a destination s3 prefix (of the format s3://bucket-name/optional/prefix) for the captured data. To disable data capture, leave this blank.

--data-capture-sample-percent <data_capture_sample_percent>¶: When data capture is enabled, the sampling percentage. Default 100%. No effect without data-capture-s3-prefix.

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

NAME¶: Required argument

serve¶

Start a dev API server serving specified BentoService

bentoml serve [OPTIONS] BENTO

Options

--port <port>¶: The port to listen on for the REST api server, default is 5000

--enable-microbatch, --disable-microbatch¶: Run API server with micro-batch enabled

--mb-max-batch-size <mb_max_batch_size>¶: Specify micro batching maximal batch size.

--mb-max-latency <mb_max_latency>¶: Specify micro batching maximal latency in milliseconds.

-c, --config <config>¶: Specify the configuration to be used for this command.

--run-with-ngrok¶: Use ngrok to relay traffic on a public endpoint to this API server on localhost

--yatai-url <yatai_url>¶: Remote YataiService URL. Optional. Example: “–yatai-url http://localhost:50050”

--enable-swagger, --disable-swagger¶: Run API server with Swagger UI enabled

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

BENTO¶: Required argument

Environment variables

BENTOML_PORT: Provide a default for --port

BENTOML_ENABLE_MICROBATCH: Provide a default for --enable-microbatch

BENTOML_MB_MAX_BATCH_SIZE: Provide a default for --mb-max-batch-size

BENTOML_MB_MAX_LATENCY: Provide a default for --mb-max-latency

BENTOML_CONFIG: Provide a default for --config

BENTOML_ENABLE_NGROK: Provide a default for --run-with-ngrok

BENTOML_ENABLE_SWAGGER: Provide a default for --enable-swagger

serve-gunicorn¶

Start a production API server serving specified BentoService

bentoml serve-gunicorn [OPTIONS] BENTO

Options

-p, --port <port>¶: The port to listen on for the REST api server, default is 5000

-w, --workers <workers>¶: Number of workers will start for the gunicorn server

--timeout <timeout>¶

--enable-microbatch, --disable-microbatch¶: Run API server with micro-batch enabled

--mb-max-batch-size <mb_max_batch_size>¶: Specify micro batching maximal batch size.

--mb-max-latency <mb_max_latency>¶: Specify micro batching maximal latency in milliseconds.

-c, --config <config>¶: Specify the configuration to be used for this command.

--microbatch-workers <microbatch_workers>¶: Number of micro-batch request dispatcher workers

--yatai-url <yatai_url>¶: Remote YataiService URL. Optional. Example: “–yatai-url http://localhost:50050”

--enable-swagger, --disable-swagger¶: Run API server with Swagger UI enabled

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Arguments

BENTO¶: Required argument

Environment variables

BENTOML_PORT: Provide a default for -p

BENTOML_GUNICORN_WORKERS: Provide a default for -w

BENTOML_ENABLE_MICROBATCH: Provide a default for --enable-microbatch

BENTOML_MB_MAX_BATCH_SIZE: Provide a default for --mb-max-batch-size

BENTOML_MB_MAX_LATENCY: Provide a default for --mb-max-latency

BENTOML_CONFIG: Provide a default for --config

BENTOML_MICROBATCH_WORKERS: Provide a default for --microbatch-workers

BENTOML_ENABLE_SWAGGER: Provide a default for --enable-swagger

yatai-service-start¶

Start BentoML YataiService for model management and deployment

bentoml yatai-service-start [OPTIONS]

Options

--db-url <db_url>¶: Database URL following RFC-1738, and usually can include username, password, hostname, database name as well as optional keyword arguments for additional configuration

--repo-base-url <repo_base_url>¶: Base URL for storing BentoML saved bundle files, this can be a file system path(POSIX/Windows), or a S3 URL, usually starting with “s3://”

--grpc-port <grpc_port>¶: Port to run YataiService gRPC server

--ui-port <ui_port>¶: Port to run YataiService Web UI server

--ui, --no-ui¶: Run YataiService with or without Web UI, when running with –no-ui, it will only run the gRPC server

--s3-endpoint-url <s3_endpoint_url>¶: S3 Endpoint URL is used for deploying with storage services that are compatible with Amazon S3, such as MinIO

--web-prefix-path <web_prefix_path>¶: Add a location prefix to the URL when running YataiServicebehind a reverse proxy server

-q, --quiet¶: Hide all warnings and info logs

--verbose, --debug¶: Show debug logs when running the command

Environment variables

BENTOML_DB_URL: Provide a default for --db-url

BENTOML_REPO_BASE_URL: Provide a default for --repo-base-url

BENTOML_GRPC_PORT: Provide a default for --grpc-port

BENTOML_WEB_UI_PORT: Provide a default for --ui-port

BENTOML_ENABLE_WEB_UI: Provide a default for --ui

BENTOML_S3_ENDPOINT_URL: Provide a default for --s3-endpoint-url

BENTOML_YATAI_WEB_PREFIX_PATH: Provide a default for --web-prefix-path