CLI Reference¶
bentoml¶
BentoML CLI tool
bentoml [OPTIONS] COMMAND [ARGS]...
Options
-
--version
¶
Show the version and exit.
azure-functions¶
Commands for Azure Functions BentoService deployment
bentoml azure-functions [OPTIONS] COMMAND [ARGS]...
delete¶
Delete Azure Functions deployment
bentoml azure-functions delete [OPTIONS] NAME
Options
-
--do-not-track
¶
Specify the option to not track usage.
-
-n
,
--namespace
<namespace>
¶ Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace
-
--force
¶
force delete the deployment record in database and ignore errors when deleting cloud resources
-
-q
,
--quiet
¶
Hide all warnings and info logs
-
--verbose
,
--debug
¶
Show debug logs when running the command
Arguments
-
NAME
¶
Required argument
Environment variables
-
BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
deploy¶
Deploy BentoService to Azure Functions
bentoml azure-functions deploy [OPTIONS] NAME
Options
-
--do-not-track
¶
Specify the option to not track usage.
-
-n
,
--namespace
<namespace>
¶ Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace
-
-b
,
--bento
,
--bento-service-bundle
<bento>
¶ Required Target BentoService to be deployed, referenced by its name and version in the format of name:version. For example: “iris_classifier:v1.2.0”
-
--location
<location>
¶ Required The Azure location name for the deployment
-
--min-instances
<min_instances>
¶ The minimum number of workers for the deployment. The default value is 1
-
--max-burst
<max_burst>
¶ The maximum number of elastic workers for the deployment. The default value is 20
The Azure Functions premium SKU for the deployment. The default value is EP1
- Options
EP1 | EP2 | EP3
-
-l
,
--labels
<labels>
¶ Key:value pairs that are attached to deployments and intended to be used to specify identifying attributes of the deployments that are meaningful to users. Multiple labels are separated with ,
-
--function-auth-level
<function_auth_level>
¶ The authorization level for the deployed Azure Functions. The default value is anonymous
- Options
anonymous | function | admin
-
-o
,
--output
<output>
¶ - Options
json | yaml
-
--wait
,
--no-wait
¶
Wait for apply action to complete or encounter an error.If set to no-wait, CLI will exit without waiting until it has verified cloud resource allocation
-
-q
,
--quiet
¶
Hide all warnings and info logs
-
--verbose
,
--debug
¶
Show debug logs when running the command
Arguments
-
NAME
¶
Required argument
Environment variables
-
BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
get¶
Get Azure Functions deployment information
bentoml azure-functions get [OPTIONS] NAME
Options
-
--do-not-track
¶
Specify the option to not track usage.
-
-n
,
--namespace
<namespace>
¶ Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace
-
-o
,
--output
<output>
¶ - Options
json | yaml | table
-
-q
,
--quiet
¶
Hide all warnings and info logs
-
--verbose
,
--debug
¶
Show debug logs when running the command
Arguments
-
NAME
¶
Required argument
Environment variables
-
BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
list¶
List Azure Functions deployments
bentoml azure-functions list [OPTIONS]
Options
-
--do-not-track
¶
Specify the option to not track usage.
-
-n
,
--namespace
<namespace>
¶ Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace
-
--limit
<limit>
¶ The maximum amount of Azure Functions deployments to be listed at once
-
-l
,
--labels
<labels>
¶ Label query to filter Azure Functions deployments, supports ‘=’, ‘!=’, ‘IN’, ‘NotIn’, ‘Exists’, and ‘DoesNotExist’. (e.g. key1=value1, key2!=value2, key3 In (value3, value3a), key4 DoesNotExist)
-
--order-by
<order_by>
¶ - Options
created_at | name
-
--asc
,
--desc
¶
Ascending or descending order for list deployments
-
-o
,
--output
<output>
¶ - Options
json | yaml | table
-
-q
,
--quiet
¶
Hide all warnings and info logs
-
--verbose
,
--debug
¶
Show debug logs when running the command
Environment variables
-
BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
update¶
Update existing Azure Functions deployment
bentoml azure-functions update [OPTIONS] NAME
Options
-
--do-not-track
¶
Specify the option to not track usage.
-
-b
,
--bento
,
--bento-service-bundle
<bento>
¶ Target BentoService to be deployed, referenced by its name and version in the format of name:version. For example: “iris_classifier:v1.2.0”
-
-n
,
--namespace
<namespace>
¶ Deployment namespace managed by BentoML, the default value is “dev” which can be changed in BentoML configuration file
-
--min-instances
<min_instances>
¶ The minimum number of workers for the deployment.
-
--max-burst
<max_burst>
¶ The maximum number of elastic workers for the deployment.
The Azure Functions premium SKU for the deployment.
- Options
EP1 | EP2 | EP3
-
-o
,
--output
<output>
¶ - Options
json | yaml
-
--wait
,
--no-wait
¶
Wait for apply action to complete or encounter an error.If set to no-wait, CLI will exit without waiting until it has verified cloud resource allocation
-
-q
,
--quiet
¶
Hide all warnings and info logs
-
--verbose
,
--debug
¶
Show debug logs when running the command
Arguments
-
NAME
¶
Required argument
Environment variables
-
BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
containerize¶
Containerizes given Bento into a ready-to-use Docker image.
bentoml containerize [OPTIONS] BENTO
Options
-
--do-not-track
¶
Specify the option to not track usage.
-
--push
¶
-
-t
,
--tag
<tag>
¶ Optional image tag. If not specified, Bento will generate one from the name of the Bento.
-
--build-arg
<build_arg>
¶ pass through docker image build arguments
-
--yatai-url
<yatai_url>
¶ Specify the YataiService for running the containerization, default to the Local YataiService with local docker daemon. Example: “–yatai-url http://localhost:50050”
-
-q
,
--quiet
¶
Hide all warnings and info logs
-
--verbose
,
--debug
¶
Show debug logs when running the command
Arguments
-
BENTO
¶
Required argument
Environment variables
-
BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
delete¶
Delete bento bundles in target YataiService. When the –yatai-url option is not specified, it will use local Yatai by default.
Specify target service bundles to remove:
Delete single bento bundle by “name:version”, e.g: bentoml delete IrisClassifier:v1
Bulk delete all bento bundles with a specific name, e.g.: bentoml delete IrisClassifier
Bulk delete multiple bento bundles by name and version, separated by “,”, e.g.: benotml delete Irisclassifier:v1,MyPredictService:v2
Bulk delete by tag, e.g.: bentoml delete –tag env=dev
Bulk delete all, e.g.: bentoml delete –all
bentoml delete [OPTIONS] [DELETE_TARGETS]
Options
-
--do-not-track
¶
Specify the option to not track usage.
-
--all
¶
Set true to delete all bento bundles
-
--labels
<labels>
¶ Label query to filter bento bundles, supports ‘=’, ‘!=’, ‘IN’, ‘NotIn’, ‘Exists’, and ‘DoesNotExist’. (e.g. key1=value1, key2!=value2, key3 In (value3, value3a), key4 DoesNotExist)
-
--yatai-url
<yatai_url>
¶ Remote YataiService URL. Optional. Example: “–yatai-url http://localhost:50050”
-
-y
,
--yes
,
--assume-yes
¶
Skip confirmation when deleting a specific bento bundle
-
-q
,
--quiet
¶
Hide all warnings and info logs
-
--verbose
,
--debug
¶
Show debug logs when running the command
Arguments
-
DELETE_TARGETS
¶
Optional argument
Environment variables
-
BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
deployment¶
Commands for managing and operating BentoService deployments
bentoml deployment [OPTIONS] COMMAND [ARGS]...
apply¶
Apply BentoService deployment from yaml file
bentoml deployment apply [OPTIONS]
Options
-
--do-not-track
¶
Specify the option to not track usage.
-
-f
,
--file
<deployment_yaml>
¶ Required
-
-o
,
--output
<output>
¶ - Options
json | yaml
-
--wait
,
--no-wait
¶
Wait for apply action to complete or encounter an error.If set to no-wait, CLI will return immediately. The default value is wait
-
-q
,
--quiet
¶
Hide all warnings and info logs
-
--verbose
,
--debug
¶
Show debug logs when running the command
Environment variables
-
BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
create¶
Create BentoService deployment from yaml file
bentoml deployment create [OPTIONS]
Options
-
--do-not-track
¶
Specify the option to not track usage.
-
-f
,
--file
<deployment_yaml>
¶ Required
-
-o
,
--output
<output>
¶ - Options
json | yaml
-
--wait
,
--no-wait
¶
Wait for apply action to complete or encounter an error.If set to no-wait, CLI will return immediately. The default value is wait
-
-q
,
--quiet
¶
Hide all warnings and info logs
-
--verbose
,
--debug
¶
Show debug logs when running the command
Environment variables
-
BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
delete¶
Delete deployment
bentoml deployment delete [OPTIONS] NAME
Options
-
--do-not-track
¶
Specify the option to not track usage.
-
-n
,
--namespace
<namespace>
¶ Deployment namespace managed by BentoML, default value is “default” which can be changed in BentoML configuration file
-
--force
¶
force delete the deployment record in database and ignore errors when deleting cloud resources
-
-q
,
--quiet
¶
Hide all warnings and info logs
-
--verbose
,
--debug
¶
Show debug logs when running the command
Arguments
-
NAME
¶
Required argument
Environment variables
-
BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
get¶
Get deployment information
bentoml deployment get [OPTIONS] NAME
Options
-
--do-not-track
¶
Specify the option to not track usage.
-
-n
,
--namespace
<namespace>
¶ Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration file
-
-o
,
--output
<output>
¶ - Options
json | yaml
-
-q
,
--quiet
¶
Hide all warnings and info logs
-
--verbose
,
--debug
¶
Show debug logs when running the command
Arguments
-
NAME
¶
Required argument
Environment variables
-
BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
list¶
List deployments
bentoml deployment list [OPTIONS]
Options
-
--do-not-track
¶
Specify the option to not track usage.
-
-n
,
--namespace
<namespace>
¶ Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration file
-
-p
,
--platform
<platform>
¶ platform
- Options
sagemaker | lambda
-
--limit
<limit>
¶ The maximum amount of deployments to be listed at once
-
--labels
<labels>
¶ Label query to filter deployments, supports ‘=’, ‘!=’, ‘IN’, ‘NotIn’, ‘Exists’, and ‘DoesNotExist’. (e.g. key1=value1, key2!=value2, key3 In (value3, value3a), key4 DoesNotExist)
-
--order-by
<order_by>
¶ - Options
created_at | name
-
--asc
,
--desc
¶
Ascending or descending order for list deployments
-
-o
,
--output
<output>
¶ - Options
json | yaml | table | wide
-
-q
,
--quiet
¶
Hide all warnings and info logs
-
--verbose
,
--debug
¶
Show debug logs when running the command
Environment variables
-
BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
ec2¶
commands for EC2
bentoml ec2 [OPTIONS] COMMAND [ARGS]...
delete¶
Delete AWS EC2 deployment
bentoml ec2 delete [OPTIONS] NAME
Options
-
--do-not-track
¶
Specify the option to not track usage.
-
-n
,
--namespace
<namespace>
¶ Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace
-
--force
¶
force delete the deployment record in database and ignore errors when deleting cloud resources
-
-q
,
--quiet
¶
Hide all warnings and info logs
-
--verbose
,
--debug
¶
Show debug logs when running the command
Arguments
-
NAME
¶
Required argument
Environment variables
-
BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
deploy¶
Deploy BentoService to EC2
bentoml ec2 deploy [OPTIONS] NAME
Options
-
--do-not-track
¶
Specify the option to not track usage.
-
-b
,
--bento
<bento>
¶ Required
-
-n
,
--namespace
<namespace>
¶
-
--region
<region>
¶ Region to deploy service in
-
--min-size
<min_size>
¶ The minimum limit helps ensure that you always have a certain number of instances running at all times.Default is 1
-
--desired-capacity
<desired_capacity>
¶ Desired number of instances size to run BentoService on.Should be between minimum and maximum capacities.Default is 1
-
--max-size
<max_size>
¶ The maximum limit lets Amazon EC2 Auto Scaling scale out the number of instances as needed to handle an increase in demand. Default is 1
-
--instance-type
<instance_type>
¶ Instance type of EC2 container.Default is t2 micro
-
--ami-id
<ami_id>
¶ AMI id.Default is Amazon Linux 2
-
-o
,
--output
<output>
¶ - Options
json | yaml
-
--wait
,
--no-wait
¶
Wait for apply action to complete or encounter an error.If set to no-wait, CLI will return immediately. The default value is wait
-
-q
,
--quiet
¶
Hide all warnings and info logs
-
--verbose
,
--debug
¶
Show debug logs when running the command
Arguments
-
NAME
¶
Required argument
Environment variables
-
BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
get¶
Get EC2 deployment
bentoml ec2 get [OPTIONS] NAME
Options
-
--do-not-track
¶
Specify the option to not track usage.
-
-n
,
--namespace
<namespace>
¶ Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace
-
-o
,
--output
<output>
¶ - Options
json | yaml | table
-
-q
,
--quiet
¶
Hide all warnings and info logs
-
--verbose
,
--debug
¶
Show debug logs when running the command
Arguments
-
NAME
¶
Required argument
Environment variables
-
BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
list¶
List AWS Lambda deployments
bentoml ec2 list [OPTIONS]
Options
-
--do-not-track
¶
Specify the option to not track usage.
-
-n
,
--namespace
<namespace>
¶ Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace
-
--limit
<limit>
¶ The maximum amount of AWS Lambda deployments to be listed at once
-
--offset
<offset>
¶ The offset for list of AWS Lambda deployments
-
-l
,
--labels
<labels>
¶ Label query to filter Lambda deployments, supports ‘=’, ‘!=’, ‘IN’, ‘NotIn’, ‘Exists’, and ‘DoesNotExist’. (e.g. key1=value1, key2!=value2, key3 In (value3, value3a), key4 DoesNotExist)
-
--order-by
<order_by>
¶ - Options
created_at | name
-
--asc
,
--desc
¶
Ascending or descending order for list deployments
-
-o
,
--output
<output>
¶ - Options
json | yaml | table | wide
-
-q
,
--quiet
¶
Hide all warnings and info logs
-
--verbose
,
--debug
¶
Show debug logs when running the command
Environment variables
-
BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
update¶
Update existing AWS EC2 deployments
bentoml ec2 update [OPTIONS] NAME
Options
-
--do-not-track
¶
Specify the option to not track usage.
-
-b
,
--bento
,
--bento-service-bundle
<bento>
¶ Target BentoService to be deployed, referenced by its name and version in format of name:version. For example: “iris_classifier:v1.2.0”
-
-n
,
--namespace
<namespace>
¶ Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace
-
--min-size
<min_size>
¶ The minimum limit helps ensure that you always have a certain number of instances running at all times.Default is 1
-
--desired-capacity
<desired_capacity>
¶ Desired number of instances to run BentoService on.Should be between minimum and maximum capacities.Default is 1
-
--max-size
<max_size>
¶ The maximum limit lets Amazon EC2 Auto Scaling scale out the number of instances as needed to handle an increase in demand. Default is 1
-
--instance-type
<instance_type>
¶ Instance type of EC2 container.Default is t2 micro
-
--ami-id
<ami_id>
¶ AMI id.Default is Amazon Linux 2
-
-o
,
--output
<output>
¶ - Options
json | yaml | table
-
--wait
,
--no-wait
¶
Wait for apply action to complete or encounter an error.If set to no-wait, CLI will return immediately. The default value is wait
-
-q
,
--quiet
¶
Hide all warnings and info logs
-
--verbose
,
--debug
¶
Show debug logs when running the command
Arguments
-
NAME
¶
Required argument
Environment variables
-
BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
get¶
Get BentoService information
bentoml get [OPTIONS] BENTO
Options
-
--do-not-track
¶
Specify the option to not track usage.
-
--limit
<limit>
¶ Limit how many resources will be retrieved
-
--ascending-order
¶
-
--print-location
¶
-
--labels
<labels>
¶ Label query to filter BentoServices, supports ‘=’, ‘!=’, ‘IN’, ‘NotIn’, ‘Exists’, and ‘DoesNotExist’. (e.g. key1=value1, key2!=value2, key3 In (value3, value3a), key4 DoesNotExist)
-
--yatai-url
<yatai_url>
¶ Remote YataiService URL. Optional. Example: “–yatai-url http://localhost:50050”
-
-o
,
--output
<output>
¶ - Options
json | yaml | table | wide
-
-q
,
--quiet
¶
Hide all warnings and info logs
-
--verbose
,
--debug
¶
Show debug logs when running the command
Arguments
-
BENTO
¶
Required argument
Environment variables
-
BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
info¶
List all APIs defined in the BentoService loaded from saved bundle
bentoml info [OPTIONS] BENTO
Options
-
--do-not-track
¶
Specify the option to not track usage.
-
--yatai-url
<yatai_url>
¶ Remote YataiService URL. Optional. Example: “–yatai-url http://localhost:50050”
-
-q
,
--quiet
¶
Hide all warnings and info logs
-
--verbose
,
--debug
¶
Show debug logs when running the command
Arguments
-
BENTO
¶
Required argument
Environment variables
-
BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
install-completion¶
Install shell command completion
bentoml install-completion [OPTIONS] [[bash|zsh|fish|powershell]] [PATH]
Options
-
--do-not-track
¶
Specify the option to not track usage.
-
--append
,
--overwrite
¶
Append the completion code to the file
-
-q
,
--quiet
¶
Hide all warnings and info logs
-
--verbose
,
--debug
¶
Show debug logs when running the command
Arguments
-
SHELL
¶
Optional argument
-
PATH
¶
Optional argument
Environment variables
-
BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
lambda¶
Commands for AWS Lambda BentoService deployments
bentoml lambda [OPTIONS] COMMAND [ARGS]...
delete¶
Delete AWS Lambda deployment
bentoml lambda delete [OPTIONS] NAME
Options
-
--do-not-track
¶
Specify the option to not track usage.
-
-n
,
--namespace
<namespace>
¶ Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace
-
--force
¶
force delete the deployment record in database and ignore errors when deleting cloud resources
-
-q
,
--quiet
¶
Hide all warnings and info logs
-
--verbose
,
--debug
¶
Show debug logs when running the command
Arguments
-
NAME
¶
Required argument
Environment variables
-
BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
deploy¶
Deploy BentoService to AWS Lambda
bentoml lambda deploy [OPTIONS] NAME
Options
-
--do-not-track
¶
Specify the option to not track usage.
-
-b
,
--bento
,
--bento-service-bundle
<bento>
¶ Required Target BentoService to be deployed, referenced by its name and version in format of name:version. For example: “iris_classifier:v1.2.0”
-
-n
,
--namespace
<namespace>
¶ Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace
-
-l
,
--labels
<labels>
¶ Key:value pairs that are attached to deployments and intended to be used to specify identifying attributes of the deployments that are meaningful to users. Multiple labels are separated with ,
-
--region
<region>
¶ AWS region name for deployment
-
--api-name
<api_name>
¶ User defined API function will be used for inference
-
--memory-size
<memory_size>
¶ Maximum Memory Capacity for AWS Lambda function, you can set the memory size in 64MB increments from 128MB to 3008MB. The default value is 1024 MB.
-
--timeout
<timeout>
¶ The amount of time that AWS Lambda allows a function to run before stopping it. The default is 3 seconds. The maximum allowed value is 900 seconds
-
-o
,
--output
<output>
¶ - Options
json | yaml
-
--wait
,
--no-wait
¶
Wait for apply action to complete or encounter an error.If set to no-wait, CLI will return immediately. The default value is wait
-
-q
,
--quiet
¶
Hide all warnings and info logs
-
--verbose
,
--debug
¶
Show debug logs when running the command
Arguments
-
NAME
¶
Required argument
Environment variables
-
BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
get¶
Get AWS Lambda deployment information
bentoml lambda get [OPTIONS] NAME
Options
-
--do-not-track
¶
Specify the option to not track usage.
-
-n
,
--namespace
<namespace>
¶ Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace
-
-o
,
--output
<output>
¶ - Options
json | yaml | table
-
-q
,
--quiet
¶
Hide all warnings and info logs
-
--verbose
,
--debug
¶
Show debug logs when running the command
Arguments
-
NAME
¶
Required argument
Environment variables
-
BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
list¶
List AWS Lambda deployments
bentoml lambda list [OPTIONS]
Options
-
--do-not-track
¶
Specify the option to not track usage.
-
-n
,
--namespace
<namespace>
¶ Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace
-
--limit
<limit>
¶ The maximum amount of AWS Lambda deployments to be listed at once
-
-l
,
--labels
<labels>
¶ Label query to filter Lambda deployments, supports ‘=’, ‘!=’, ‘IN’, ‘NotIn’, ‘Exists’, and ‘DoesNotExist’. (e.g. key1=value1, key2!=value2, key3 In (value3, value3a), key4 DoesNotExist)
-
--order-by
<order_by>
¶ - Options
created_at | name
-
--asc
,
--desc
¶
Ascending or descending order for list deployments
-
-o
,
--output
<output>
¶ - Options
json | yaml | table | wide
-
-q
,
--quiet
¶
Hide all warnings and info logs
-
--verbose
,
--debug
¶
Show debug logs when running the command
Environment variables
-
BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
update¶
Update existing AWS Lambda deployment
bentoml lambda update [OPTIONS] NAME
Options
-
--do-not-track
¶
Specify the option to not track usage.
-
-b
,
--bento
,
--bento-service-bundle
<bento>
¶ Target BentoService to be deployed, referenced by its name and version in format of name:version. For example: “iris_classifier:v1.2.0”
-
-n
,
--namespace
<namespace>
¶ Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace
-
--memory-size
<memory_size>
¶ Maximum memory capacity for AWS Lambda function in MB, you can set the memory size in 64MB increments from 128 to 3008. The default value is 1024
-
--timeout
<timeout>
¶ The amount of time that AWS Lambda allows a function to run before stopping it. The default is 3 seconds. The maximum allowed value is 900 seconds
-
-o
,
--output
<output>
¶ - Options
json | yaml
-
--wait
,
--no-wait
¶
Wait for apply action to complete or encounter an error.If set to no-wait, CLI will return immediately. The default value is wait
-
-q
,
--quiet
¶
Hide all warnings and info logs
-
--verbose
,
--debug
¶
Show debug logs when running the command
Arguments
-
NAME
¶
Required argument
Environment variables
-
BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
list¶
List BentoServices information
bentoml list [OPTIONS]
Options
-
--do-not-track
¶
Specify the option to not track usage.
-
--limit
<limit>
¶ Limit how many BentoServices will be retrieved
-
--offset
<offset>
¶ How many BentoServices will be skipped
-
--labels
<labels>
¶ Label query to filter BentoServices, supports ‘=’, ‘!=’, ‘IN’, ‘NotIn’, ‘Exists’, and ‘DoesNotExist’. (e.g. key1=value1, key2!=value2, key3 In (value3, value3a), key4 DoesNotExist)
-
--order-by
<order_by>
¶ - Options
created_at | name
-
--ascending-order
¶
-
--yatai-url
<yatai_url>
¶ Remote YataiService URL. Optional. Example: “–yatai-url http://localhost:50050”
-
-o
,
--output
<output>
¶ - Options
json | yaml | table | wide
-
-q
,
--quiet
¶
Hide all warnings and info logs
-
--verbose
,
--debug
¶
Show debug logs when running the command
Environment variables
-
BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
open-api-spec¶
Display API specification JSON in Open-API format
bentoml open-api-spec [OPTIONS] BENTO
Options
-
--do-not-track
¶
Specify the option to not track usage.
-
--yatai-url
<yatai_url>
¶ Remote YataiService URL. Optional. Example: “–yatai-url http://localhost:50050”
-
-q
,
--quiet
¶
Hide all warnings and info logs
-
--verbose
,
--debug
¶
Show debug logs when running the command
Arguments
-
BENTO
¶
Required argument
Environment variables
-
BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
pull¶
Pull BentoService from remote yatai server
bentoml pull [OPTIONS] BENTO
Options
-
--do-not-track
¶
Specify the option to not track usage.
-
--yatai-url
<yatai_url>
¶ Required Remote YataiService URL. Example: “–yatai-url http://localhost:50050”
-
-q
,
--quiet
¶
Hide all warnings and info logs
-
--verbose
,
--debug
¶
Show debug logs when running the command
Arguments
-
BENTO
¶
Required argument
Environment variables
-
BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
push¶
Push BentoService to remote yatai server
bentoml push [OPTIONS] BENTO
Options
-
--do-not-track
¶
Specify the option to not track usage.
-
--yatai-url
<yatai_url>
¶ Required Remote YataiService URL. Example: “–yatai-url http://localhost:50050”
-
--with-labels
,
--without-labels
¶
Retain bento bundle’s labels or not for push Bento bundle to remote yatai. When running with –without-labels, labels are not retained in the remote yatai server
-
-q
,
--quiet
¶
Hide all warnings and info logs
-
--verbose
,
--debug
¶
Show debug logs when running the command
Arguments
-
BENTO
¶
Required argument
Environment variables
-
BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
retrieve¶
Retrieve
bentoml retrieve [OPTIONS] BENTO
Options
-
--do-not-track
¶
Specify the option to not track usage.
-
--yatai-url
<yatai_url>
¶ Remote YataiService URL. Example: “–yatai-url http://localhost:50050”
-
--target_dir
<target_dir>
¶ Target directory to save BentoService. Defaults to the current directory
-
-q
,
--quiet
¶
Hide all warnings and info logs
-
--verbose
,
--debug
¶
Show debug logs when running the command
Arguments
-
BENTO
¶
Required argument
Environment variables
-
BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
run¶
Run a API defined in saved BentoService bundle from command line
bentoml run [OPTIONS] BENTO API_NAME [RUN_ARGS]...
Options
-
--do-not-track
¶
Specify the option to not track usage.
-
-q
,
--quiet
¶
Hide all warnings and info logs
-
--verbose
,
--debug
¶
Show debug logs when running the command
Arguments
-
BENTO
¶
Required argument
-
API_NAME
¶
Required argument
-
RUN_ARGS
¶
Optional argument(s)
Environment variables
-
BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
sagemaker¶
Commands for AWS Sagemaker BentoService deployments
bentoml sagemaker [OPTIONS] COMMAND [ARGS]...
delete¶
Delete AWS Sagemaker deployment
bentoml sagemaker delete [OPTIONS] NAME
Options
-
--do-not-track
¶
Specify the option to not track usage.
-
-n
,
--namespace
<namespace>
¶ Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace
-
--force
¶
force delete the deployment record in database and ignore errors when deleting cloud resources
-
-q
,
--quiet
¶
Hide all warnings and info logs
-
--verbose
,
--debug
¶
Show debug logs when running the command
Arguments
-
NAME
¶
Required argument
Environment variables
-
BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
deploy¶
Deploy BentoService to AWS Sagemaker
bentoml sagemaker deploy [OPTIONS] NAME
Options
-
--do-not-track
¶
Specify the option to not track usage.
-
-b
,
--bento
,
--bento-service-bundle
<bento>
¶ Required Target BentoService to be deployed, referenced by its name and version in format of name:version. For example: “iris_classifier:v1.2.0”
-
-n
,
--namespace
<namespace>
¶ Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace
-
-l
,
--labels
<labels>
¶ Key:value pairs that are attached to deployments and intended to be used to specify identifying attributes of the deployments that are meaningful to users. Multiple labels are separated with ,
-
--region
<region>
¶ AWS region name for deployment
-
--api-name
<api_name>
¶ Required User defined API function will be used for inference.
-
--instance-type
<instance_type>
¶ Type of instance will be used for inference. Default to “m1.m4.xlarge”
-
--instance-count
<instance_count>
¶ Number of instance will be used. Default value is 1
-
--num-of-gunicorn-workers-per-instance
<num_of_gunicorn_workers_per_instance>
¶ Number of gunicorn worker will be used per instance. Default value for gunicorn worker is based on the instance’ cpu core counts. The formula is num_of_cpu/2 + 1
-
--timeout
<timeout>
¶ The amount of time Sagemaker will wait before return response
-
-o
,
--output
<output>
¶ - Options
json | yaml
-
--wait
,
--no-wait
¶
Wait for apply action to complete or encounter an error.If set to no-wait, CLI will return immediately. The default value is wait
-
--data-capture-s3-prefix
<data_capture_s3_prefix>
¶ To enable data capture (input and output), provide a destination s3 prefix (of the format s3://bucket-name/optional/prefix) for the captured data. To disable data capture, leave this blank.
-
--data-capture-sample-percent
<data_capture_sample_percent>
¶ When data capture is enabled, the sampling percentage. Default 100%. No effect without data-capture-s3-prefix.
-
-q
,
--quiet
¶
Hide all warnings and info logs
-
--verbose
,
--debug
¶
Show debug logs when running the command
Arguments
-
NAME
¶
Required argument
Environment variables
-
BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
get¶
Get AWS Sagemaker deployment information
bentoml sagemaker get [OPTIONS] NAME
Options
-
--do-not-track
¶
Specify the option to not track usage.
-
-n
,
--namespace
<namespace>
¶ Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace
-
-o
,
--output
<output>
¶ - Options
json | yaml | table
-
-q
,
--quiet
¶
Hide all warnings and info logs
-
--verbose
,
--debug
¶
Show debug logs when running the command
Arguments
-
NAME
¶
Required argument
Environment variables
-
BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
list¶
List AWS Sagemaker deployment information
bentoml sagemaker list [OPTIONS]
Options
-
--do-not-track
¶
Specify the option to not track usage.
-
-n
,
--namespace
<namespace>
¶ Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration yatai_service/default_namespace
-
--limit
<limit>
¶ The maximum amount of AWS Sagemaker deployments to be listed at once
-
-l
,
--labels
<labels>
¶ Label query to filter Sagemaker deployments, supports ‘=’, ‘!=’, ‘IN’, ‘NotIn’, ‘Exists’, and ‘DoesNotExist’. (e.g. key1=value1, key2!=value2, key3 In (value3, value3a), key4 DoesNotExist)
-
--order-by
<order_by>
¶ - Options
created_at | name
-
--asc
,
--desc
¶
Ascending or descending order for list deployments
-
-o
,
--output
<output>
¶ - Options
json | yaml | table | wide
-
-q
,
--quiet
¶
Hide all warnings and info logs
-
--verbose
,
--debug
¶
Show debug logs when running the command
Environment variables
-
BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
update¶
Update existing AWS Sagemaker deployment
bentoml sagemaker update [OPTIONS] NAME
Options
-
--do-not-track
¶
Specify the option to not track usage.
-
-b
,
--bento
,
--bento-service-bundle
<bento>
¶ Target BentoService to be deployed, referenced by its name and version in format of name:version. For example: “iris_classifier:v1.2.0”
-
-n
,
--namespace
<namespace>
¶ Deployment namespace managed by BentoML, default value is “dev” which can be changed in BentoML configuration file
-
--instance-type
<instance_type>
¶ Type of instance will be used for inference. Default to “m1.m4.xlarge”
-
--instance-count
<instance_count>
¶ Number of instance will be used. Default value is 1
-
--num-of-gunicorn-workers-per-instance
<num_of_gunicorn_workers_per_instance>
¶ Number of gunicorn worker will be used per instance. Default value for gunicorn worker is based on the instance’ cpu core counts. The formula is num_of_cpu/2 + 1
-
--api-name
<api_name>
¶ User defined API function will be used for inference.
-
--timeout
<timeout>
¶ The amount of time Sagemaker will wait before return response
-
-o
,
--output
<output>
¶ - Options
json | yaml
-
--wait
,
--no-wait
¶
Wait for apply action to complete or encounter an error.If set to no-wait, CLI will return immediately. The default value is wait
-
--data-capture-s3-prefix
<data_capture_s3_prefix>
¶ To enable data capture (input and output), provide a destination s3 prefix (of the format s3://bucket-name/optional/prefix) for the captured data. To disable data capture, leave this blank.
-
--data-capture-sample-percent
<data_capture_sample_percent>
¶ When data capture is enabled, the sampling percentage. Default 100%. No effect without data-capture-s3-prefix.
-
-q
,
--quiet
¶
Hide all warnings and info logs
-
--verbose
,
--debug
¶
Show debug logs when running the command
Arguments
-
NAME
¶
Required argument
Environment variables
-
BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
serve¶
Start a dev API server serving specified BentoService
bentoml serve [OPTIONS] BENTO
Options
-
--do-not-track
¶
Specify the option to not track usage.
-
--port
<port>
¶ The port to listen on for the REST api server, default is 5000
-
--enable-microbatch
,
--disable-microbatch
¶
Deprecated option for running API server with micro-batch enabled.
-
--mb-max-batch-size
<mb_max_batch_size>
¶ Specify micro batching maximal batch size.
-
--mb-max-latency
<mb_max_latency>
¶ Specify micro batching maximal latency in milliseconds.
-
--run-with-ngrok
¶
Use ngrok to relay traffic on a public endpoint to this API server on localhost
-
--yatai-url
<yatai_url>
¶ Remote YataiService URL. Optional. Example: “–yatai-url http://localhost:50050”
-
--enable-swagger
,
--disable-swagger
¶
Run API server with Swagger UI enabled
-
-q
,
--quiet
¶
Hide all warnings and info logs
-
--verbose
,
--debug
¶
Show debug logs when running the command
Arguments
-
BENTO
¶
Required argument
Environment variables
-
BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
-
BENTOML_PORT
Provide a default for
--port
-
BENTOML_ENABLE_MICROBATCH
Provide a default for
--enable-microbatch
-
BENTOML_MB_MAX_BATCH_SIZE
Provide a default for
--mb-max-batch-size
-
BENTOML_MB_MAX_LATENCY
Provide a default for
--mb-max-latency
-
BENTOML_ENABLE_NGROK
Provide a default for
--run-with-ngrok
-
BENTOML_ENABLE_SWAGGER
Provide a default for
--enable-swagger
serve-gunicorn¶
Start a production API server serving specified BentoService
bentoml serve-gunicorn [OPTIONS] BENTO
Options
-
--do-not-track
¶
Specify the option to not track usage.
-
-p
,
--port
<port>
¶ The port to listen on for the REST api server, default is 5000
-
-w
,
--workers
<workers>
¶ Number of workers will start for the gunicorn server
-
--timeout
<timeout>
¶ Number of seconds for an API server request to timeout
-
--enable-microbatch
,
--disable-microbatch
¶
Deprecated option for running API server with micro-batch enabled.
-
--mb-max-batch-size
<mb_max_batch_size>
¶ Specify micro batching maximal batch size.
-
--mb-max-latency
<mb_max_latency>
¶ Specify micro batching maximal latency in milliseconds.
-
--microbatch-workers
<microbatch_workers>
¶ Number of micro-batch request dispatcher workers
-
--yatai-url
<yatai_url>
¶ Remote YataiService URL. Optional. Example: “–yatai-url http://localhost:50050”
-
--enable-swagger
,
--disable-swagger
¶
Run API server with Swagger UI enabled
-
-q
,
--quiet
¶
Hide all warnings and info logs
-
--verbose
,
--debug
¶
Show debug logs when running the command
Arguments
-
BENTO
¶
Required argument
Environment variables
-
BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
-
BENTOML_PORT
Provide a default for
-p
-
BENTOML_GUNICORN_WORKERS
Provide a default for
-w
-
BENTOML_ENABLE_MICROBATCH
Provide a default for
--enable-microbatch
-
BENTOML_MB_MAX_BATCH_SIZE
Provide a default for
--mb-max-batch-size
-
BENTOML_MB_MAX_LATENCY
Provide a default for
--mb-max-latency
-
BENTOML_MICROBATCH_WORKERS
Provide a default for
--microbatch-workers
-
BENTOML_ENABLE_SWAGGER
Provide a default for
--enable-swagger
yatai-service-start¶
Start BentoML YataiService for model management and deployment
bentoml yatai-service-start [OPTIONS]
Options
-
--do-not-track
¶
Specify the option to not track usage.
-
--db-url
<db_url>
¶ Database URL following RFC-1738, and usually can include username, password, hostname, database name as well as optional keyword arguments for additional configuration
-
--repo-base-url
<repo_base_url>
¶ Deprecated option for base URL for storing BentoML saved bundle files, this can be a file system path(POSIX/Windows), or a S3 URL, usually starting with “s3://”
-
--grpc-port
<grpc_port>
¶ Port to run YataiService gRPC server
-
--ui-port
<ui_port>
¶ Port to run YataiService Web UI server
-
--ui
,
--no-ui
¶
Run YataiService with or without Web UI, when running with –no-ui, it will only run the gRPC server
-
--web-prefix-path
<web_prefix_path>
¶ Add a location prefix to the URL when running YataiServicebehind a reverse proxy server
-
--repository-type
<repository_type>
¶ Type of the repository implementation
- Options
file_system | s3 | gcs
-
--file-system-directory
<file_system_directory>
¶ Specifies the directory path for the file system repository type
-
--s3-url
<s3_url>
¶ Specifies the S3 URL for the S3 repository type
-
--s3-endpoint-url
<s3_endpoint_url>
¶ S3 Endpoint URL is used for deploying with storage services that are compatible with Amazon S3, such as MinIO
-
--gcs-url
<gcs_url>
¶ Specifies the GCS URL for the GCS repository type
-
-q
,
--quiet
¶
Hide all warnings and info logs
-
--verbose
,
--debug
¶
Show debug logs when running the command
Environment variables
-
BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
-
BENTOML_DB_URL
Provide a default for
--db-url
-
BENTOML_REPO_BASE_URL
Provide a default for
--repo-base-url
-
BENTOML_GRPC_PORT
Provide a default for
--grpc-port
-
BENTOML_WEB_UI_PORT
Provide a default for
--ui-port
-
BENTOML_ENABLE_WEB_UI
Provide a default for
--ui
-
BENTOML_YATAI_WEB_PREFIX_PATH
Provide a default for
--web-prefix-path
-
BENTOML_YATAI_REPOSITORY_TYPE
Provide a default for
--repository-type
-
BENTOML_YATAI_FILE_SYSTEM_DIRECTORY
Provide a default for
--file-system-directory
-
BENTOML_YATAI_S3_URL
Provide a default for
--s3-url
-
BENTOML_S3_ENDPOINT_URL
Provide a default for
--s3-endpoint-url
-
BENTOML_YATAI_GCS_URL
Provide a default for
--gcs-url