CLI Reference¶
bentoml¶
BentoML CLI tool
bentoml [OPTIONS] COMMAND [ARGS]...
Options
-
--version¶ Show the version and exit.
azure-functions¶
Commands for Azure Functions BentoService deployment
bentoml azure-functions [OPTIONS] COMMAND [ARGS]...
delete¶
Delete Azure Functions deployment
bentoml azure-functions delete [OPTIONS] NAME
Options
-
-n,--namespace<namespace>¶ Deployment namespace managed by BentoML, default value is “dev” whichcan be changed in BentoML configuration yatai_service/default_namespace
-
--force¶ force delete the deployment record in database and ignore errors when deleting cloud resources
-
-q,--quiet¶ Hide all warnings and info logs
-
--verbose,--debug¶ Show debug logs when running the command
Arguments
-
NAME¶ Required argument
deploy¶
Deploy BentoService to Azure Functions
bentoml azure-functions deploy [OPTIONS] NAME
Options
-
-n,--namespace<namespace>¶ Deployment namespace managed by BentoML, default value is “dev” whichcan be changed in BentoML configuration yatai_service/default_namespace
-
-b,--bento,--bento-service-bundle<bento>¶ Required Target BentoService to be deployed, referenced by its name and version in the format of name:version. For example: “iris_classifier:v1.2.0”
-
--location<location>¶ Required The Azure location name for the deployment
-
--min-instances<min_instances>¶ The minimum number of workers for the deployment. The default value is 1
-
--max-burst<max_burst>¶ The maximum number of elastic workers for the deployment. The default value is 20
The Azure Functions premium SKU for the deployment. The default value is EP1
- Options
EP1|EP2|EP3
-
-l,--labels<labels>¶ Key:value pairs that are attached to deployments and intended to be used to specify identifying attributes of the deployments that are meaningful to users. Multiple labels are separated with ,
-
--function-auth-level<function_auth_level>¶ The authorization level for the deployed Azure Functions. The default value is anonymous
- Options
anonymous|function|admin
-
-o,--output<output>¶ - Options
json|yaml
-
--wait,--no-wait¶ Wait for apply action to complete or encounter an error.If set to no-wait, CLI will exit without waiting until it has verified cloud resource allocation
-
-q,--quiet¶ Hide all warnings and info logs
-
--verbose,--debug¶ Show debug logs when running the command
Arguments
-
NAME¶ Required argument
get¶
Get Azure Functions deployment information
bentoml azure-functions get [OPTIONS] NAME
Options
-
-n,--namespace<namespace>¶ Deployment namespace managed by BentoML, default value is “dev” whichcan be changed in BentoML configuration yatai_service/default_namespace
-
-o,--output<output>¶ - Options
json|yaml|table
-
-q,--quiet¶ Hide all warnings and info logs
-
--verbose,--debug¶ Show debug logs when running the command
Arguments
-
NAME¶ Required argument
list¶
List Azure Functions deployments
bentoml azure-functions list [OPTIONS]
Options
-
-n,--namespace<namespace>¶ Deployment namespace managed by BentoML, default value is “dev” whichcan be changed in BentoML configuration yatai_service/default_namespace
-
--limit<limit>¶ The maximum amount of Azure Functions deployments to be listed at once
-
-l,--labels<labels>¶ List deployments matching the giving labels
-
--order-by<order_by>¶ - Options
created_at|name
-
--asc,--desc¶ Ascending or descending order for list deployments
-
-o,--output<output>¶ - Options
json|yaml|table
-
-q,--quiet¶ Hide all warnings and info logs
-
--verbose,--debug¶ Show debug logs when running the command
update¶
Update existing Azure Functions deployment
bentoml azure-functions update [OPTIONS] NAME
Options
-
-b,--bento,--bento-service-bundle<bento>¶ Target BentoService to be deployed, referenced by its name and version in the format of name:version. For example: “iris_classifier:v1.2.0”
-
-n,--namespace<namespace>¶ Deployment namespace managed by BentoML, the default value is “dev” which can be changed in BentoML configuration file
-
--min-instances<min_instances>¶ The minimum number of workers for the deployment.
-
--max-burst<max_burst>¶ The maximum number of elastic workers for the deployment.
The Azure Functions premium SKU for the deployment.
- Options
EP1|EP2|EP3
-
-o,--output<output>¶ - Options
json|yaml
-
--wait,--no-wait¶ Wait for apply action to complete or encounter an error.If set to no-wait, CLI will exit without waiting until it has verified cloud resource allocation
-
-q,--quiet¶ Hide all warnings and info logs
-
--verbose,--debug¶ Show debug logs when running the command
Arguments
-
NAME¶ Required argument
config¶
Configure BentoML configurations and settings
bentoml config [OPTIONS] COMMAND [ARGS]...
reset¶
Reset all local BentoML configs to default
bentoml config reset [OPTIONS]
Options
-
-q,--quiet¶ Hide all warnings and info logs
-
--verbose,--debug¶ Show debug logs when running the command
set¶
Set config value in local BentoML configuration file
bentoml config set [OPTIONS] [UPDATES]...
Options
-
-q,--quiet¶ Hide all warnings and info logs
-
--verbose,--debug¶ Show debug logs when running the command
Arguments
-
UPDATES¶ Optional argument(s)
unset¶
Unset config in local BentoML configuration file
bentoml config unset [OPTIONS] [UPDATES]...
Options
-
-q,--quiet¶ Hide all warnings and info logs
-
--verbose,--debug¶ Show debug logs when running the command
Arguments
-
UPDATES¶ Optional argument(s)
containerize¶
Containerizes given Bento into a ready-to-use Docker image.
bentoml containerize [OPTIONS] BENTO
Options
-
-p,--push¶
-
-t,--tag<tag>¶ Optional image tag. If not specified, Bento will generate one from the name of the Bento.
-
-u,--username<username>¶
-
-p,--password<password>¶
-
-q,--quiet¶ Hide all warnings and info logs
-
--verbose,--debug¶ Show debug logs when running the command
Arguments
-
BENTO¶ Required argument
delete¶
Delete saved BentoService.
BENTO is the target BentoService to be deleted, referenced by its name and version in format of name:version. For example: “iris_classifier:v1.2.0”
bentoml delete command also supports deleting multiple saved BentoService at once, by providing name version tag separated by “,”, for example:
bentoml delete iris_classifier:v1.2.0,my_svc:v1,my_svc2:v3
bentoml delete [OPTIONS] BENTOS
Options
-
-y,--yes,--assume-yes¶ Automatic yes to prompts
-
-q,--quiet¶ Hide all warnings and info logs
-
--verbose,--debug¶ Show debug logs when running the command
Arguments
-
BENTOS¶ Required argument
deployment¶
Commands for manageing and operating BentoService deployments
bentoml deployment [OPTIONS] COMMAND [ARGS]...
apply¶
Apply BentoService deployment from yaml file
bentoml deployment apply [OPTIONS]
Options
-
-f,--file<deployment_yaml>¶ Required
-
-o,--output<output>¶ - Options
json|yaml
-
--wait,--no-wait¶ Wait for apply action to complete or encounter an error.If set to no-wait, CLI will return immediately. The default value is wait
-
-q,--quiet¶ Hide all warnings and info logs
-
--verbose,--debug¶ Show debug logs when running the command
create¶
Create BentoService deployment from yaml file
bentoml deployment create [OPTIONS]
Options
-
-f,--file<deployment_yaml>¶ Required
-
-o,--output<output>¶ - Options
json|yaml
-
--wait,--no-wait¶ Wait for apply action to complete or encounter an error.If set to no-wait, CLI will return immediately. The default value is wait
-
-q,--quiet¶ Hide all warnings and info logs
-
--verbose,--debug¶ Show debug logs when running the command
delete¶
Delete deployment
bentoml deployment delete [OPTIONS] NAME
Options
-
-n,--namespace<namespace>¶ Deployment namespace managed by BentoML, default value is “default” whichcan be changed in BentoML configuration file
-
--force¶ force delete the deployment record in database and ignore errors when deleting cloud resources
-
-q,--quiet¶ Hide all warnings and info logs
-
--verbose,--debug¶ Show debug logs when running the command
Arguments
-
NAME¶ Required argument
get¶
Get deployment information
bentoml deployment get [OPTIONS] NAME
Options
-
-n,--namespace<namespace>¶ Deployment namespace managed by BentoML, default value is “dev” whichcan be changed in BentoML configuration file
-
-o,--output<output>¶ - Options
json|yaml
-
-q,--quiet¶ Hide all warnings and info logs
-
--verbose,--debug¶ Show debug logs when running the command
Arguments
-
NAME¶ Required argument
list¶
List deployments
bentoml deployment list [OPTIONS]
Options
-
-n,--namespace<namespace>¶ Deployment namespace managed by BentoML, default value is “dev” whichcan be changed in BentoML configuration file
-
-p,--platform<platform>¶ platform
- Options
sagemaker|lambda
-
--limit<limit>¶ The maximum amount of deployments to be listed at once
-
-l,--labels<labels>¶ List deployments matching the giving labels
-
--order-by<order_by>¶ - Options
created_at|name
-
--asc,--desc¶ Ascending or descending order for list deployments
-
-o,--output<output>¶ - Options
json|yaml|table
-
-q,--quiet¶ Hide all warnings and info logs
-
--verbose,--debug¶ Show debug logs when running the command
get¶
Get BentoService information
bentoml get [OPTIONS] BENTO
Options
-
--limit<limit>¶ Limit how many resources will be retrieved
-
--ascending-order¶
-
--print-location¶
-
-o,--output<output>¶ - Options
json|yaml|table|wide
-
-q,--quiet¶ Hide all warnings and info logs
-
--verbose,--debug¶ Show debug logs when running the command
Arguments
-
BENTO¶ Required argument
info¶
List all APIs defined in the BentoService loaded from saved bundle
bentoml info [OPTIONS] BENTO
Options
-
-q,--quiet¶ Hide all warnings and info logs
-
--verbose,--debug¶ Show debug logs when running the command
Arguments
-
BENTO¶ Required argument
install-completion¶
Install shell command completion
bentoml install-completion [OPTIONS] [[bash|zsh|fish|powershell]] [PATH]
Options
-
--append,--overwrite¶ Append the completion code to the file
-
-q,--quiet¶ Hide all warnings and info logs
-
--verbose,--debug¶ Show debug logs when running the command
Arguments
-
SHELL¶ Optional argument
-
PATH¶ Optional argument
lambda¶
Commands for AWS Lambda BentoService deployments
bentoml lambda [OPTIONS] COMMAND [ARGS]...
delete¶
Delete AWS Lambda deployment
bentoml lambda delete [OPTIONS] NAME
Options
-
-n,--namespace<namespace>¶ Deployment namespace managed by BentoML, default value is “dev” whichcan be changed in BentoML configuration yatai_service/default_namespace
-
--force¶ force delete the deployment record in database and ignore errors when deleting cloud resources
-
-q,--quiet¶ Hide all warnings and info logs
-
--verbose,--debug¶ Show debug logs when running the command
Arguments
-
NAME¶ Required argument
deploy¶
Deploy BentoService to AWS Lambda
bentoml lambda deploy [OPTIONS] NAME
Options
-
-b,--bento,--bento-service-bundle<bento>¶ Required Target BentoService to be deployed, referenced by its name and version in format of name:version. For example: “iris_classifier:v1.2.0”
-
-n,--namespace<namespace>¶ Deployment namespace managed by BentoML, default value is “dev” whichcan be changed in BentoML configuration yatai_service/default_namespace
-
-l,--labels<labels>¶ Key:value pairs that are attached to deployments and intended to be usedto specify identifying attributes of the deployments that are meaningful to users. Multiple labels are separated with ,
-
--region<region>¶ AWS region name for deployment
-
--api-name<api_name>¶ User defined API function will be used for inference
-
--memory-size<memory_size>¶ Maximum Memory Capacity for AWS Lambda function, you can set the memory size in 64MB increments from 128MB to 3008MB. The default value is 1024 MB.
-
--timeout<timeout>¶ The amount of time that AWS Lambda allows a function to run before stopping it. The default is 3 seconds. The maximum allowed value is 900 seconds
-
-o,--output<output>¶ - Options
json|yaml
-
--wait,--no-wait¶ Wait for apply action to complete or encounter an error.If set to no-wait, CLI will return immediately. The default value is wait
-
-q,--quiet¶ Hide all warnings and info logs
-
--verbose,--debug¶ Show debug logs when running the command
Arguments
-
NAME¶ Required argument
get¶
Get AWS Lambda deployment information
bentoml lambda get [OPTIONS] NAME
Options
-
-n,--namespace<namespace>¶ Deployment namespace managed by BentoML, default value is “dev” whichcan be changed in BentoML configuration yatai_service/default_namespace
-
-o,--output<output>¶ - Options
json|yaml|table
-
-q,--quiet¶ Hide all warnings and info logs
-
--verbose,--debug¶ Show debug logs when running the command
Arguments
-
NAME¶ Required argument
list¶
List AWS Lambda deployments
bentoml lambda list [OPTIONS]
Options
-
-n,--namespace<namespace>¶ Deployment namespace managed by BentoML, default value is “dev” whichcan be changed in BentoML configuration yatai_service/default_namespace
-
--limit<limit>¶ The maximum amount of AWS Lambda deployments to be listed at once
-
-l,--labels<labels>¶ List deployments matching the giving labels
-
--order-by<order_by>¶ - Options
created_at|name
-
--asc,--desc¶ Ascending or descending order for list deployments
-
-o,--output<output>¶ - Options
json|yaml|table
-
-q,--quiet¶ Hide all warnings and info logs
-
--verbose,--debug¶ Show debug logs when running the command
update¶
Update existing AWS Lambda deployment
bentoml lambda update [OPTIONS] NAME
Options
-
-b,--bento,--bento-service-bundle<bento>¶ Target BentoService to be deployed, referenced by its name and version in format of name:version. For example: “iris_classifier:v1.2.0”
-
-n,--namespace<namespace>¶ Deployment namespace managed by BentoML, default value is “dev” whichcan be changed in BentoML configuration yatai_service/default_namespace
-
--memory-size<memory_size>¶ Maximum memory capacity for AWS Lambda function in MB, you can set the memory size in 64MB increments from 128 to 3008. The default value is 1024
-
--timeout<timeout>¶ The amount of time that AWS Lambda allows a function to run before stopping it. The default is 3 seconds. The maximum allowed value is 900 seconds
-
-o,--output<output>¶ - Options
json|yaml
-
--wait,--no-wait¶ Wait for apply action to complete or encounter an error.If set to no-wait, CLI will return immediately. The default value is wait
-
-q,--quiet¶ Hide all warnings and info logs
-
--verbose,--debug¶ Show debug logs when running the command
Arguments
-
NAME¶ Required argument
list¶
List BentoServices information
bentoml list [OPTIONS]
Options
-
--limit<limit>¶ Limit how many BentoServices will be retrieved
-
--offset<offset>¶ How many BentoServices will be skipped
-
--order-by<order_by>¶ - Options
created_at|name
-
--ascending-order¶
-
-o,--output<output>¶ - Options
json|yaml|table|wide
-
-q,--quiet¶ Hide all warnings and info logs
-
--verbose,--debug¶ Show debug logs when running the command
open-api-spec¶
Display API specification JSON in Open-API format
bentoml open-api-spec [OPTIONS] BENTO
Options
-
-q,--quiet¶ Hide all warnings and info logs
-
--verbose,--debug¶ Show debug logs when running the command
Arguments
-
BENTO¶ Required argument
retrieve¶
Retrieves BentoService artifacts into a target directory
bentoml retrieve [OPTIONS] BENTO
Options
-
--target_dir<target_dir>¶ Directory to put artifacts into. Defaults to pwd.
-
-q,--quiet¶ Hide all warnings and info logs
-
--verbose,--debug¶ Show debug logs when running the command
Arguments
-
BENTO¶ Required argument
run¶
Run a API defined in saved BentoService bundle from command line
bentoml run [OPTIONS] BENTO API_NAME [RUN_ARGS]...
Options
-
-q,--quiet¶ Hide all warnings and info logs
-
--verbose,--debug¶ Show debug logs when running the command
Arguments
-
BENTO¶ Required argument
-
API_NAME¶ Required argument
-
RUN_ARGS¶ Optional argument(s)
sagemaker¶
Commands for AWS Sagemaker BentoService deployments
bentoml sagemaker [OPTIONS] COMMAND [ARGS]...
delete¶
Delete AWS Sagemaker deployment
bentoml sagemaker delete [OPTIONS] NAME
Options
-
-n,--namespace<namespace>¶ Deployment namespace managed by BentoML, default value is “dev” whichcan be changed in BentoML configuration yatai_service/default_namespace
-
--force¶ force delete the deployment record in database and ignore errors when deleting cloud resources
-
-q,--quiet¶ Hide all warnings and info logs
-
--verbose,--debug¶ Show debug logs when running the command
Arguments
-
NAME¶ Required argument
deploy¶
Deploy BentoService to AWS Sagemaker
bentoml sagemaker deploy [OPTIONS] NAME
Options
-
-b,--bento,--bento-service-bundle<bento>¶ Required Target BentoService to be deployed, referenced by its name and version in format of name:version. For example: “iris_classifier:v1.2.0”
-
-n,--namespace<namespace>¶ Deployment namespace managed by BentoML, default value is “dev” whichcan be changed in BentoML configuration yatai_service/default_namespace
-
-l,--labels<labels>¶ Key:value pairs that are attached to deployments and intended to be usedto specify identifying attributes of the deployments that are meaningful to users. Multiple labels are separated with ,
-
--region<region>¶ AWS region name for deployment
-
--api-name<api_name>¶ Required User defined API function will be used for inference.
-
--instance-type<instance_type>¶ Type of instance will be used for inference. Default to “m1.m4.xlarge”
-
--instance-count<instance_count>¶ Number of instance will be used. Default value is 1
-
--num-of-gunicorn-workers-per-instance<num_of_gunicorn_workers_per_instance>¶ Number of gunicorn worker will be used per instance. Default value for gunicorn worker is based on the instance’ cpu core counts. The formula is num_of_cpu/2 + 1
-
--timeout<timeout>¶ The amount of time Sagemaker will wait before return response
-
-o,--output<output>¶ - Options
json|yaml
-
--wait,--no-wait¶ Wait for apply action to complete or encounter an error.If set to no-wait, CLI will return immediately. The default value is wait
-
-q,--quiet¶ Hide all warnings and info logs
-
--verbose,--debug¶ Show debug logs when running the command
Arguments
-
NAME¶ Required argument
get¶
Get AWS Sagemaker deployment information
bentoml sagemaker get [OPTIONS] NAME
Options
-
-n,--namespace<namespace>¶ Deployment namespace managed by BentoML, default value is “dev” whichcan be changed in BentoML configuration yatai_service/default_namespace
-
-o,--output<output>¶ - Options
json|yaml|table
-
-q,--quiet¶ Hide all warnings and info logs
-
--verbose,--debug¶ Show debug logs when running the command
Arguments
-
NAME¶ Required argument
list¶
List AWS Sagemaker deployment information
bentoml sagemaker list [OPTIONS]
Options
-
-n,--namespace<namespace>¶ Deployment namespace managed by BentoML, default value is “dev” whichcan be changed in BentoML configuration yatai_service/default_namespace
-
--limit<limit>¶ The maximum amount of AWS Sagemaker deployments to be listed at once
-
-l,--labels<labels>¶ List deployments matching the giving labels
-
--order-by<order_by>¶ - Options
created_at|name
-
--asc,--desc¶ Ascending or descending order for list deployments
-
-o,--output<output>¶ - Options
json|yaml|table
-
-q,--quiet¶ Hide all warnings and info logs
-
--verbose,--debug¶ Show debug logs when running the command
update¶
Update existing AWS Sagemaker deployment
bentoml sagemaker update [OPTIONS] NAME
Options
-
-b,--bento,--bento-service-bundle<bento>¶ Target BentoService to be deployed, referenced by its name and version in format of name:version. For example: “iris_classifier:v1.2.0”
-
-n,--namespace<namespace>¶ Deployment namespace managed by BentoML, default value is “dev” whichcan be changed in BentoML configuration file
-
--instance-type<instance_type>¶ Type of instance will be used for inference. Default to “m1.m4.xlarge”
-
--instance-count<instance_count>¶ Number of instance will be used. Default value is 1
-
--num-of-gunicorn-workers-per-instance<num_of_gunicorn_workers_per_instance>¶ Number of gunicorn worker will be used per instance. Default value for gunicorn worker is based on the instance’ cpu core counts. The formula is num_of_cpu/2 + 1
-
--api-name<api_name>¶ User defined API function will be used for inference.
-
--timeout<timeout>¶ The amount of time Sagemaker will wait before return response
-
-o,--output<output>¶ - Options
json|yaml
-
--wait,--no-wait¶ Wait for apply action to complete or encounter an error.If set to no-wait, CLI will return immediately. The default value is wait
-
-q,--quiet¶ Hide all warnings and info logs
-
--verbose,--debug¶ Show debug logs when running the command
Arguments
-
NAME¶ Required argument
serve¶
Start a dev API server serving specified BentoService
bentoml serve [OPTIONS] BENTO
Options
-
--port<port>¶ The port to listen on for the REST api server, default is 5000
-
--enable-microbatch,--disable-microbatch¶ Run API server with micro-batch enabled
-
--run-with-ngrok¶ Use ngrok to relay traffic on a public endpoint to thisAPI server on localhost
-
-q,--quiet¶ Hide all warnings and info logs
-
--verbose,--debug¶ Show debug logs when running the command
Arguments
-
BENTO¶ Required argument
Environment variables
-
BENTOML_PORT Provide a default for
--port
-
BENTOML_ENABLE_MICROBATCH Provide a default for
--enable-microbatch
-
BENTOML_ENABLE_NGROK Provide a default for
--run-with-ngrok
serve-gunicorn¶
Start a production API server serving specified BentoService
bentoml serve-gunicorn [OPTIONS] BENTO
Options
-
-p,--port<port>¶ The port to listen on for the REST api server, default is 5000
-
-w,--workers<workers>¶ Number of workers will start for the gunicorn server
-
--timeout<timeout>¶
-
--enable-microbatch,--disable-microbatch¶ Run API server with micro batch enabled
-
--microbatch-workers<microbatch_workers>¶ Number of micro-batch request dispatcher workers
-
-q,--quiet¶ Hide all warnings and info logs
-
--verbose,--debug¶ Show debug logs when running the command
Arguments
-
BENTO¶ Required argument
Environment variables
-
BENTOML_PORT Provide a default for
-p
-
BENTOML_GUNICORN_WORKERS Provide a default for
-w
-
BENTOML_ENABLE_MICROBATCH Provide a default for
--enable-microbatch
-
BENTOML_MICROBATCH_WORKERS Provide a default for
--microbatch-workers
yatai-service-start¶
Start BentoML YataiService for model management and deployment
bentoml yatai-service-start [OPTIONS]
Options
-
--db-url<db_url>¶ Database URL following RFC-1738, and usually can include username, password, hostname, database name as well as optional keyword arguments for additional configuration
-
--repo-base-url<repo_base_url>¶ Base URL for storing BentoML saved bundle files, this can be a filesystempath(POSIX/Windows), or a S3 URL, usually starting with “s3://”
-
--grpc-port<grpc_port>¶ Port to run YataiService gRPC server
-
--ui-port<ui_port>¶ Port to run YataiService Web UI server
-
--ui,--no-ui¶ Run YataiService with or without Web UI, when running with –no-ui, it will only run the gRPC server
-
--s3-endpoint-url<s3_endpoint_url>¶ S3 Endpoint URL is used for deploying with storage services that are compatible with Amazon S3, such as MinIO
-
-q,--quiet¶ Hide all warnings and info logs
-
--verbose,--debug¶ Show debug logs when running the command
Environment variables
-
BENTOML_DB_URL Provide a default for
--db-url
-
BENTOML_REPO_BASE_URL Provide a default for
--repo-base-url
-
BENTOML_GRPC_PORT Provide a default for
--grpc-port
-
BENTOML_WEB_UI_PORT Provide a default for
--ui-port
-
BENTOML_ENABLE_WEB_UI Provide a default for
--ui
-
BENTOML_S3_ENDPOINT_URL Provide a default for
--s3-endpoint-url