BentoML CLI#

Note

BentoML CLI command itself also comes usage documentation. You can learn more from running bentoml --help. The --help flag also applies to sub-commands for viewing detailed usage of a command, e.g.: bentoml serve --help.

bentoml#

██████╗░███████╗███╗░░██╗████████╗░█████╗░███╗░░░███╗██╗░░░░░
██╔══██╗██╔════╝████╗░██║╚══██╔══╝██╔══██╗████╗░████║██║░░░░░
██████╦╝█████╗░░██╔██╗██║░░░██║░░░██║░░██║██╔████╔██║██║░░░░░
██╔══██╗██╔══╝░░██║╚████║░░░██║░░░██║░░██║██║╚██╔╝██║██║░░░░░
██████╦╝███████╗██║░╚███║░░░██║░░░╚█████╔╝██║░╚═╝░██║███████╗
╚═════╝░╚══════╝╚═╝░░╚══╝░░░╚═╝░░░░╚════╝░╚═╝░░░░░╚═╝╚══════╝
bentoml [OPTIONS] COMMAND [ARGS]...

Options

-v, --version#

Show the version and exit.

build#

Build a new Bento from current directory.

bentoml build [OPTIONS] [BUILD_CTX]

Options

-f, --bentofile <bentofile>#
--version <version>#
-q, --quiet#

Suppress all warnings and info logs

--verbose, --debug#

Generate debug information

--do-not-track#

Do not send usage info

Arguments

BUILD_CTX#

Optional argument

Environment variables

BENTOML_QUIET

Provide a default for -q

BENTOML_DEBUG

Provide a default for --verbose

BENTOML_DO_NOT_TRACK

Provide a default for --do-not-track

containerize#

Containerizes given Bento into a ready-to-use Docker image.

BENTO is the target BentoService to be containerized, referenced by its name
and version in format of name:version. For example: “iris_classifier:v1.2.0”

‘bentoml containerize’ command also supports the use of the ‘latest’ tag which will automatically use the last built version of your Bento.

You can provide a tag for the image built by Bento using the ‘–docker-image-tag’ flag. Additionally, you can provide a ‘–push’ flag, which will push the built image to the Docker repository specified by the image tag.

You can also prefixing the tag with a hostname for the repository you wish to push to. e.g. ‘bentoml containerize IrisClassifier:latest –push –tag repo-address.com:username/iris’ would build a Docker image called ‘username/iris:latest’ and push that to docker repository at repo-address.com.

By default, the ‘containerize’ command will use the current credentials provided by Docker daemon.

‘bentoml containerize’ also uses Docker Buildx as backend, in place for normal ‘docker build’. By doing so, BentoML will leverage Docker Buildx features such as multi-node builds for cross-platform images, Full BuildKit capabilities with all of the familiar UI from ‘docker build’.

bentoml containerize [OPTIONS] BENTO_TAG

Options

-t, --docker-image-tag <docker_image_tag>#

Name and optionally a tag (format: ‘name:tag’), defaults to bento tag.

--add-host <add_host>#

Add a custom host-to-IP mapping (format: ‘host:ip’).

--allow <allow>#

Allow extra privileged entitlement (e.g., ‘network.host’, ‘security.insecure’).

--build-arg <build_arg>#

Set build-time variables.

--build-context <build_context>#

Additional build contexts (e.g., name=path).

--builder <builder>#

Override the configured builder instance.

--cache-from <cache_from>#

External cache sources (e.g., ‘user/app:cache’, ‘type=local,src=path/to/dir’).

--cache-to <cache_to>#

Cache export destinations (e.g., ‘user/app:cache’, ‘type=local,dest=path/to/dir’).

--cgroup-parent <cgroup_parent>#

Optional parent cgroup for the container.

--iidfile <iidfile>#

Write the image ID to the file.

--label <label>#

Set metadata for an image.

--load#

Shorthand for ‘–output=type=docker’. Note that ‘–push’ and ‘–load’ are mutually exclusive.

--metadata-file <metadata_file>#

Write build result metadata to the file.

--network <network>#

Set the networking mode for the ‘RUN’ instructions during build (default ‘default’).

--no-cache#

Do not use cache when building the image.

--no-cache-filter <no_cache_filter>#

Do not cache specified stages.

--output <output>#

Output destination (format: ‘type=local,dest=path’).

--platform <platform>#

Set target platform for build.

--progress <progress>#

Set type of progress output (‘auto’, ‘plain’, ‘tty’). Use plain to show container output.

Options

auto | tty | plain

--pull#

Always attempt to pull all referenced images.

--push#

Shorthand for ‘–output=type=registry’. Note that ‘–push’ and ‘–load’ are mutually exclusive.

--secret <secret>#

Secret to expose to the build (format: ‘id=mysecret[,src=/local/secret]’).

--shm-size <shm_size>#

Size of ‘/dev/shm’.

--ssh <ssh>#

SSH agent socket or keys to expose to the build (format: ‘default|<id>[=<socket>|<key>[,<key>]]’).

--target <target>#

Set the target build stage to build.

--ulimit <ulimit>#

Ulimit options (default []).

--enable-features <features,>#

Enable additional BentoML features. Available features are: tracing, grpc, aws, all, io-image, io-pandas, tracing-zipkin, tracing-jaeger, tracing-otlp.

-q, --quiet#

Suppress all warnings and info logs

--verbose, --debug#

Generate debug information

--do-not-track#

Do not send usage info

Arguments

BENTO_TAG#

Required argument

Environment variables

BENTOML_QUIET

Provide a default for -q

BENTOML_DEBUG

Provide a default for --verbose

BENTOML_DO_NOT_TRACK

Provide a default for --do-not-track

delete#

Delete Bento in local bento store.

Examples:
* Delete single bento bundle by “name:version”, e.g: bentoml delete IrisClassifier:v1
* Bulk delete all bento bundles with a specific name, e.g.: bentoml delete IrisClassifier
* Bulk delete multiple bento bundles by name and version, separated by “,”, e.g.: benotml delete Irisclassifier:v1,MyPredictService:v2
* Bulk delete without confirmation, e.g.: bentoml delete IrisClassifier –yes
bentoml delete [OPTIONS] DELETE_TARGETS

Options

-y, --yes, --assume-yes#

Skip confirmation when deleting a specific bento bundle

-q, --quiet#

Suppress all warnings and info logs

--verbose, --debug#

Generate debug information

--do-not-track#

Do not send usage info

Arguments

DELETE_TARGETS#

Required argument

Environment variables

BENTOML_QUIET

Provide a default for -q

BENTOML_DEBUG

Provide a default for --verbose

BENTOML_DO_NOT_TRACK

Provide a default for --do-not-track

env#

Print environment info and exit

bentoml env [OPTIONS]

Options

-o, --output <output>#

Output format. ‘-o plain’ to display without format.

Default

md

Options

md | plain

-q, --quiet#

Suppress all warnings and info logs

--verbose, --debug#

Generate debug information

--do-not-track#

Do not send usage info

Environment variables

BENTOML_QUIET

Provide a default for -q

BENTOML_DEBUG

Provide a default for --verbose

BENTOML_DO_NOT_TRACK

Provide a default for --do-not-track

export#

Export a Bento to an external file archive

Arguments:
BENTO_TAG: bento identifier
OUT_PATH: output path of exported bento.

If out_path argument is not provided, bento is exported to name-version.bento in the current directory. Beside the native .bento format, we also support (‘tar’), tar.gz (‘gz’), tar.xz (‘xz’), tar.bz2 (‘bz2’), and zip.

Examples:
bentoml export FraudDetector:20210709_DE14C9
bentoml export FraudDetector:20210709_DE14C9 ./my_bento.bento
bentoml export FraudDetector:latest ./my_bento.bento
bentoml export FraudDetector:latest s3://mybucket/bentos/my_bento.bento
bentoml export [OPTIONS] BENTO_TAG [OUT_PATH]

Options

-q, --quiet#

Suppress all warnings and info logs

--verbose, --debug#

Generate debug information

--do-not-track#

Do not send usage info

Arguments

BENTO_TAG#

Required argument

OUT_PATH#

Optional argument

Environment variables

BENTOML_QUIET

Provide a default for -q

BENTOML_DEBUG

Provide a default for --verbose

BENTOML_DO_NOT_TRACK

Provide a default for --do-not-track

get#

Print Bento details by providing the bento_tag.

bentoml get iris_classifier:qojf5xauugwqtgxi
bentoml get iris_classifier:qojf5xauugwqtgxi –output=json
bentoml get [OPTIONS] BENTO_TAG

Options

-o, --output <output>#
Options

json | yaml | path

-q, --quiet#

Suppress all warnings and info logs

--verbose, --debug#

Generate debug information

--do-not-track#

Do not send usage info

Arguments

BENTO_TAG#

Required argument

Environment variables

BENTOML_QUIET

Provide a default for -q

BENTOML_DEBUG

Provide a default for --verbose

BENTOML_DO_NOT_TRACK

Provide a default for --do-not-track

import#

Import a previously exported Bento archive file

Arguments:
BENTO_PATH: path of Bento archive file
Examples:
bentoml import ./my_bento.bento
bentoml import s3://mybucket/bentos/my_bento.bento
bentoml import [OPTIONS] BENTO_PATH

Options

-q, --quiet#

Suppress all warnings and info logs

--verbose, --debug#

Generate debug information

--do-not-track#

Do not send usage info

Arguments

BENTO_PATH#

Required argument

Environment variables

BENTOML_QUIET

Provide a default for -q

BENTOML_DEBUG

Provide a default for --verbose

BENTOML_DO_NOT_TRACK

Provide a default for --do-not-track

list#

List Bentos in local store

# show all bentos saved
$ bentoml list
# show all verions of bento with the name FraudDetector
$ bentoml list FraudDetector
bentoml list [OPTIONS] [BENTO_NAME]

Options

-o, --output <output>#
Options

json | yaml | table

-q, --quiet#

Suppress all warnings and info logs

--verbose, --debug#

Generate debug information

--do-not-track#

Do not send usage info

Arguments

BENTO_NAME#

Optional argument

Environment variables

BENTOML_QUIET

Provide a default for -q

BENTOML_DEBUG

Provide a default for --verbose

BENTOML_DO_NOT_TRACK

Provide a default for --do-not-track

models#

Model Subcommands Groups

bentoml models [OPTIONS] COMMAND [ARGS]...

delete#

Delete Model in local model store.

Examples:
* Delete single model by “name:version”, e.g: bentoml models delete iris_clf:v1
* Bulk delete all models with a specific name, e.g.: bentoml models delete iris_clf
* Bulk delete multiple models by name and version, separated by “,”, e.g.: benotml models delete iris_clf:v1,iris_clf:v2
* Bulk delete without confirmation, e.g.: bentoml models delete IrisClassifier –yes
bentoml models delete [OPTIONS] DELETE_TARGETS

Options

-y, --yes, --assume-yes#

Skip confirmation when deleting a specific model

-q, --quiet#

Suppress all warnings and info logs

--verbose, --debug#

Generate debug information

--do-not-track#

Do not send usage info

Arguments

DELETE_TARGETS#

Required argument

Environment variables

BENTOML_QUIET

Provide a default for -q

BENTOML_DEBUG

Provide a default for --verbose

BENTOML_DO_NOT_TRACK

Provide a default for --do-not-track

export#

Export a Model to an external archive file

arguments:

MODEL_TAG: model identifier
OUT_PATH: output path of exported model.
If this argument is not provided, model is exported to name-version.bentomodel in the current directory.
Besides native .bentomodel format, we also support formats like tar(‘tar’), tar.gz (‘gz’), tar.xz (‘xz’), tar.bz2 (‘bz2’), and zip.

examples:

bentoml models export FraudDetector:latest
bentoml models export FraudDetector:latest ./my_model.bentomodel
bentoml models export FraudDetector:20210709_DE14C9 ./my_model.bentomodel
bentoml models export FraudDetector:20210709_DE14C9 s3://mybucket/models/my_model.bentomodel
bentoml models export [OPTIONS] MODEL_TAG [OUT_PATH]

Options

-q, --quiet#

Suppress all warnings and info logs

--verbose, --debug#

Generate debug information

--do-not-track#

Do not send usage info

Arguments

MODEL_TAG#

Required argument

OUT_PATH#

Optional argument

Environment variables

BENTOML_QUIET

Provide a default for -q

BENTOML_DEBUG

Provide a default for --verbose

BENTOML_DO_NOT_TRACK

Provide a default for --do-not-track

get#

Print Model details by providing the model_tag

bentoml get iris_clf:qojf5xauugwqtgxi
bentoml get iris_clf:qojf5xauugwqtgxi –output=json
bentoml models get [OPTIONS] MODEL_TAG

Options

-o, --output <output>#
Options

json | yaml | path

-q, --quiet#

Suppress all warnings and info logs

--verbose, --debug#

Generate debug information

--do-not-track#

Do not send usage info

Arguments

MODEL_TAG#

Required argument

Environment variables

BENTOML_QUIET

Provide a default for -q

BENTOML_DEBUG

Provide a default for --verbose

BENTOML_DO_NOT_TRACK

Provide a default for --do-not-track

import#

Import a previously exported Model archive file

bentoml models import ./my_model.bentomodel bentoml models import s3://mybucket/models/my_model.bentomodel

bentoml models import [OPTIONS] MODEL_PATH

Options

-q, --quiet#

Suppress all warnings and info logs

--verbose, --debug#

Generate debug information

--do-not-track#

Do not send usage info

Arguments

MODEL_PATH#

Required argument

Environment variables

BENTOML_QUIET

Provide a default for -q

BENTOML_DEBUG

Provide a default for --verbose

BENTOML_DO_NOT_TRACK

Provide a default for --do-not-track

list#

List Models in local store

# show all models saved
$ bentoml models list
# show all verions of bento with the name FraudDetector
$ bentoml models list FraudDetector
bentoml models list [OPTIONS] [MODEL_NAME]

Options

-o, --output <output>#
Options

json | yaml | table

-q, --quiet#

Suppress all warnings and info logs

--verbose, --debug#

Generate debug information

--do-not-track#

Do not send usage info

Arguments

MODEL_NAME#

Optional argument

Environment variables

BENTOML_QUIET

Provide a default for -q

BENTOML_DEBUG

Provide a default for --verbose

BENTOML_DO_NOT_TRACK

Provide a default for --do-not-track

pull#

Pull Model from a yatai server.

bentoml models pull [OPTIONS] MODEL_TAG

Options

-f, --force#

Force pull from yatai to local and overwrite even if it already exists in local

-q, --quiet#

Suppress all warnings and info logs

--verbose, --debug#

Generate debug information

--do-not-track#

Do not send usage info

Arguments

MODEL_TAG#

Required argument

Environment variables

BENTOML_QUIET

Provide a default for -q

BENTOML_DEBUG

Provide a default for --verbose

BENTOML_DO_NOT_TRACK

Provide a default for --do-not-track

push#

Push Model to a yatai server.

bentoml models push [OPTIONS] MODEL_TAG

Options

-f, --force#

Forced push to yatai even if it exists in yatai

-t, --threads <threads>#

Number of threads to use for upload

-q, --quiet#

Suppress all warnings and info logs

--verbose, --debug#

Generate debug information

--do-not-track#

Do not send usage info

Arguments

MODEL_TAG#

Required argument

Environment variables

BENTOML_QUIET

Provide a default for -q

BENTOML_DEBUG

Provide a default for --verbose

BENTOML_DO_NOT_TRACK

Provide a default for --do-not-track

pull#

Pull Bento from a yatai server.

bentoml pull [OPTIONS] BENTO_TAG

Options

-f, --force#

Force pull from yatai to local and overwrite even if it already exists in local

-q, --quiet#

Suppress all warnings and info logs

--verbose, --debug#

Generate debug information

--do-not-track#

Do not send usage info

Arguments

BENTO_TAG#

Required argument

Environment variables

BENTOML_QUIET

Provide a default for -q

BENTOML_DEBUG

Provide a default for --verbose

BENTOML_DO_NOT_TRACK

Provide a default for --do-not-track

push#

Push Bento to a yatai server.

bentoml push [OPTIONS] BENTO_TAG

Options

-f, --force#

Forced push to yatai even if it exists in yatai

-t, --threads <threads>#

Number of threads to use for upload

-q, --quiet#

Suppress all warnings and info logs

--verbose, --debug#

Generate debug information

--do-not-track#

Do not send usage info

Arguments

BENTO_TAG#

Required argument

Environment variables

BENTOML_QUIET

Provide a default for -q

BENTOML_DEBUG

Provide a default for --verbose

BENTOML_DO_NOT_TRACK

Provide a default for --do-not-track

serve#

Start a HTTP BentoServer from a given 🍱

BENTO is the serving target, it can be the import as:
- the import path of a ‘bentoml.Service’ instance
- a tag to a Bento in local Bento store
- a folder containing a valid ‘bentofile.yaml’ build file with a ‘service’ field, which provides the import path of a ‘bentoml.Service’ instance
- a path to a built Bento (for internal & debug use only)

e.g.:

Serve from a bentoml.Service instance source code (for development use only):
‘bentoml serve fraud_detector.py:svc’
Serve from a Bento built in local store:
‘bentoml serve fraud_detector:4tht2icroji6zput3suqi5nl2’
‘bentoml serve fraud_detector:latest’
Serve from a Bento directory:
‘bentoml serve ./fraud_detector_bento’
If ‘–reload’ is provided, BentoML will detect code and model store changes during development, and restarts the service automatically.
The ‘–reload’ flag will:
- be default, all file changes under ‘–working-dir’ (default to current directory) will trigger a restart
- when specified, respect ‘include’ and ‘exclude’ under ‘bentofile.yaml’ as well as the ‘.bentoignore’ file in ‘–working-dir’, for code and file changes
- all model store changes will also trigger a restart (new model saved or existing model removed)
bentoml serve [OPTIONS] [BENTO]

Options

--production#

Run the BentoServer in production mode

Default

False

-p, --port <port>#

The port to listen on for the REST api server

Default

3000

--host <host>#

The host to bind for the REST api server

Default

0.0.0.0

--api-workers <api_workers>#

Specify the number of API server workers to start. Default to number of available CPU cores in production mode

--backlog <backlog>#

The maximum number of pending connections.

Default

2048

--reload#

Reload Service when code changes detected, this is only available in development mode

Default

False

--working-dir <working_dir>#

When loading from source code, specify the directory to find the Service instance

Default

.

--ssl-certfile <ssl_certfile>#

SSL certificate file

--ssl-keyfile <ssl_keyfile>#

SSL key file

--ssl-keyfile-password <ssl_keyfile_password>#

SSL keyfile password

--ssl-version <ssl_version>#

SSL version to use (see stdlib ‘ssl’ module)

--ssl-cert-reqs <ssl_cert_reqs>#

Whether client certificate is required (see stdlib ‘ssl’ module)

--ssl-ca-certs <ssl_ca_certs>#

CA certificates file

--ssl-ciphers <ssl_ciphers>#

Ciphers to use (see stdlib ‘ssl’ module)

-q, --quiet#

Suppress all warnings and info logs

--verbose, --debug#

Generate debug information

--do-not-track#

Do not send usage info

Arguments

BENTO#

Optional argument

Environment variables

BENTOML_PORT

Provide a default for -p

BENTOML_HOST

Provide a default for --host

BENTOML_API_WORKERS

Provide a default for --api-workers

BENTOML_QUIET

Provide a default for -q

BENTOML_DEBUG

Provide a default for --verbose

BENTOML_DO_NOT_TRACK

Provide a default for --do-not-track

serve-grpc#

[EXPERIMENTAL] Start a gRPC BentoServer from a given 🍱

BENTO is the serving target, it can be the import as:
- the import path of a ‘bentoml.Service’ instance
- a tag to a Bento in local Bento store
- a folder containing a valid ‘bentofile.yaml’ build file with a ‘service’ field, which provides the import path of a ‘bentoml.Service’ instance
- a path to a built Bento (for internal & debug use only)

e.g.:

Serve from a bentoml.Service instance source code (for development use only):
‘bentoml serve-grpc fraud_detector.py:svc’
Serve from a Bento built in local store:
‘bentoml serve-grpc fraud_detector:4tht2icroji6zput3suqi5nl2’
‘bentoml serve-grpc fraud_detector:latest’
Serve from a Bento directory:
‘bentoml serve-grpc ./fraud_detector_bento’

If ‘–reload’ is provided, BentoML will detect code and model store changes during development, and restarts the service automatically.

The ‘–reload’ flag will:
- be default, all file changes under ‘–working-dir’ (default to current directory) will trigger a restart
- when specified, respect ‘include’ and ‘exclude’ under ‘bentofile.yaml’ as well as the ‘.bentoignore’ file in ‘–working-dir’, for code and file changes
- all model store changes will also trigger a restart (new model saved or existing model removed)
bentoml serve-grpc [OPTIONS] [BENTO]

Options

--production#

Run the BentoServer in production mode

Default

False

-p, --port <port>#

The port to listen on for the REST api server

Default

3000

--host <host>#

The host to bind for the gRPC server

Default

0.0.0.0

--api-workers <api_workers>#

Specify the number of API server workers to start. Default to number of available CPU cores in production mode

--reload#

Reload Service when code changes detected, this is only available in development mode

Default

False

--backlog <backlog>#

The maximum number of pending connections.

Default

2048

--working-dir <working_dir>#

When loading from source code, specify the directory to find the Service instance

Default

.

--enable-reflection#

Enable reflection.

Default

False

--max-concurrent-streams <max_concurrent_streams>#

Maximum number of concurrent incoming streams to allow on a http2 connection.

-q, --quiet#

Suppress all warnings and info logs

--verbose, --debug#

Generate debug information

--do-not-track#

Do not send usage info

Arguments

BENTO#

Optional argument

Environment variables

BENTOML_PORT

Provide a default for -p

BENTOML_HOST

Provide a default for --host

BENTOML_API_WORKERS

Provide a default for --api-workers

BENTOML_QUIET

Provide a default for -q

BENTOML_DEBUG

Provide a default for --verbose

BENTOML_DO_NOT_TRACK

Provide a default for --do-not-track

yatai#

Yatai Subcommands Groups

bentoml yatai [OPTIONS] COMMAND [ARGS]...

login#

Login to Yatai server.

bentoml yatai login [OPTIONS]

Options

--endpoint <endpoint>#

Yatai endpoint, i.e: https://yatai.com

--api-token <api_token>#

Yatai user API token

-q, --quiet#

Suppress all warnings and info logs

--verbose, --debug#

Generate debug information

--do-not-track#

Do not send usage info

Environment variables

BENTOML_QUIET

Provide a default for -q

BENTOML_DEBUG

Provide a default for --verbose

BENTOML_DO_NOT_TRACK

Provide a default for --do-not-track