BentoML CLI#
Note
BentoML CLI command itself also comes usage documentation. You can learn more from
running bentoml --help
. The --help
flag also applies to sub-commands
for viewing detailed usage of a command, e.g.: bentoml serve --help
.
bentoml#
bentoml [OPTIONS] COMMAND [ARGS]...
Options
- -v, --version#
Show the version and exit.
build#
Build a new Bento from current directory.
bentoml build [OPTIONS] [BUILD_CTX]
Options
- -f, --bentofile <bentofile>#
Path to bentofile. Default to ‘bentofile.yaml’
- --version <version>#
Bento version. By default the version will be generated.
- -o, --output <output>#
Output log format. ‘-o tag’ to display only bento tag.
- Default:
default
- Options:
tag | default
- -q, --quiet#
Suppress all warnings and info logs
- --verbose, --debug#
Generate debug information
- --do-not-track#
Do not send usage info
Arguments
- BUILD_CTX#
Optional argument
Environment variables
- BENTOML_QUIET
Provide a default for
-q
- BENTOML_DEBUG
Provide a default for
--verbose
- BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
containerize#
Containerizes given Bento into an OCI-compliant container, with any given OCI builder.
BENTO
is the target BentoService to be containerized, referenced by its name
and version in format of name:version. For example: iris_classifier:v1.2.0
bentoml containerize
command also supports the use of the latest
tag--image-tag
flag.$ bentoml containerize iris_classifier:latest -t repo-address.com:username/iris
This would build an image called username/iris:latest
and push it to docker repository at repo-address.com
.
containerize
command will use the current credentials provided by each backend implementation.bentoml containerize
also leverage BuildKit features such as multi-nodeDOCKER_BUILDKIT=0
:$ DOCKER_BUILDKIT=0 bentoml containerize iris_classifier:latest
bentoml containerize
is backend-agnostic, meaning that it can use any OCI-compliant builder to build the container image.docker
backend, which will use the local Docker daemon to build the image.--backend
flag to specify a different backend. Note that backend flags will now be unified via --opt
.--opt
accepts multiple flags, and it works with all backend options.
--no-cache
), use --opt no-cache
.--build-arg FOO=BAR
), use --opt build-arg=FOO=BAR
or --opt build-arg:FOO=BAR
. Make sure to quote the value if it contains spaces.--cgroup-parent cgroupv2
), use --opt cgroup-parent=cgroupv2
or --opt cgroup-parent:cgroupv2
.bentoml containerize [OPTIONS] BENTO:TAG
Options
- -t, --image-tag, --docker-image-tag <NAME:TAG>#
Name and optionally a tag (format:
name:tag
) for building container, defaults to the bento tag.
- --backend <backend>#
Define builder backend. Available:
podman
,buildah
,nerdctl
,buildctl
,docker
,buildx
- --enable-features <FEATURE[,FEATURE>#
Enable additional BentoML features. Available features are:
grpc-channelz
,tracing
,io-image
,grpc
,io-file
,tracing-jaeger
,io-pandas
,aws
,monitor-otlp
,triton
,all
,io-json
,tracing-otlp
,io
,tracing-zipkin
,grpc-reflection
.
- --opt <ARG=VALUE[,ARG=VALUE>#
Define options for given backend. (format:
--opt target=foo --opt build-arg=foo=BAR --opt iidfile:/path/to/file --opt no-cache
)
- --run-as-root#
Whether to run backend with as root.
- --add-host <HOST:IP,>#
(Equivalent to
--opt add-host=host:ip
): Add a custom host-to-IP mapping.
- --build-arg <KEY=VALUE,>#
(Equivalent to
--opt build-arg=FOO=bar
): Set build-time variables.
- --cache-from <NAME|type=TYPE[,KEY=VALUE>#
(Equivalent to
--opt cache-from=user/app:cache
): External cache sources (e.g.:type=local,src=path/to/dir
).
- --iidfile <PATH>#
(Equivalent to
--opt iidfile=/path/to/file
): Write the image ID to the file.
- --label <NAME|[KEY=VALUE],>#
(Equivalent to
--opt label=io.container.capabilities=baz
): Set metadata for an image. This is equivalent to LABEL in given Dockerfile.
- --network <default|VALUE>#
(Equivalent to
--opt network=default
): Set the networking mode for theRUN
instructions during build (defaultdefault
).
- --no-cache#
(Equivalent to
--opt no-cache
): Do not use cache when building the image.
- --output <PATH,-,type=TYPE[,KEY=VALUE>#
(Equivalent to
--opt output=type=local,dest=/path/to/dir
): Output destination.
- --platform <VALUE[,VALUE>#
(Equivalent to
--opt platform=linux/amd64,linux/arm/v7
): Set target platform for build.
- --progress <VALUE>#
(Equivalent to
--opt progress=auto
): Set type of progress output. Use plain to show container output.- Options:
auto | tty | plain
- --pull#
(Equivalent to
--opt pull
): Always attempt to pull all referenced images.
- --secret <type=TYPE[,KEY=VALUE>#
(Equivalent to
--opt secret=id=aws,src=$HOME/.aws/crendentials
): Secret to expose to the build.
- --ssh <default|<id>[=<socket>|<key>[,<key>#
(Equivalent to
--opt ssh=default=$SSH_AUTH_SOCK
): SSH agent socket or keys to expose to the build. See https://docs.docker.com/engine/reference/commandline/buildx_build/#ssh
- --target <VALUE>#
(Equivalent to
--opt target=release-stage
): Set the target build stage to build.
- --allow <ENTITLEMENT>#
(Equivalent to
--opt allow=network.host
): Allow extra privileged entitlement (e.g.,network.host
,security.insecure
).
- --build-context <name=VALUE>#
(Equivalent to
--opt build-context=project=path/to/project/source
): Additional build contexts (e.g.,name=path
).
- --builder <builder>#
(Equivalent to
--opt builder=default
): Override the configured builder instance. Same asdocker buildx --builder
- --cache-to <NAME|type=TYPE[,KEY=VALUE>#
(Equivalent to
--opt cache-to=type=registry,ref=user/app
): Cache export destinations (e.g.,user/app:cache
,type=local,dest=path/to/dir
).
- --load#
(Equivalent to
--opt load
): Shorthand for--output=type=docker
. Note that--push
and--load
are mutually exclusive.
- --push#
(Equivalent to
--opt push
): Shorthand for--output=type=registry
. Note that--push
and--load
are mutually exclusive.
- --quiet#
(Equivalent to
--opt quiet
): Suppress the build output and print image ID on success
- --cgroup-parent <cgroup_parent>#
(Equivalent to
--opt cgroup-parent=cgroupv2
): Optional parent cgroup for the container.
- --ulimit <type>=<soft limit>[:<hard limit>#
(Equivalent to
--opt ulimit=nofile=1024:1024
): Ulimit options.
- --no-cache-filter <no_cache_filter>#
(Equivalent to
--opt no-cache-filter
): Do not cache specified stages.
- --metadata-file <metadata_file>#
(Equivalent to
--opt metadata-file=/path/to/file
): Write build result metadata to the file.
- --shm-size <shm_size>#
(Equivalent to
--opt shm-size=8192Mb
): Size of/dev/shm
.
- -q, --quiet#
Suppress all warnings and info logs
- --verbose, --debug#
Generate debug information
- --do-not-track#
Do not send usage info
Arguments
- BENTO:TAG#
Required argument
Environment variables
- BENTOML_CONTAINERIZE_BACKEND
Provide a default for
--backend
- BENTOML_QUIET
Provide a default for
-q
- BENTOML_DEBUG
Provide a default for
--verbose
- BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
delete#
Delete Bento in local bento store.
bentoml delete [OPTIONS] DELETE_TARGETS...
Options
- -y, --yes, --assume-yes#
Skip confirmation when deleting a specific bento bundle
- -q, --quiet#
Suppress all warnings and info logs
- --verbose, --debug#
Generate debug information
- --do-not-track#
Do not send usage info
Arguments
- DELETE_TARGETS#
Required argument(s)
Environment variables
- BENTOML_QUIET
Provide a default for
-q
- BENTOML_DEBUG
Provide a default for
--verbose
- BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
env#
Print environment info and exit
bentoml env [OPTIONS]
Options
- -o, --output <output>#
Output format. ‘-o bash’ to display without format.
- Default:
md
- Options:
md | bash
- -q, --quiet#
Suppress all warnings and info logs
- --verbose, --debug#
Generate debug information
- --do-not-track#
Do not send usage info
Environment variables
- BENTOML_QUIET
Provide a default for
-q
- BENTOML_DEBUG
Provide a default for
--verbose
- BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
export#
Export a Bento to an external file archive
If out_path argument is not provided, bento is exported to name-version.bento in the current directory. Beside the native .bento format, we also support (‘tar’), tar.gz (‘gz’), tar.xz (‘xz’), tar.bz2 (‘bz2’), and zip.
bentoml export [OPTIONS] BENTO_TAG [OUT_PATH]
Options
- -q, --quiet#
Suppress all warnings and info logs
- --verbose, --debug#
Generate debug information
- --do-not-track#
Do not send usage info
Arguments
- BENTO_TAG#
Required argument
- OUT_PATH#
Optional argument
Environment variables
- BENTOML_QUIET
Provide a default for
-q
- BENTOML_DEBUG
Provide a default for
--verbose
- BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
get#
Print Bento details by providing the bento_tag.
bentoml get [OPTIONS] BENTO_TAG
Options
- -o, --output <output>#
- Options:
json | yaml | path
- -q, --quiet#
Suppress all warnings and info logs
- --verbose, --debug#
Generate debug information
- --do-not-track#
Do not send usage info
Arguments
- BENTO_TAG#
Required argument
Environment variables
- BENTOML_QUIET
Provide a default for
-q
- BENTOML_DEBUG
Provide a default for
--verbose
- BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
import#
Import a previously exported Bento archive file
bentoml import [OPTIONS] BENTO_PATH
Options
- -q, --quiet#
Suppress all warnings and info logs
- --verbose, --debug#
Generate debug information
- --do-not-track#
Do not send usage info
Arguments
- BENTO_PATH#
Required argument
Environment variables
- BENTOML_QUIET
Provide a default for
-q
- BENTOML_DEBUG
Provide a default for
--verbose
- BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
list#
List Bentos in local store
bentoml list [OPTIONS] [BENTO_NAME]
Options
- -o, --output <output>#
- Options:
json | yaml | table
- -q, --quiet#
Suppress all warnings and info logs
- --verbose, --debug#
Generate debug information
- --do-not-track#
Do not send usage info
Arguments
- BENTO_NAME#
Optional argument
Environment variables
- BENTOML_QUIET
Provide a default for
-q
- BENTOML_DEBUG
Provide a default for
--verbose
- BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
models#
Model Subcommands Groups
bentoml models [OPTIONS] COMMAND [ARGS]...
delete#
Delete Model in local model store.
bentoml models delete [OPTIONS] DELETE_TARGETS...
Options
- -y, --yes, --assume-yes#
Skip confirmation when deleting a specific model
- -q, --quiet#
Suppress all warnings and info logs
- --verbose, --debug#
Generate debug information
- --do-not-track#
Do not send usage info
Arguments
- DELETE_TARGETS#
Required argument(s)
Environment variables
- BENTOML_QUIET
Provide a default for
-q
- BENTOML_DEBUG
Provide a default for
--verbose
- BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
export#
Export a Model to an external archive file
arguments:
examples:
bentoml models export [OPTIONS] MODEL_TAG [OUT_PATH]
Options
- -q, --quiet#
Suppress all warnings and info logs
- --verbose, --debug#
Generate debug information
- --do-not-track#
Do not send usage info
Arguments
- MODEL_TAG#
Required argument
- OUT_PATH#
Optional argument
Environment variables
- BENTOML_QUIET
Provide a default for
-q
- BENTOML_DEBUG
Provide a default for
--verbose
- BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
get#
Print Model details by providing the model_tag
bentoml models get [OPTIONS] MODEL_TAG
Options
- -o, --output <output>#
- Options:
json | yaml | path
- -q, --quiet#
Suppress all warnings and info logs
- --verbose, --debug#
Generate debug information
- --do-not-track#
Do not send usage info
Arguments
- MODEL_TAG#
Required argument
Environment variables
- BENTOML_QUIET
Provide a default for
-q
- BENTOML_DEBUG
Provide a default for
--verbose
- BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
import#
Import a previously exported Model archive file
bentoml models import ./my_model.bentomodel bentoml models import s3://mybucket/models/my_model.bentomodel
bentoml models import [OPTIONS] MODEL_PATH
Options
- -q, --quiet#
Suppress all warnings and info logs
- --verbose, --debug#
Generate debug information
- --do-not-track#
Do not send usage info
Arguments
- MODEL_PATH#
Required argument
Environment variables
- BENTOML_QUIET
Provide a default for
-q
- BENTOML_DEBUG
Provide a default for
--verbose
- BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
list#
List Models in local store
bentoml models list [OPTIONS] [MODEL_NAME]
Options
- -o, --output <output>#
- Options:
json | yaml | table
- -q, --quiet#
Suppress all warnings and info logs
- --verbose, --debug#
Generate debug information
- --do-not-track#
Do not send usage info
Arguments
- MODEL_NAME#
Optional argument
Environment variables
- BENTOML_QUIET
Provide a default for
-q
- BENTOML_DEBUG
Provide a default for
--verbose
- BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
pull#
Pull Model from a yatai server.
bentoml models pull [OPTIONS] MODEL_TAG
Options
- -f, --force#
Force pull from yatai to local and overwrite even if it already exists in local
- --context <context>#
Yatai context name.
- -q, --quiet#
Suppress all warnings and info logs
- --verbose, --debug#
Generate debug information
- --do-not-track#
Do not send usage info
Arguments
- MODEL_TAG#
Required argument
Environment variables
- BENTOML_QUIET
Provide a default for
-q
- BENTOML_DEBUG
Provide a default for
--verbose
- BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
push#
Push Model to a yatai server.
bentoml models push [OPTIONS] MODEL_TAG
Options
- -f, --force#
Forced push to yatai even if it exists in yatai
- -t, --threads <threads>#
Number of threads to use for upload
- --context <context>#
Yatai context name.
- -q, --quiet#
Suppress all warnings and info logs
- --verbose, --debug#
Generate debug information
- --do-not-track#
Do not send usage info
Arguments
- MODEL_TAG#
Required argument
Environment variables
- BENTOML_QUIET
Provide a default for
-q
- BENTOML_DEBUG
Provide a default for
--verbose
- BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
pull#
Pull Bento from a yatai server.
bentoml pull [OPTIONS] BENTO_TAG
Options
- -f, --force#
Force pull from yatai to local and overwrite even if it already exists in local
- --context <context>#
Yatai context name.
- -q, --quiet#
Suppress all warnings and info logs
- --verbose, --debug#
Generate debug information
- --do-not-track#
Do not send usage info
Arguments
- BENTO_TAG#
Required argument
Environment variables
- BENTOML_QUIET
Provide a default for
-q
- BENTOML_DEBUG
Provide a default for
--verbose
- BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
push#
Push Bento to a yatai server.
bentoml push [OPTIONS] BENTO_TAG
Options
- -f, --force#
Forced push to yatai even if it exists in yatai
- -t, --threads <threads>#
Number of threads to use for upload
- --context <context>#
Yatai context name.
- -q, --quiet#
Suppress all warnings and info logs
- --verbose, --debug#
Generate debug information
- --do-not-track#
Do not send usage info
Arguments
- BENTO_TAG#
Required argument
Environment variables
- BENTOML_QUIET
Provide a default for
-q
- BENTOML_DEBUG
Provide a default for
--verbose
- BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
serve#
Start a HTTP BentoServer from a given 🍱
e.g.:
bentoml serve [OPTIONS] [BENTO]
Options
- --development#
Run the BentoServer in development mode
- Default:
False
- --production#
Run BentoServer in production mode (Deprecated)
- -p, --port <port>#
The port to listen on for the REST api server
- Default:
3000
- --host <host>#
The host to bind for the REST api server
- Default:
0.0.0.0
- --api-workers <api_workers>#
Specify the number of API server workers to start. Default to number of available CPU cores in production mode
- Default:
2
- --backlog <backlog>#
The maximum number of pending connections.
- Default:
2048
- --reload#
Reload Service when code changes detected
- Default:
False
- --working-dir <working_dir>#
When loading from source code, specify the directory to find the Service instance
- --ssl-certfile <ssl_certfile>#
SSL certificate file
- --ssl-keyfile <ssl_keyfile>#
SSL key file
- --ssl-keyfile-password <ssl_keyfile_password>#
SSL keyfile password
- --ssl-version <ssl_version>#
SSL version to use (see stdlib ‘ssl’ module)
- Default:
17
- --ssl-cert-reqs <ssl_cert_reqs>#
Whether client certificate is required (see stdlib ‘ssl’ module)
- Default:
0
- --ssl-ca-certs <ssl_ca_certs>#
CA certificates file
- --ssl-ciphers <ssl_ciphers>#
Ciphers to use (see stdlib ‘ssl’ module)
- Default:
TLSv1
- --env <env>#
Environment to run the command in
- Options:
conda
- -q, --quiet#
Suppress all warnings and info logs
- --verbose, --debug#
Generate debug information
- --do-not-track#
Do not send usage info
Arguments
- BENTO#
Optional argument
Environment variables
- BENTOML_PORT
Provide a default for
-p
- BENTOML_HOST
Provide a default for
--host
- BENTOML_API_WORKERS
Provide a default for
--api-workers
- BENTOML_QUIET
Provide a default for
-q
- BENTOML_DEBUG
Provide a default for
--verbose
- BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
serve-grpc#
[EXPERIMENTAL] Start a gRPC BentoServer from a given 🍱
e.g.:
If ‘–reload’ is provided, BentoML will detect code and model store changes during development, and restarts the service automatically.
bentoml serve-grpc [OPTIONS] [BENTO]
Options
- --development#
Run the BentoServer in development mode
- Default:
False
- --production#
Run BentoServer in production mode (Deprecated)
- -p, --port <port>#
The port to listen on for the REST api server
- Default:
3000
- --host <host>#
The host to bind for the gRPC server
- Default:
0.0.0.0
- --api-workers <api_workers>#
Specify the number of API server workers to start. Default to number of available CPU cores in production mode
- Default:
2
- --reload#
Reload Service when code changes detected
- Default:
False
- --backlog <backlog>#
The maximum number of pending connections.
- Default:
2048
- --working-dir <working_dir>#
When loading from source code, specify the directory to find the Service instance
- --enable-reflection#
Enable reflection.
- Default:
False
- --enable-channelz#
Enable Channelz. See https://github.com/grpc/proposal/blob/master/A14-channelz.md.
- --max-concurrent-streams <max_concurrent_streams>#
Maximum number of concurrent incoming streams to allow on a http2 connection.
- --ssl-certfile <ssl_certfile>#
SSL certificate file
- --ssl-keyfile <ssl_keyfile>#
SSL key file
- --ssl-ca-certs <ssl_ca_certs>#
CA certificates file
- -pv, --protocol-version <protocol_version>#
Determine the version of generated gRPC stubs to use.
- Default:
v1
- Options:
v1 | v1alpha1
- --env <env>#
Environment to run the command in
- Options:
conda
- -q, --quiet#
Suppress all warnings and info logs
- --verbose, --debug#
Generate debug information
- --do-not-track#
Do not send usage info
Arguments
- BENTO#
Optional argument
Environment variables
- BENTOML_PORT
Provide a default for
-p
- BENTOML_HOST
Provide a default for
--host
- BENTOML_API_WORKERS
Provide a default for
--api-workers
- BENTOML_QUIET
Provide a default for
-q
- BENTOML_DEBUG
Provide a default for
--verbose
- BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track
yatai#
Yatai Subcommands Groups
bentoml yatai [OPTIONS] COMMAND [ARGS]...
login#
Login to Yatai server.
bentoml yatai login [OPTIONS]
Options
- --endpoint <endpoint>#
Yatai endpoint, i.e: https://yatai.com
- --api-token <api_token>#
Yatai user API token
- --context <context>#
Yatai context name for the endpoint and API token
- -q, --quiet#
Suppress all warnings and info logs
- --verbose, --debug#
Generate debug information
- --do-not-track#
Do not send usage info
Environment variables
- BENTOML_QUIET
Provide a default for
-q
- BENTOML_DEBUG
Provide a default for
--verbose
- BENTOML_DO_NOT_TRACK
Provide a default for
--do-not-track