BentoML CLI#

Note

BentoML CLI command itself also comes usage documentation. You can learn more from running bentoml --help. The --help flag also applies to sub-commands for viewing detailed usage of a command, e.g.: bentoml serve --help.

bentoml#

██████╗ ███████╗███╗   ██╗████████╗ ██████╗ ███╗   ███╗██╗
██╔══██╗██╔════╝████╗  ██║╚══██╔══╝██╔═══██╗████╗ ████║██║
██████╔╝█████╗  ██╔██╗ ██║   ██║   ██║   ██║██╔████╔██║██║
██╔══██╗██╔══╝  ██║╚██╗██║   ██║   ██║   ██║██║╚██╔╝██║██║
██████╔╝███████╗██║ ╚████║   ██║   ╚██████╔╝██║ ╚═╝ ██║███████╗
╚═════╝ ╚══════╝╚═╝  ╚═══╝   ╚═╝    ╚═════╝ ╚═╝     ╚═╝╚══════╝

bentoml [OPTIONS] COMMAND [ARGS]...

Options

-v, --version#: Show the version and exit.

build#

Build a new Bento from current directory.

bentoml build [OPTIONS] [BUILD_CTX]

Options

-f, --bentofile <bentofile>#: Path to bentofile. Default to ‘bentofile.yaml’

--version <version>#: Bento version. By default the version will be generated.

-o, --output <output>#

Output log format. ‘-o tag’ to display only bento tag.

Default:: default
Options:: tag | default

--containerize#: Whether to containerize the Bento after building. ‘–containerize’ is the shortcut of ‘bentoml build && bentoml containerize’.

--push#: Whether to push the result bento to BentoCloud. Make sure to login with ‘bentoml cloud login’ first.

--force#: Forced push to BentoCloud

--threads <threads>#: Number of threads to use for upload

-q, --quiet#: Suppress all warnings and info logs

--verbose, --debug#: Generate debug information

--do-not-track#: Do not send usage info

--context <cloud_context>#: BentoCloud context name.

Arguments

BUILD_CTX#: Optional argument

Environment variables

BENTOML_QUIET: Provide a default for -q

BENTOML_DEBUG: Provide a default for --verbose

BENTOML_DO_NOT_TRACK: Provide a default for --do-not-track

cloud#

BentoCloud Subcommands Groups

Note that ‘bentoml yatai’ is mainly for backward compatible reason. It is recommended

to use ‘bentoml cloud’.

bentoml cloud [OPTIONS] COMMAND [ARGS]...

current-context#

Get current cloud context.

bentoml cloud current-context [OPTIONS]

Options

-q, --quiet#: Suppress all warnings and info logs

--verbose, --debug#: Generate debug information

--do-not-track#: Do not send usage info

--context <cloud_context>#: BentoCloud context name.

Environment variables

BENTOML_QUIET: Provide a default for -q

BENTOML_DEBUG: Provide a default for --verbose

BENTOML_DO_NOT_TRACK: Provide a default for --do-not-track

list-context#

List all available context.

bentoml cloud list-context [OPTIONS]

Options

-q, --quiet#: Suppress all warnings and info logs

--verbose, --debug#: Generate debug information

--do-not-track#: Do not send usage info

--context <cloud_context>#: BentoCloud context name.

Environment variables

BENTOML_QUIET: Provide a default for -q

BENTOML_DEBUG: Provide a default for --verbose

BENTOML_DO_NOT_TRACK: Provide a default for --do-not-track

login#

bentoml cloud login [OPTIONS]

Options

--endpoint <endpoint>#: BentoCloud or Yatai endpoint, i.e: https://cloud.bentoml.com

--api-token <api_token>#: BentoCloud or Yatai user API token

-q, --quiet#: Suppress all warnings and info logs

--verbose, --debug#: Generate debug information

--do-not-track#: Do not send usage info

--context <cloud_context>#: BentoCloud context name.

Environment variables

BENTOML_QUIET: Provide a default for -q

BENTOML_DEBUG: Provide a default for --verbose

BENTOML_DO_NOT_TRACK: Provide a default for --do-not-track

update-current-context#

Update current context

bentoml cloud update-current-context [OPTIONS] CONTEXT_NAME

Options

-q, --quiet#: Suppress all warnings and info logs

--verbose, --debug#: Generate debug information

--do-not-track#: Do not send usage info

--context <cloud_context>#: BentoCloud context name.

Arguments

CONTEXT_NAME#: Required argument

Environment variables

BENTOML_QUIET: Provide a default for -q

BENTOML_DEBUG: Provide a default for --verbose

BENTOML_DO_NOT_TRACK: Provide a default for --do-not-track

containerize#

Containerizes given Bento into an OCI-compliant container, with any given OCI builder.

BENTO is the target BentoService to be containerized, referenced by its name and version in format of name:version. For example: iris_classifier:v1.2.0

bentoml containerize command also supports the use of the latest tag

which will automatically use the last built version of your Bento.

You can provide a tag for the image built by Bento using the

--image-tag flag.

You can also prefixing the tag with a hostname for the repository you wish to push to.

$ bentoml containerize iris_classifier:latest -t repo-address.com:username/iris

This would build an image called username/iris:latest and push it to docker repository at repo-address.com.

By default, the containerize command will use the current credentials provided by each backend implementation.

bentoml containerize also leverage BuildKit features such as multi-node
builds for cross-platform images, more efficient caching, and more secure builds.
BuildKit will be enabled by default by each backend implementation. To opt-out of BuildKit, set DOCKER_BUILDKIT=0:

$ DOCKER_BUILDKIT=0 bentoml containerize iris_classifier:latest

bentoml containerize is backend-agnostic, meaning that it can use any OCI-compliant builder to build the container image.
By default, it will use the docker backend, which will use the local Docker daemon to build the image.
You can use the --backend flag to specify a different backend. Note that backend flags will now be unified via --opt.

--opt accepts multiple flags, and it works with all backend options.

To pass a boolean flag (e.g: --no-cache), use --opt no-cache.

To pass a key-value pair (e.g: --build-arg FOO=BAR), use --opt build-arg=FOO=BAR or --opt build-arg:FOO=BAR. Make sure to quote the value if it contains spaces.

To pass a argument with one value (path, integer, etc) (e.g: --cgroup-parent cgroupv2), use --opt cgroup-parent=cgroupv2 or --opt cgroup-parent:cgroupv2.

bentoml containerize [OPTIONS] BENTO:TAG

Options

-t, --image-tag, --docker-image-tag <NAME:TAG>#: Name and optionally a tag (format: name:tag) for building container, defaults to the bento tag.

--backend <backend>#: Define builder backend. Available: podman, buildah, docker, buildctl, nerdctl, buildx

--enable-features <FEATURE[,FEATURE>#: Enable additional BentoML features. Available features are: tracing, all, io-image, io, grpc, grpc-reflection, aws, io-pandas, io-file, grpc-channelz, io-json, tracing-zipkin, triton, tracing-otlp, tracing-jaeger, monitor-otlp.

--opt <ARG=VALUE[,ARG=VALUE>#: Define options for given backend. (format: --opt target=foo --opt build-arg=foo=BAR --opt iidfile:/path/to/file --opt no-cache)

--run-as-root#: Whether to run backend with as root.

--add-host <HOST:IP,>#: (Equivalent to --opt add-host=host:ip): Add a custom host-to-IP mapping.

--build-arg <KEY=VALUE,>#: (Equivalent to --opt build-arg=FOO=bar): Set build-time variables.

--cache-from <NAME|type=TYPE[,KEY=VALUE>#: (Equivalent to --opt cache-from=user/app:cache): External cache sources (e.g.: type=local,src=path/to/dir).

--iidfile <PATH>#: (Equivalent to --opt iidfile=/path/to/file): Write the image ID to the file.

--label <NAME|[KEY=VALUE],>#: (Equivalent to --opt label=io.container.capabilities=baz): Set metadata for an image. This is equivalent to LABEL in given Dockerfile.

--network <default|VALUE>#: (Equivalent to --opt network=default): Set the networking mode for the RUN instructions during build (default default).

--no-cache#: (Equivalent to --opt no-cache): Do not use cache when building the image.

--output <PATH,-,type=TYPE[,KEY=VALUE>#: (Equivalent to --opt output=type=local,dest=/path/to/dir): Output destination.

--platform <VALUE[,VALUE>#: (Equivalent to --opt platform=linux/amd64,linux/arm/v7): Set target platform for build.

--progress <VALUE>#

(Equivalent to --opt progress=auto): Set type of progress output. Use plain to show container output.

Options:: auto | tty | plain

--pull#: (Equivalent to --opt pull): Always attempt to pull all referenced images.

--secret <type=TYPE[,KEY=VALUE>#: (Equivalent to --opt secret=id=aws,src=$HOME/.aws/crendentials): Secret to expose to the build.

--ssh <default|<id>[=<socket>|<key>[,<key>#: (Equivalent to --opt ssh=default=$SSH_AUTH_SOCK): SSH agent socket or keys to expose to the build. See https://docs.docker.com/engine/reference/commandline/buildx_build/#ssh

--target <VALUE>#: (Equivalent to --opt target=release-stage): Set the target build stage to build.

--allow <ENTITLEMENT>#: (Equivalent to --opt allow=network.host): Allow extra privileged entitlement (e.g., network.host, security.insecure).

--attest <NAME|type=TYPE[,KEY=VALUE>#: (Equivalent to --opt attest=type=sbom,generator=image): Attestation parameter (e.g., type=local,ref=path/to/dir).

--build-context <name=VALUE>#: (Equivalent to --opt build-context=project=path/to/project/source): Additional build contexts (e.g., name=path).

--builder <builder>#: (Equivalent to --opt builder=default): Override the configured builder instance. Same as docker buildx --builder

--cache-to <NAME|type=TYPE[,KEY=VALUE>#: (Equivalent to --opt cache-to=type=registry,ref=user/app): Cache export destinations (e.g., user/app:cache, type=local,dest=path/to/dir).

--load#: (Equivalent to --opt load): Shorthand for --output=type=docker. Note that --push and --load are mutually exclusive.

--push#: (Equivalent to --opt push): Shorthand for --output=type=registry. Note that --push and --load are mutually exclusive.

--provenance <provenance>#: (Equivalent to --opt provenance=generator=image): Shorthand for --attest=type=provenance.

--quiet#: (Equivalent to --opt quiet): Suppress the build output and print image ID on success

--cgroup-parent <cgroup_parent>#: (Equivalent to --opt cgroup-parent=cgroupv2): Optional parent cgroup for the container.

--ulimit <type>=<soft limit>[:<hard limit>#: (Equivalent to --opt ulimit=nofile=1024:1024): Ulimit options.

--no-cache-filter <no_cache_filter>#: (Equivalent to --opt no-cache-filter): Do not cache specified stages.

--metadata-file <metadata_file>#: (Equivalent to --opt metadata-file=/path/to/file): Write build result metadata to the file.

--sbom <sbom>#: (Equivalent to --opt sbom=generator=image): Shorthand for --attest=type=sbom.

--shm-size <shm_size>#: (Equivalent to --opt shm-size=8192Mb): Size of /dev/shm.

-q, --quiet#: Suppress all warnings and info logs

--verbose, --debug#: Generate debug information

--do-not-track#: Do not send usage info

--context <cloud_context>#: BentoCloud context name.

Arguments

BENTO:TAG#: Required argument

Environment variables

BENTOML_CONTAINERIZE_BACKEND: Provide a default for --backend

BENTOML_QUIET: Provide a default for -q

BENTOML_DEBUG: Provide a default for --verbose

BENTOML_DO_NOT_TRACK: Provide a default for --do-not-track

delete#

Delete Bento in local bento store.

Examples:

* Delete single bento bundle by “name:version”, e.g: bentoml delete IrisClassifier:v1
* Bulk delete all bento bundles with a specific name, e.g.: bentoml delete IrisClassifier
* Bulk delete multiple bento bundles by name and version, separated by “,”, e.g.: bentoml delete Irisclassifier:v1,MyPredictService:v2
* Bulk delete multiple bento bundles by name and version, separated by “ “, e.g.: bentoml delete Irisclassifier:v1 MyPredictService:v2
* Bulk delete without confirmation, e.g.: bentoml delete IrisClassifier –yes

bentoml delete [OPTIONS] DELETE_TARGETS...

Options

-y, --yes, --assume-yes#: Skip confirmation when deleting a specific bento bundle

-q, --quiet#: Suppress all warnings and info logs

--verbose, --debug#: Generate debug information

--do-not-track#: Do not send usage info

--context <cloud_context>#: BentoCloud context name.

Arguments

DELETE_TARGETS#: Required argument(s)

Environment variables

BENTOML_QUIET: Provide a default for -q

BENTOML_DEBUG: Provide a default for --verbose

BENTOML_DO_NOT_TRACK: Provide a default for --do-not-track

deployment#

Deployment Subcommands Groups

bentoml deployment [OPTIONS] COMMAND [ARGS]...

create#

Create a deployment on BentoCloud.

A deployment can be created using a json file with configurations.

The json file has the exact format as the one on BentoCloud Deployment UI.

bentoml deployment create [OPTIONS]

Options

-f, --file <file>#: JSON file path for the deployment configuration

-o, --output <output>#

Display the output of this command.

Options:: json | default

-q, --quiet#: Suppress all warnings and info logs

--verbose, --debug#: Generate debug information

--do-not-track#: Do not send usage info

--context <cloud_context>#: BentoCloud context name.

Environment variables

BENTOML_QUIET: Provide a default for -q

BENTOML_DEBUG: Provide a default for --verbose

BENTOML_DO_NOT_TRACK: Provide a default for --do-not-track

delete#

Delete a deployment on BentoCloud.

bentoml deployment delete [OPTIONS] DEPLOYMENT_NAME

Options

--cluster-name <cluster_name>#: Name of the cluster.

--kube-namespace <kube_namespace>#: Kubernetes namespace.

-o, --output <output>#

Display the output of this command.

Options:: json | default

-q, --quiet#: Suppress all warnings and info logs

--verbose, --debug#: Generate debug information

--do-not-track#: Do not send usage info

--context <cloud_context>#: BentoCloud context name.

Arguments

DEPLOYMENT_NAME#: Required argument

Environment variables

BENTOML_QUIET: Provide a default for -q

BENTOML_DEBUG: Provide a default for --verbose

BENTOML_DO_NOT_TRACK: Provide a default for --do-not-track

get#

Get a deployment on BentoCloud.

bentoml deployment get [OPTIONS] DEPLOYMENT_NAME

Options

--cluster-name <cluster_name>#: Name of the cluster.

--kube-namespace <kube_namespace>#: Kubernetes namespace.

-o, --output <output>#

Display the output of this command.

Options:: json | default

-q, --quiet#: Suppress all warnings and info logs

--verbose, --debug#: Generate debug information

--do-not-track#: Do not send usage info

--context <cloud_context>#: BentoCloud context name.

Arguments

DEPLOYMENT_NAME#: Required argument

Environment variables

BENTOML_QUIET: Provide a default for -q

BENTOML_DEBUG: Provide a default for --verbose

BENTOML_DO_NOT_TRACK: Provide a default for --do-not-track

list#

List existing deployments on BentoCloud.

bentoml deployment list [OPTIONS]

Options

--cluster-name <cluster_name>#: Name of the cluster.

--query <query>#: Query for list request.

--search <search>#: Search for list request.

--start <start>#: Start for list request.

--count <count>#: Count for list request.

-o, --output <output>#

Display the output of this command.

Options:: json | default | table

-q, --quiet#: Suppress all warnings and info logs

--verbose, --debug#: Generate debug information

--do-not-track#: Do not send usage info

--context <cloud_context>#: BentoCloud context name.

Environment variables

BENTOML_QUIET: Provide a default for -q

BENTOML_DEBUG: Provide a default for --verbose

BENTOML_DO_NOT_TRACK: Provide a default for --do-not-track

terminate#

Terminate a deployment on BentoCloud.

bentoml deployment terminate [OPTIONS] DEPLOYMENT_NAME

Options

--cluster-name <cluster_name>#: Name of the cluster.

--kube-namespace <kube_namespace>#: Kubernetes namespace.

-o, --output <output>#

Display the output of this command.

Options:: json | default

-q, --quiet#: Suppress all warnings and info logs

--verbose, --debug#: Generate debug information

--do-not-track#: Do not send usage info

--context <cloud_context>#: BentoCloud context name.

Arguments

DEPLOYMENT_NAME#: Required argument

Environment variables

BENTOML_QUIET: Provide a default for -q

BENTOML_DEBUG: Provide a default for --verbose

BENTOML_DO_NOT_TRACK: Provide a default for --do-not-track

update#

Update a deployment on BentoCloud.

A deployment can be updated using a json file with needed configurations.

The json file has the exact format as the one on BentoCloud Deployment UI.

bentoml deployment update [OPTIONS] [DEPLOYMENT_NAME]

Options

--cluster-name <cluster_name>#: Name of the cluster.

--kube-namespace <kube_namespace>#: Kubernetes namespace.

-o, --output <output>#

Display the output of this command.

Options:: json | default

-f, --file <file>#: JSON file path for the deployment configuration

-n, --name <name>#: Deployment name (deprecated)

--bento <bento>#: Bento tag

-q, --quiet#: Suppress all warnings and info logs

--verbose, --debug#: Generate debug information

--do-not-track#: Do not send usage info

--context <cloud_context>#: BentoCloud context name.

Arguments

DEPLOYMENT_NAME#: Optional argument

Environment variables

BENTOML_QUIET: Provide a default for -q

BENTOML_DEBUG: Provide a default for --verbose

BENTOML_DO_NOT_TRACK: Provide a default for --do-not-track

env#

Print environment info and exit

bentoml env [OPTIONS]

Options

-o, --output <output>#

Output format. ‘-o bash’ to display without format.

Default:: md
Options:: md | bash

-q, --quiet#: Suppress all warnings and info logs

--verbose, --debug#: Generate debug information

--do-not-track#: Do not send usage info

--context <cloud_context>#: BentoCloud context name.

Environment variables

BENTOML_QUIET: Provide a default for -q

BENTOML_DEBUG: Provide a default for --verbose

BENTOML_DO_NOT_TRACK: Provide a default for --do-not-track

export#

Export a Bento to an external file archive

Arguments:

BENTO_TAG: bento identifier

OUT_PATH: output path of exported bento.

If out_path argument is not provided, bento is exported to name-version.bento in the current directory. Beside the native .bento format, we also support (‘tar’), tar.gz (‘gz’), tar.xz (‘xz’), tar.bz2 (‘bz2’), and zip.

Examples:

bentoml export FraudDetector:20210709_DE14C9
bentoml export FraudDetector:20210709_DE14C9 ./my_bento.bento
bentoml export FraudDetector:latest ./my_bento.bento
bentoml export FraudDetector:latest s3://mybucket/bentos/my_bento.bento

bentoml export [OPTIONS] BENTO_TAG [OUT_PATH]

Options

-q, --quiet#: Suppress all warnings and info logs

--verbose, --debug#: Generate debug information

--do-not-track#: Do not send usage info

--context <cloud_context>#: BentoCloud context name.

Arguments

BENTO_TAG#: Required argument

OUT_PATH#: Optional argument

Environment variables

BENTOML_QUIET: Provide a default for -q

BENTOML_DEBUG: Provide a default for --verbose

BENTOML_DO_NOT_TRACK: Provide a default for --do-not-track

get#

Print Bento details by providing the bento_tag.

bentoml get iris_classifier:qojf5xauugwqtgxi

bentoml get iris_classifier:qojf5xauugwqtgxi –output=json

bentoml get [OPTIONS] BENTO_TAG

Options

-o, --output <output>#

Options:: json | yaml | path

-q, --quiet#: Suppress all warnings and info logs

--verbose, --debug#: Generate debug information

--do-not-track#: Do not send usage info

--context <cloud_context>#: BentoCloud context name.

Arguments

BENTO_TAG#: Required argument

Environment variables

BENTOML_QUIET: Provide a default for -q

BENTOML_DEBUG: Provide a default for --verbose

BENTOML_DO_NOT_TRACK: Provide a default for --do-not-track

import#

Import a previously exported Bento archive file

Arguments:

BENTO_PATH: path of Bento archive file

Examples:

bentoml import ./my_bento.bento

bentoml import s3://mybucket/bentos/my_bento.bento

bentoml import [OPTIONS] BENTO_PATH

Options

-q, --quiet#: Suppress all warnings and info logs

--verbose, --debug#: Generate debug information

--do-not-track#: Do not send usage info

--context <cloud_context>#: BentoCloud context name.

Arguments

BENTO_PATH#: Required argument

Environment variables

BENTOML_QUIET: Provide a default for -q

BENTOML_DEBUG: Provide a default for --verbose

BENTOML_DO_NOT_TRACK: Provide a default for --do-not-track

list#

List Bentos in local store

# show all bentos saved

$ bentoml list

# show all versions of bento with the name FraudDetector

$ bentoml list FraudDetector

bentoml list [OPTIONS] [BENTO_NAME]

Options

-o, --output <output>#

Options:: json | yaml | table

-q, --quiet#: Suppress all warnings and info logs

--verbose, --debug#: Generate debug information

--do-not-track#: Do not send usage info

--context <cloud_context>#: BentoCloud context name.

Arguments

BENTO_NAME#: Optional argument

Environment variables

BENTOML_QUIET: Provide a default for -q

BENTOML_DEBUG: Provide a default for --verbose

BENTOML_DO_NOT_TRACK: Provide a default for --do-not-track

models#

Model Subcommands Groups

bentoml models [OPTIONS] COMMAND [ARGS]...

delete#

Delete Model in local model store.

Examples:

* Delete single model by “name:version”, e.g: bentoml models delete iris_clf:v1
* Bulk delete all models with a specific name, e.g.: bentoml models delete iris_clf
* Bulk delete multiple models by name and version, separated by “,”, e.g.: bentoml models delete iris_clf:v1,iris_clf:v2
* Bulk delete multiple models by name and version, separated by “ “, e.g.: bentoml models delete iris_clf:v1 iris_clf:v2
* Bulk delete without confirmation, e.g.: bentoml models delete IrisClassifier –yes

bentoml models delete [OPTIONS] DELETE_TARGETS...

Options

-y, --yes, --assume-yes#: Skip confirmation when deleting a specific model

-q, --quiet#: Suppress all warnings and info logs

--verbose, --debug#: Generate debug information

--do-not-track#: Do not send usage info

--context <cloud_context>#: BentoCloud context name.

Arguments

DELETE_TARGETS#: Required argument(s)

Environment variables

BENTOML_QUIET: Provide a default for -q

BENTOML_DEBUG: Provide a default for --verbose

BENTOML_DO_NOT_TRACK: Provide a default for --do-not-track

export#

Export a Model to an external archive file

arguments:

MODEL_TAG: model identifier

OUT_PATH: output path of exported model.

If this argument is not provided, model is exported to name-version.bentomodel in the current directory.

Besides native .bentomodel format, we also support formats like tar(‘tar’), tar.gz (‘gz’), tar.xz (‘xz’), tar.bz2 (‘bz2’), and zip.

examples:

bentoml models export FraudDetector:latest
bentoml models export FraudDetector:latest ./my_model.bentomodel
bentoml models export FraudDetector:20210709_DE14C9 ./my_model.bentomodel
bentoml models export FraudDetector:20210709_DE14C9 s3://mybucket/models/my_model.bentomodel

bentoml models export [OPTIONS] MODEL_TAG [OUT_PATH]

Options

-q, --quiet#: Suppress all warnings and info logs

--verbose, --debug#: Generate debug information

--do-not-track#: Do not send usage info

--context <cloud_context>#: BentoCloud context name.

Arguments

MODEL_TAG#: Required argument

OUT_PATH#: Optional argument

Environment variables

BENTOML_QUIET: Provide a default for -q

BENTOML_DEBUG: Provide a default for --verbose

BENTOML_DO_NOT_TRACK: Provide a default for --do-not-track

get#

Print Model details by providing the model_tag

bentoml get iris_clf:qojf5xauugwqtgxi

bentoml get iris_clf:qojf5xauugwqtgxi –output=json

bentoml models get [OPTIONS] MODEL_TAG

Options

-o, --output <output>#

Options:: json | yaml | path

-q, --quiet#: Suppress all warnings and info logs

--verbose, --debug#: Generate debug information

--do-not-track#: Do not send usage info

--context <cloud_context>#: BentoCloud context name.

Arguments

MODEL_TAG#: Required argument

Environment variables

BENTOML_QUIET: Provide a default for -q

BENTOML_DEBUG: Provide a default for --verbose

BENTOML_DO_NOT_TRACK: Provide a default for --do-not-track

import#

Import a previously exported Model archive file

bentoml models import ./my_model.bentomodel bentoml models import s3://mybucket/models/my_model.bentomodel

bentoml models import [OPTIONS] MODEL_PATH

Options

-q, --quiet#: Suppress all warnings and info logs

--verbose, --debug#: Generate debug information

--do-not-track#: Do not send usage info

--context <cloud_context>#: BentoCloud context name.

Arguments

MODEL_PATH#: Required argument

Environment variables

BENTOML_QUIET: Provide a default for -q

BENTOML_DEBUG: Provide a default for --verbose

BENTOML_DO_NOT_TRACK: Provide a default for --do-not-track

list#

List Models in local store

# show all models saved

$ bentoml models list

# show all versions of bento with the name FraudDetector

$ bentoml models list FraudDetector

bentoml models list [OPTIONS] [MODEL_NAME]

Options

-o, --output <output>#

Options:: json | yaml | table

-q, --quiet#: Suppress all warnings and info logs

--verbose, --debug#: Generate debug information

--do-not-track#: Do not send usage info

--context <cloud_context>#: BentoCloud context name.

Arguments

MODEL_NAME#: Optional argument

Environment variables

BENTOML_QUIET: Provide a default for -q

BENTOML_DEBUG: Provide a default for --verbose

BENTOML_DO_NOT_TRACK: Provide a default for --do-not-track

pull#

Pull Model from a remote model store. If model_tag is not provided, it will pull models defined in bentofile.yaml.

bentoml models pull [OPTIONS] [MODEL_TAG]

Options

-f, --force#: Force pull from remote model store to local and overwrite even if it already exists in local

-F, --bentofile <bentofile>#: Path to bentofile. Default to ‘bentofile.yaml’

-q, --quiet#: Suppress all warnings and info logs

--verbose, --debug#: Generate debug information

--do-not-track#: Do not send usage info

--context <cloud_context>#: BentoCloud context name.

Arguments

MODEL_TAG#: Optional argument

Environment variables

BENTOML_QUIET: Provide a default for -q

BENTOML_DEBUG: Provide a default for --verbose

BENTOML_DO_NOT_TRACK: Provide a default for --do-not-track

push#

Push Model to a remote model store.

bentoml models push [OPTIONS] MODEL_TAG

Options

-f, --force#: Forced push to remote model store even if it exists in remote

-t, --threads <threads>#: Number of threads to use for upload

-q, --quiet#: Suppress all warnings and info logs

--verbose, --debug#: Generate debug information

--do-not-track#: Do not send usage info

--context <cloud_context>#: BentoCloud context name.

Arguments

MODEL_TAG#: Required argument

Environment variables

BENTOML_QUIET: Provide a default for -q

BENTOML_DEBUG: Provide a default for --verbose

BENTOML_DO_NOT_TRACK: Provide a default for --do-not-track

pull#

Pull Bento from a remote Bento store server.

bentoml pull [OPTIONS] BENTO_TAG

Options

-f, --force#: Force pull from remote Bento store to local and overwrite even if it already exists in local

-q, --quiet#: Suppress all warnings and info logs

--verbose, --debug#: Generate debug information

--do-not-track#: Do not send usage info

--context <cloud_context>#: BentoCloud context name.

Arguments

BENTO_TAG#: Required argument

Environment variables

BENTOML_QUIET: Provide a default for -q

BENTOML_DEBUG: Provide a default for --verbose

BENTOML_DO_NOT_TRACK: Provide a default for --do-not-track

push#

Push Bento to a remote Bento store server.

bentoml push [OPTIONS] BENTO_TAG

Options

-f, --force#: Forced push to remote Bento store even if it exists in remote

-t, --threads <threads>#: Number of threads to use for upload

-q, --quiet#: Suppress all warnings and info logs

--verbose, --debug#: Generate debug information

--do-not-track#: Do not send usage info

--context <cloud_context>#: BentoCloud context name.

Arguments

BENTO_TAG#: Required argument

Environment variables

BENTOML_QUIET: Provide a default for -q

BENTOML_DEBUG: Provide a default for --verbose

BENTOML_DO_NOT_TRACK: Provide a default for --do-not-track

serve#

Start a HTTP BentoServer from a given 🍱

BENTO is the serving target, it can be the import as:
- the import path of a ‘bentoml.Service’ instance
- a tag to a Bento in local Bento store
- a folder containing a valid ‘bentofile.yaml’ build file with a ‘service’ field, which provides the import path of a ‘bentoml.Service’ instance
- a path to a built Bento (for internal & debug use only)

e.g.:

Serve from a bentoml.Service instance source code (for development use only):

‘bentoml serve fraud_detector.py:svc’

Serve from a Bento built in local store:

‘bentoml serve fraud_detector:4tht2icroji6zput3suqi5nl2’

‘bentoml serve fraud_detector:latest’

Serve from a Bento directory:

‘bentoml serve ./fraud_detector_bento’

If ‘–reload’ is provided, BentoML will detect code and model store changes during development, and restarts the service automatically.

The ‘–reload’ flag will:
- be default, all file changes under ‘–working-dir’ (default to current directory) will trigger a restart
- when specified, respect ‘include’ and ‘exclude’ under ‘bentofile.yaml’ as well as the ‘.bentoignore’ file in ‘–working-dir’, for code and file changes
- all model store changes will also trigger a restart (new model saved or existing model removed)

bentoml serve [OPTIONS] [BENTO]

Options

--development#

Run the BentoServer in development mode

Default:: False

--production#: Run BentoServer in production mode (Deprecated)

-p, --port <port>#

The port to listen on for the REST api server

Default:: 3000

--host <host>#

The host to bind for the REST api server

Default:: 0.0.0.0

--api-workers <api_workers>#

Specify the number of API server workers to start. Default to number of available CPU cores in production mode

Default:: 2

--timeout <timeout>#: Specify the timeout (seconds) for API server and runners

--backlog <backlog>#

The maximum number of pending connections.

Default:: 2048

--reload#

Reload Service when code changes detected

Default:: False

--working-dir <working_dir>#: When loading from source code, specify the directory to find the Service instance

--ssl-certfile <ssl_certfile>#: SSL certificate file

--ssl-keyfile <ssl_keyfile>#: SSL key file

--ssl-keyfile-password <ssl_keyfile_password>#: SSL keyfile password

--ssl-version <ssl_version>#

SSL version to use (see stdlib ‘ssl’ module)

Default:: 17

--ssl-cert-reqs <ssl_cert_reqs>#

Whether client certificate is required (see stdlib ‘ssl’ module)

Default:: 0

--ssl-ca-certs <ssl_ca_certs>#: CA certificates file

--ssl-ciphers <ssl_ciphers>#

Ciphers to use (see stdlib ‘ssl’ module)

Default:: TLSv1

--env <env>#

Environment to run the command in

Options:: conda

-q, --quiet#: Suppress all warnings and info logs

--verbose, --debug#: Generate debug information

--do-not-track#: Do not send usage info

--context <cloud_context>#: BentoCloud context name.

Arguments

BENTO#: Optional argument

Environment variables

BENTOML_PORT: Provide a default for -p

BENTOML_HOST: Provide a default for --host

BENTOML_API_WORKERS: Provide a default for --api-workers

BENTOML_TIMEOUT: Provide a default for --timeout

BENTOML_QUIET: Provide a default for -q

BENTOML_DEBUG: Provide a default for --verbose

BENTOML_DO_NOT_TRACK: Provide a default for --do-not-track

serve-grpc#

Start a gRPC BentoServer from a given 🍱

BENTO is the serving target, it can be the import as:
- the import path of a ‘bentoml.Service’ instance
- a tag to a Bento in local Bento store
- a folder containing a valid ‘bentofile.yaml’ build file with a ‘service’ field, which provides the import path of a ‘bentoml.Service’ instance
- a path to a built Bento (for internal & debug use only)

e.g.:

Serve from a bentoml.Service instance source code (for development use only):

‘bentoml serve-grpc fraud_detector.py:svc’

Serve from a Bento built in local store:

‘bentoml serve-grpc fraud_detector:4tht2icroji6zput3suqi5nl2’

‘bentoml serve-grpc fraud_detector:latest’

Serve from a Bento directory:

‘bentoml serve-grpc ./fraud_detector_bento’

If ‘–reload’ is provided, BentoML will detect code and model store changes during development, and restarts the service automatically.

The ‘–reload’ flag will:
- be default, all file changes under ‘–working-dir’ (default to current directory) will trigger a restart
- when specified, respect ‘include’ and ‘exclude’ under ‘bentofile.yaml’ as well as the ‘.bentoignore’ file in ‘–working-dir’, for code and file changes
- all model store changes will also trigger a restart (new model saved or existing model removed)

bentoml serve-grpc [OPTIONS] [BENTO]

Options

--development#

Run the BentoServer in development mode

Default:: False

--production#: Run BentoServer in production mode (Deprecated)

-p, --port <port>#

The port to listen on for the REST api server

Default:: 3000

--host <host>#

The host to bind for the gRPC server

Default:: 0.0.0.0

--api-workers <api_workers>#

Specify the number of API server workers to start. Default to number of available CPU cores in production mode

Default:: 2

--reload#

Reload Service when code changes detected

Default:: False

--backlog <backlog>#

The maximum number of pending connections.

Default:: 2048

--working-dir <working_dir>#: When loading from source code, specify the directory to find the Service instance

--enable-reflection#

Enable reflection.

Default:: False

--enable-channelz#: Enable Channelz. See https://github.com/grpc/proposal/blob/master/A14-channelz.md.

--max-concurrent-streams <max_concurrent_streams>#: Maximum number of concurrent incoming streams to allow on a http2 connection.

--ssl-certfile <ssl_certfile>#: SSL certificate file

--ssl-keyfile <ssl_keyfile>#: SSL key file

--ssl-ca-certs <ssl_ca_certs>#: CA certificates file

-pv, --protocol-version <protocol_version>#

Determine the version of generated gRPC stubs to use.

Default:: v1
Options:: v1 | v1alpha1

--env <env>#

Environment to run the command in

Options:: conda

-q, --quiet#: Suppress all warnings and info logs

--verbose, --debug#: Generate debug information

--do-not-track#: Do not send usage info

--context <cloud_context>#: BentoCloud context name.

Arguments

BENTO#: Optional argument

Environment variables

BENTOML_PORT: Provide a default for -p

BENTOML_HOST: Provide a default for --host

BENTOML_API_WORKERS: Provide a default for --api-workers

BENTOML_QUIET: Provide a default for -q

BENTOML_DEBUG: Provide a default for --verbose

BENTOML_DO_NOT_TRACK: Provide a default for --do-not-track