Core Components#

bentoml.Service#

class bentoml.Service(name: str, *, runners: Optional[List[Runner]] = None, models: Optional[List[Model]] = None)[source]#

The service definition is the manifestation of the Service Oriented Architecture and the core building block in BentoML where users define the service runtime architecture and model serving logic.

A BentoML service is defined via instantiate this Service class. When creating a Service instance, user must provide a Service name and list of runners that are required by this Service. The instance can then be used to define InferenceAPIs via the api decorator.

add_asgi_middleware(middleware_cls: t.Type[ext.AsgiMiddleware], **options: t.Any) → None[source]#

api(input: IODescriptor[Any], output: IODescriptor[Any], name: Optional[str] = None, doc: Optional[str] = None, route: Optional[str] = None) → Callable[[Callable[[...], Any]], Callable[[...], Any]][source]#: Decorator for adding InferenceAPI to this service

apis: t.Dict[str, InferenceAPI]#

mount_asgi_app(app: ext.ASGIApp, path: str = '/', name: t.Optional[str] = None) → None[source]#

mount_wsgi_app(app: ext.WSGIApp, path: str = '/', name: t.Optional[str] = None) → None[source]#

runners: t.List[Runner]#

bentoml.load(bento_identifier: str, working_dir: t.Optional[str] = None, standalone_load: bool = False) → Service[source]#

Load a Service instance by the bento_identifier

Parameters

bento_identifier – target Service to import or Bento to load
working_dir – when importing from service, set the working_dir
standalone_load – treat target Service as standalone. This will change global current working directory and global model store.

The argument bento_identifier can be one of the following forms:

Tag pointing to a Bento in local Bento store under BENTOML_HOME/bentos
File path to a Bento directory
“import_str” for loading a service instance from the working_dir

Example load from Bento usage:

# load from local bento store
load("FraudDetector:latest")
load("FraudDetector:4tht2icroji6zput")

# load from bento directory
load("~/bentoml/bentos/iris_classifier/4tht2icroji6zput")

Example load from working directory by “import_str” usage:

# When multiple service defined in the same module
load("fraud_detector:svc_a")
load("fraud_detector:svc_b")

# Find svc by Python module name or file path
load("fraud_detector:svc")
load("fraud_detector.py:svc")
load("foo.bar.fraud_detector:svc")
load("./def/abc/fraud_detector.py:svc")

# When there's only one Service instance in the target module, the attributes
# part in the svc_import_path can be omitted
load("fraud_detector.py")
load("fraud_detector")

Limitations when standalone_load=False: * Models used in the Service being imported, if not accessed during module import,

must be presented in the global model store

Files required for the Service to run, if not accessed during module import, must
be presented in the current working directory

Todo

Add docstring to the following classes/functions

bentoml.build#

bentoml.bentos.build(service: str, *, labels: t.Optional[t.Dict[str, str]] = None, description: t.Optional[str] = None, include: t.Optional[t.List[str]] = None, exclude: t.Optional[t.List[str]] = None, docker: t.Optional[t.Dict[str, t.Any]] = None, python: t.Optional[t.Dict[str, t.Any]] = None, conda: t.Optional[t.Dict[str, t.Any]] = None, version: t.Optional[str] = None, build_ctx: t.Optional[str] = None, _bento_store: BentoStore = <simple_di.providers.SingletonFactory object>) → Bento[source]#

User-facing API for building a Bento. The available build options are identical to the keys of a valid ‘bentofile.yaml’ file.

This API will not respect any ‘bentofile.yaml’ files. Build options should instead be provided via function call parameters.

Parameters

service – import str for finding the bentoml.Service instance build target
labels – optional immutable labels for carrying contextual info
description – optional description string in markdown format
include – list of file paths and patterns specifying files to include in Bento, default is all files under build_ctx, beside the ones excluded from the exclude parameter or a .bentoignore file for a given directory
exclude – list of file paths and patterns to exclude from the final Bento archive
docker – dictionary for configuring Bento’s containerization process, see details in bentoml._internal.bento.build_config.DockerOptions
python – dictionary for configuring Bento’s python dependencies, see details in bentoml._internal.bento.build_config.PythonOptions
conda – dictionary for configuring Bento’s conda dependencies, see details in bentoml._internal.bento.build_config.CondaOptions
version – Override the default auto generated version str
build_ctx – Build context directory, when used as
_bento_store – save Bento created to this BentoStore

Returns

a Bento instance representing the materialized Bento saved in BentoStore

Return type

Bento

Example

import bentoml

bentoml.build(
    service="fraud_detector.py:svc",
    version="any_version_label",  # override default version generator
    description=open("README.md").read(),
    include=['*'],
    exclude=[], # files to exclude can also be specified with a .bentoignore file
    labels={
        "foo": "bar",
        "team": "abc"
    },
    python=dict(
        packages=["tensorflow", "numpy"],
        # requirements_txt="./requirements.txt",
        index_url="http://<api token>:@mycompany.com/pypi/simple",
        trusted_host=["mycompany.com"],
        find_links=['thirdparty..'],
        extra_index_url=["..."],
        pip_args="ANY ADDITIONAL PIP INSTALL ARGS",
        wheels=["./wheels/*"],
        lock_packages=True,
    ),
    docker=dict(
        distro="amazonlinux2",
        setup_script="setup_docker_container.sh",
        python_version="3.8",
    ),
)

bentoml.bentos.build_bentofile(bentofile: str = 'bentofile.yaml', *, version: t.Optional[str] = None, build_ctx: t.Optional[str] = None, _bento_store: BentoStore = <simple_di.providers.SingletonFactory object>) → Bento[source]#

Build a Bento base on options specified in a bentofile.yaml file.

By default, this function will look for a bentofile.yaml file in current working directory.

Parameters

bentofile – The file path to build config yaml file
version – Override the default auto generated version str
build_ctx – Build context directory, when used as
_bento_store – save Bento created to this BentoStore

bentoml.bentos.containerize(tag: Tag | str, docker_image_tag: t.Iterable[str] | None = None, *, features: t.Sequence[str] | None = None, add_host: dict[str, str] | None = None, allow: t.List[str] | None = None, build_args: dict[str, str] | None = None, build_context: dict[str, str] | None = None, builder: str | None = None, cache_from: str | t.Iterable[str] | dict[str, str] | None = None, cache_to: str | t.Iterable[str] | dict[str, str] | None = None, cgroup_parent: str | None = None, iidfile: PathType | None = None, labels: dict[str, str] | None = None, load: bool = True, metadata_file: PathType | None = None, network: str | None = None, no_cache: bool = False, no_cache_filter: t.Iterable[str] | None = None, output: str | dict[str, str] | None = None, platform: str | t.Iterable[str] | None = None, progress: t.Literal['auto', 'tty', 'plain'] = 'auto', pull: bool = False, push: bool = False, quiet: bool = False, secrets: str | t.Iterable[str] | None = None, shm_size: str | int | None = None, rm: bool = False, ssh: str | None = None, target: str | None = None, ulimit: str | None = None, _bento_store: BentoStore = <simple_di.providers.SingletonFactory object>) → bool[source]#

bentoml.Bento#

class bentoml.Bento(tag: Tag, bento_fs: FS, info: BentoInfo)[source]#

property doc: str#

property info: BentoInfo#

property path: str#

path_of(item: str) → str[source]#

property tag: Tag#

bentoml.Runner#

class bentoml.Runner(runnable_class: ~typing.Type[~bentoml._internal.runner.runnable.Runnable], *, runnable_init_params: ~typing.Optional[~typing.Dict[str, ~typing.Any]] = None, name: ~typing.Optional[str] = None, scheduling_strategy: ~typing.Type[~bentoml._internal.runner.strategy.Strategy] = <class 'bentoml._internal.runner.strategy.DefaultStrategy'>, models: ~typing.Optional[~typing.List[~bentoml._internal.models.model.Model]] = None, max_batch_size: ~typing.Optional[int] = None, max_latency_ms: ~typing.Optional[int] = None, method_configs: ~typing.Optional[~typing.Dict[str, ~typing.Dict[str, int]]] = None)[source]#

bentoml.Runnable#

class bentoml.Runnable[source]#

static method(meth: t.Callable[t.Concatenate[T, P], R], *, batchable: bool = False, batch_dim: tuple[int, int] | int = 0, input_spec: AnyType | tuple[AnyType, ...] | None = None, output_spec: AnyType | None = None) → RunnableMethod[T, P, R][source]#
static method(meth: None = None, *, batchable: bool = False, batch_dim: tuple[int, int] | int = 0, input_spec: AnyType | tuple[AnyType, ...] | None = None, output_spec: AnyType | None = None) → t.Callable[[t.Callable[t.Concatenate[T, P], R]], RunnableMethod[T, P, R]]

Tag#

class bentoml.Tag(name: str, version: Optional[str] = None)[source]#

Model#

class bentoml.Model(tag: Tag, model_fs: FS, info: ModelInfo, custom_objects: Optional[dict[str, Any]] = None, *, _internal: bool = False)[source]#

property info: ModelInfo#

property path: str#

path_of(item: str) → str[source]#

to_runnable() → t.Type[Runnable][source]#

to_runner(name: str = '', max_batch_size: int | None = None, max_latency_ms: int | None = None, method_configs: dict[str, dict[str, int]] | None = None) → Runner[source]#

TODO(chaoyu): add docstring

Parameters

name –
max_batch_size –
max_latency_ms –
runnable_method_configs –

Returns:

with_options(**kwargs: Any) → Model[source]#

YataiClient#

class bentoml.YataiClient[source]#