Core Components#

bentoml.Service#

class bentoml.Service(name: str, *, runners: t.List[Runner] | None = None, models: t.List[Model] | None = None)[source]#

The service definition is the manifestation of the Service Oriented Architecture and the core building block in BentoML where users define the service runtime architecture and model serving logic.

A BentoML service is defined via instantiate this Service class. When creating a Service instance, user must provide a Service name and list of runners that are required by this Service. The instance can then be used to define InferenceAPIs via the api decorator.

add_asgi_middleware(middleware_cls: Type[ext.AsgiMiddleware], **options: Any) None[source]#
api(input: IODescriptor[Any], output: IODescriptor[Any], name: Optional[str] = None, doc: Optional[str] = None, route: Optional[str] = None) Callable[[Callable[[...], Any]], Callable[[...], Any]][source]#

Decorator for adding InferenceAPI to this service

apis: t.Dict[str, InferenceAPI]#
mount_asgi_app(app: ext.ASGIApp, path: str = '/', name: Optional[str] = None) None[source]#
mount_wsgi_app(app: WSGI_APP, path: str = '/', name: t.Optional[str] = None) None[source]#
runners: t.List[Runner]#
bentoml.load(bento_identifier: str, working_dir: Optional[str] = None, standalone_load: bool = False) Service[source]#

Load a Service instance by the bento_identifier

Parameters
  • bento_identifier – target Service to import or Bento to load

  • working_dir – when importing from service, set the working_dir

  • standalone_load – treat target Service as standalone. This will change global current working directory and global model store.

The argument bento_identifier can be one of the following forms:

  • Tag pointing to a Bento in local Bento store under BENTOML_HOME/bentos

  • File path to a Bento directory

  • “import_str” for loading a service instance from the working_dir

Example load from Bento usage:

# load from local bento store
load("FraudDetector:latest")
load("FraudDetector:4tht2icroji6zput")

# load from bento directory
load("~/bentoml/bentos/iris_classifier/4tht2icroji6zput")

Example load from working directory by “import_str” usage:

# When multiple service defined in the same module
load("fraud_detector:svc_a")
load("fraud_detector:svc_b")

# Find svc by Python module name or file path
load("fraud_detector:svc")
load("fraud_detector.py:svc")
load("foo.bar.fraud_detector:svc")
load("./def/abc/fraud_detector.py:svc")

# When there's only one Service instance in the target module, the attributes
# part in the svc_import_path can be omitted
load("fraud_detector.py")
load("fraud_detector")

Limitations when standalone_load=False: * Models used in the Service being imported, if not accessed during module import,

must be presented in the global model store

  • Files required for the Service to run, if not accessed during module import, must

    be presented in the current working directory

Todo

Add docstring to the following classes/functions

bentoml.build#

bentoml.bentos.build(service: str, *, labels: ~typing.Optional[~typing.Dict[str, str]] = None, description: ~typing.Optional[str] = None, include: ~typing.Optional[~typing.List[str]] = None, exclude: ~typing.Optional[~typing.List[str]] = None, docker: ~typing.Optional[~typing.Dict[str, ~typing.Any]] = None, python: ~typing.Optional[~typing.Dict[str, ~typing.Any]] = None, conda: ~typing.Optional[~typing.Dict[str, ~typing.Any]] = None, version: ~typing.Optional[str] = None, build_ctx: ~typing.Optional[str] = None, _bento_store: BentoStore = <simple_di.providers.SingletonFactory object>, _model_store: ModelStore = <simple_di.providers.SingletonFactory object>) Bento[source]#

User-facing API for building a Bento. The available build options are identical to the keys of a valid ‘bentofile.yaml’ file.

This API will not respect any ‘bentofile.yaml’ files. Build options should instead be provided via function call parameters.

Parameters
  • service – import str for finding the bentoml.Service instance build target

  • labels – optional immutable labels for carrying contextual info

  • description – optional description string in markdown format

  • include – list of file paths and patterns specifying files to include in Bento, default is all files under build_ctx, beside the ones excluded from the exclude parameter or a .bentoignore file for a given directory

  • exclude – list of file paths and patterns to exclude from the final Bento archive

  • docker – dictionary for configuring Bento’s containerization process, see details in bentoml._internal.bento.build_config.DockerOptions

  • python – dictionary for configuring Bento’s python dependencies, see details in bentoml._internal.bento.build_config.PythonOptions

  • conda – dictionary for configuring Bento’s conda dependencies, see details in bentoml._internal.bento.build_config.CondaOptions

  • version – Override the default auto generated version str

  • build_ctx – Build context directory, when used as

  • _bento_store – save Bento created to this BentoStore

  • _model_store – pull Models required from this ModelStore

Returns

a Bento instance representing the materialized Bento saved in BentoStore

Return type

Bento

Example

import bentoml

bentoml.build(
    service="fraud_detector.py:svc",
    version="any_version_label",  # override default version generator
    description=open("README.md").read(),
    include=['*'],
    exclude=[], # files to exclude can also be specified with a .bentoignore file
    labels={
        "foo": "bar",
        "team": "abc"
    },
    python=dict(
        packages=["tensorflow", "numpy"],
        # requirements_txt="./requirements.txt",
        index_url="http://<api token>:@mycompany.com/pypi/simple",
        trusted_host=["mycompany.com"],
        find_links=['thirdparty..'],
        extra_index_url=["..."],
        pip_args="ANY ADDITIONAL PIP INSTALL ARGS",
        wheels=["./wheels/*"],
        lock_packages=True,
    ),
    docker=dict(
        distro="amazonlinux2",
        setup_script="setup_docker_container.sh",
        python_version="3.8",
    ),
)
bentoml.bentos.build_bentofile(bentofile: str = 'bentofile.yaml', *, version: ~typing.Optional[str] = None, build_ctx: ~typing.Optional[str] = None, _bento_store: BentoStore = <simple_di.providers.SingletonFactory object>, _model_store: ModelStore = <simple_di.providers.SingletonFactory object>) Bento[source]#

Build a Bento base on options specified in a bentofile.yaml file.

By default, this function will look for a bentofile.yaml file in current working directory.

Parameters
  • bentofile – The file path to build config yaml file

  • version – Override the default auto generated version str

  • build_ctx – Build context directory, when used as

  • _bento_store – save Bento created to this BentoStore

  • _model_store – pull Models required from this ModelStore

bentoml.bentos.containerize(tag: Tag | str, docker_image_tag: str | None = None, *, add_host: dict[str, str] | None = None, allow: t.List[str] | None = None, build_args: dict[str, str] | None = None, build_context: dict[str, str] | None = None, builder: str | None = None, cache_from: str | t.List[str] | dict[str, str] | None = None, cache_to: str | t.List[str] | dict[str, str] | None = None, cgroup_parent: str | None = None, iidfile: PathType | None = None, labels: dict[str, str] | None = None, load: bool = True, metadata_file: PathType | None = None, network: str | None = None, no_cache: bool = False, no_cache_filter: t.List[str] | None = None, output: str | dict[str, str] | None = None, platform: str | t.List[str] | None = None, progress: t.Literal[('auto', 'tty', 'plain')] = 'auto', pull: bool = False, push: bool = False, quiet: bool = False, secrets: str | t.List[str] | None = None, shm_size: str | int | None = None, rm: bool = False, ssh: str | None = None, target: str | None = None, ulimit: str | None = None, _bento_store: BentoStore = <simple_di.providers.SingletonFactory object>) bool[source]#

bentoml.Bento#

class bentoml.Bento(tag: Tag, bento_fs: FS, info: BentoInfo)[source]#
property doc: str#
property info: bentoml._internal.bento.bento.BentoInfo#
property path: str#
path_of(item: str) str[source]#
property tag: bentoml._internal.tag.Tag#

bentoml.Runner#

class bentoml.Runner(runnable_class: t.Type[Runnable], *, runnable_init_params: t.Dict[str, t.Any] | None = None, name: str | None = None, scheduling_strategy: t.Type[Strategy] = <class 'bentoml._internal.runner.strategy.DefaultStrategy'>, models: t.List[Model] | None = None, max_batch_size: int | None = None, max_latency_ms: int | None = None, method_configs: t.Dict[str, t.Dict[str, int]] | None = None)[source]#

bentoml.Runnable#

class bentoml.Runnable[source]#

Tag#

class bentoml.Tag(name: str, version: Optional[str] = None)[source]#

Model#

class bentoml.Model(tag: Tag, model_fs: FS, info: ModelInfo, custom_objects: dict[str, t.Any] | None = None, *, _internal: bool = False)[source]#
property info: bentoml._internal.models.model.ModelInfo#
property path: str#
path_of(item: str) str[source]#
to_runnable() t.Type[Runnable][source]#
to_runner(name: str = '', max_batch_size: int | None = None, max_latency_ms: int | None = None, method_configs: dict[str, dict[str, int]] | None = None) Runner[source]#

TODO(chaoyu): add docstring

Parameters
  • name

  • max_batch_size

  • max_latency_ms

  • runnable_method_configs

Returns:

with_options(**kwargs: Any) Model[source]#

YataiClient#

class bentoml.YataiClient[source]#