Core Components#
bentoml.Service#
- class bentoml.Service(name: str, *, runners: list[bentoml._internal.runner.runner.AbstractRunner] | None = None, models: list[bentoml._internal.models.model.Model] | None = None)[source]#
The service definition is the manifestation of the Service Oriented Architecture and the core building block in BentoML where users define the service runtime architecture and model serving logic.
A BentoML service is defined via instantiate this Service class. When creating a Service instance, user must provide a Service name and list of runners that are required by this Service. The instance can then be used to define InferenceAPIs via the api decorator.
- bentoml.load(bento_identifier: str | Tag | Bento, working_dir: t.Optional[str] = None, standalone_load: bool = False) Service [source]#
Load a Service instance by the bento_identifier
- Parameters:
bento_identifier – target Service to import or Bento to load
working_dir – when importing from service, set the working_dir
standalone_load – treat target Service as standalone. This will change global current working directory and global model store.
- Returns:
The loaded
bentoml.Service
instance.
The argument
bento_identifier
can be one of the following forms:Tag pointing to a Bento in local Bento store under BENTOML_HOME/bentos
File path to a Bento directory
“import_str” for loading a service instance from the working_dir
Example load from Bento usage:
# load from local bento store load("FraudDetector:latest") load("FraudDetector:4tht2icroji6zput") # load from bento directory load("~/bentoml/bentos/iris_classifier/4tht2icroji6zput")
Example load from working directory by “import_str” usage:
# When multiple service defined in the same module load("fraud_detector:svc_a") load("fraud_detector:svc_b") # Find svc by Python module name or file path load("fraud_detector:svc") load("fraud_detector.py:svc") load("foo.bar.fraud_detector:svc") load("./def/abc/fraud_detector.py:svc") # When there's only one Service instance in the target module, the attributes # part in the svc_import_path can be omitted load("fraud_detector.py") load("fraud_detector")
Limitations when standalone_load=False:
Models used in the Service being imported, if not accessed during module import, must be presented in the global model store
Files required for the Service to run, if not accessed during module import, must be presented in the current working directory
Todo
Add docstring to the following classes/functions
bentoml.build#
- bentoml.bentos.build(service: str, *, name: str | None = None, labels: dict[str, str] | None = None, description: str | None = None, include: t.List[str] | None = None, exclude: t.List[str] | None = None, docker: DockerOptions | dict[str, t.Any] | None = None, python: PythonOptions | dict[str, t.Any] | None = None, conda: CondaOptions | dict[str, t.Any] | None = None, version: str | None = None, build_ctx: str | None = None, _bento_store: BentoStore = <simple_di.providers.SingletonFactory object>) Bento [source]#
User-facing API for building a Bento. The available build options are identical to the keys of a valid ‘bentofile.yaml’ file.
This API will not respect any ‘bentofile.yaml’ files. Build options should instead be provided via function call parameters.
- Parameters:
service – import str for finding the bentoml.Service instance build target
labels – optional immutable labels for carrying contextual info
description – optional description string in markdown format
include – list of file paths and patterns specifying files to include in Bento, default is all files under build_ctx, beside the ones excluded from the exclude parameter or a
.bentoignore
file for a given directoryexclude – list of file paths and patterns to exclude from the final Bento archive
docker – dictionary for configuring Bento’s containerization process, see details in
bentoml._internal.bento.build_config.DockerOptions
python – dictionary for configuring Bento’s python dependencies, see details in
bentoml._internal.bento.build_config.PythonOptions
conda – dictionary for configuring Bento’s conda dependencies, see details in
bentoml._internal.bento.build_config.CondaOptions
version – Override the default auto generated version str
build_ctx – Build context directory, when used as
_bento_store – save Bento created to this BentoStore
- Returns:
a Bento instance representing the materialized Bento saved in BentoStore
- Return type:
Example
import bentoml bentoml.build( service="fraud_detector.py:svc", version="any_version_label", # override default version generator description=open("README.md").read(), include=['*'], exclude=[], # files to exclude can also be specified with a .bentoignore file labels={ "foo": "bar", "team": "abc" }, python=dict( packages=["tensorflow", "numpy"], # requirements_txt="./requirements.txt", index_url="http://<api token>:@mycompany.com/pypi/simple", trusted_host=["mycompany.com"], find_links=['thirdparty..'], extra_index_url=["..."], pip_args="ANY ADDITIONAL PIP INSTALL ARGS", wheels=["./wheels/*"], lock_packages=True, ), docker=dict( distro="amazonlinux2", setup_script="setup_docker_container.sh", python_version="3.8", ), )
- bentoml.bentos.build_bentofile(bentofile: str = 'bentofile.yaml', *, version: str | None = None, build_ctx: str | None = None, _bento_store: BentoStore = <simple_di.providers.SingletonFactory object>) Bento [source]#
Build a Bento base on options specified in a bentofile.yaml file.
By default, this function will look for a bentofile.yaml file in current working directory.
- Parameters:
bentofile – The file path to build config yaml file
version – Override the default auto generated version str
build_ctx – Build context directory, when used as
_bento_store – save Bento created to this BentoStore
bentoml.Bento#
bentoml.Runner#
- class bentoml.Runner(runnable_class: type[bentoml._internal.runner.runnable.Runnable], *, runnable_init_params: dict[str, typing.Any] | None = None, name: str | None = None, scheduling_strategy: type[bentoml._internal.runner.strategy.Strategy] = <class 'bentoml._internal.runner.strategy.DefaultStrategy'>, models: list[bentoml._internal.models.model.Model] | None = None, max_batch_size: int | None = None, max_latency_ms: int | None = None, method_configs: dict[str, dict[str, int]] | None = None, embedded: bool = False)[source]#
bentoml.Runnable#
- class bentoml.Runnable[source]#
- static method(meth: t.Callable[t.Concatenate[T, P], R], *, batchable: bool = False, batch_dim: tuple[int, int] | int = 0, input_spec: AnyType | tuple[AnyType, ...] | None = None, output_spec: AnyType | None = None) RunnableMethod[T, P, R] [source]#
- static method(meth: None = None, *, batchable: bool = False, batch_dim: tuple[int, int] | int = 0, input_spec: AnyType | tuple[AnyType, ...] | None = None, output_spec: AnyType | None = None) t.Callable[[t.Callable[t.Concatenate[T, P], R]], RunnableMethod[T, P, R]]
Tag#
Model#
- class bentoml.Model(tag: Tag, model_fs: FS, info: ModelInfo, custom_objects: dict[str, Any] | None = None, *, _internal: bool = False)[source]#
- property info: ModelInfo#
- to_runner(name: str = '', max_batch_size: int | None = None, max_latency_ms: int | None = None, method_configs: dict[str, dict[str, int]] | None = None, embedded: bool = False, scheduling_strategy: type[Strategy] | None = None) Runner [source]#
TODO(chaoyu): add docstring
- Parameters:
name –
max_batch_size –
max_latency_ms –
runnable_method_configs –
Returns: