Core Components#
bentoml.Service#
- class bentoml.Service(name: str, *, runners: Optional[list[bentoml._internal.runner.runner.AbstractRunner]] = None, models: Optional[list[bentoml._internal.models.model.Model]] = None)[source]#
The service definition is the manifestation of the Service Oriented Architecture and the core building block in BentoML where users define the service runtime architecture and model serving logic.
A BentoML service is defined via instantiate this Service class. When creating a Service instance, user must provide a Service name and list of runners that are required by this Service. The instance can then be used to define InferenceAPIs via the api decorator.
- bentoml.load(bento_identifier: str, working_dir: t.Optional[str] = None, standalone_load: bool = False) Service [source]#
Load a Service instance by the bento_identifier
- Parameters:
bento_identifier – target Service to import or Bento to load
working_dir – when importing from service, set the working_dir
standalone_load – treat target Service as standalone. This will change global current working directory and global model store.
The argument bento_identifier can be one of the following forms:
Tag pointing to a Bento in local Bento store under BENTOML_HOME/bentos
File path to a Bento directory
“import_str” for loading a service instance from the working_dir
Example load from Bento usage:
# load from local bento store load("FraudDetector:latest") load("FraudDetector:4tht2icroji6zput") # load from bento directory load("~/bentoml/bentos/iris_classifier/4tht2icroji6zput")
Example load from working directory by “import_str” usage:
# When multiple service defined in the same module load("fraud_detector:svc_a") load("fraud_detector:svc_b") # Find svc by Python module name or file path load("fraud_detector:svc") load("fraud_detector.py:svc") load("foo.bar.fraud_detector:svc") load("./def/abc/fraud_detector.py:svc") # When there's only one Service instance in the target module, the attributes # part in the svc_import_path can be omitted load("fraud_detector.py") load("fraud_detector")
Limitations when standalone_load=False: * Models used in the Service being imported, if not accessed during module import,
must be presented in the global model store
- Files required for the Service to run, if not accessed during module import, must
be presented in the current working directory
Todo
Add docstring to the following classes/functions
bentoml.build#
- bentoml.bentos.build(service: str, *, labels: t.Optional[t.Dict[str, str]] = None, description: t.Optional[str] = None, include: t.Optional[t.List[str]] = None, exclude: t.Optional[t.List[str]] = None, docker: t.Optional[t.Dict[str, t.Any]] = None, python: t.Optional[t.Dict[str, t.Any]] = None, conda: t.Optional[t.Dict[str, t.Any]] = None, version: t.Optional[str] = None, build_ctx: t.Optional[str] = None, _bento_store: BentoStore = <simple_di.providers.SingletonFactory object>) Bento [source]#
User-facing API for building a Bento. The available build options are identical to the keys of a valid ‘bentofile.yaml’ file.
This API will not respect any ‘bentofile.yaml’ files. Build options should instead be provided via function call parameters.
- Parameters:
service – import str for finding the bentoml.Service instance build target
labels – optional immutable labels for carrying contextual info
description – optional description string in markdown format
include – list of file paths and patterns specifying files to include in Bento, default is all files under build_ctx, beside the ones excluded from the exclude parameter or a
.bentoignore
file for a given directoryexclude – list of file paths and patterns to exclude from the final Bento archive
docker – dictionary for configuring Bento’s containerization process, see details in
bentoml._internal.bento.build_config.DockerOptions
python – dictionary for configuring Bento’s python dependencies, see details in
bentoml._internal.bento.build_config.PythonOptions
conda – dictionary for configuring Bento’s conda dependencies, see details in
bentoml._internal.bento.build_config.CondaOptions
version – Override the default auto generated version str
build_ctx – Build context directory, when used as
_bento_store – save Bento created to this BentoStore
- Returns:
a Bento instance representing the materialized Bento saved in BentoStore
- Return type:
Example
import bentoml bentoml.build( service="fraud_detector.py:svc", version="any_version_label", # override default version generator description=open("README.md").read(), include=['*'], exclude=[], # files to exclude can also be specified with a .bentoignore file labels={ "foo": "bar", "team": "abc" }, python=dict( packages=["tensorflow", "numpy"], # requirements_txt="./requirements.txt", index_url="http://<api token>:@mycompany.com/pypi/simple", trusted_host=["mycompany.com"], find_links=['thirdparty..'], extra_index_url=["..."], pip_args="ANY ADDITIONAL PIP INSTALL ARGS", wheels=["./wheels/*"], lock_packages=True, ), docker=dict( distro="amazonlinux2", setup_script="setup_docker_container.sh", python_version="3.8", ), )
- bentoml.bentos.build_bentofile(bentofile: str = 'bentofile.yaml', *, version: t.Optional[str] = None, build_ctx: t.Optional[str] = None, _bento_store: BentoStore = <simple_di.providers.SingletonFactory object>) Bento [source]#
Build a Bento base on options specified in a bentofile.yaml file.
By default, this function will look for a bentofile.yaml file in current working directory.
- Parameters:
bentofile – The file path to build config yaml file
version – Override the default auto generated version str
build_ctx – Build context directory, when used as
_bento_store – save Bento created to this BentoStore
bentoml.Bento#
bentoml.Runner#
- class bentoml.Runner(runnable_class: type[bentoml._internal.runner.runnable.Runnable], *, runnable_init_params: ~typing.Optional[dict[str, typing.Any]] = None, name: ~typing.Optional[str] = None, scheduling_strategy: type[bentoml._internal.runner.strategy.Strategy] = <class 'bentoml._internal.runner.strategy.DefaultStrategy'>, models: ~typing.Optional[list[bentoml._internal.models.model.Model]] = None, max_batch_size: ~typing.Optional[int] = None, max_latency_ms: ~typing.Optional[int] = None, method_configs: ~typing.Optional[dict[str, dict[str, int]]] = None)[source]#
bentoml.Runnable#
- class bentoml.Runnable[source]#
- static method(meth: t.Callable[t.Concatenate[T, P], R], *, batchable: bool = False, batch_dim: tuple[int, int] | int = 0, input_spec: AnyType | tuple[AnyType, ...] | None = None, output_spec: AnyType | None = None) RunnableMethod[T, P, R] [source]#
- static method(meth: None = None, *, batchable: bool = False, batch_dim: tuple[int, int] | int = 0, input_spec: AnyType | tuple[AnyType, ...] | None = None, output_spec: AnyType | None = None) t.Callable[[t.Callable[t.Concatenate[T, P], R]], RunnableMethod[T, P, R]]
Tag#
Model#
- class bentoml.Model(tag: Tag, model_fs: FS, info: ModelInfo, custom_objects: Optional[dict[str, Any]] = None, *, _internal: bool = False)[source]#
- property info: ModelInfo#