BentoService API¶
BentoService¶
-
class
bentoml.
BentoService
¶ BentoService is the base component for building prediction services using BentoML.
BentoService provide an abstraction for describing model artifacts and environment dependencies required for a prediction service. And allows users to create inference APIs that defines the inferencing logic and how the underlying model can be served. Each BentoService can contain multiple models and serve multiple inference APIs.
Usage example:
>>> from bentoml import BentoService, env, api, artifacts >>> from bentoml.adapters import DataframeInput >>> from bentoml.frameworks.sklearn import SklearnModelArtifact >>> >>> @artifacts([SklearnModelArtifact('clf')]) >>> @env(pip_packages=["scikit-learn"]) >>> class MyMLService(BentoService): >>> >>> @api(input=DataframeInput(), batch=True) >>> def predict(self, df): >>> return self.artifacts.clf.predict(df) >>> >>> if __name__ == "__main__": >>> bento_service = MyMLService() >>> bento_service.pack('clf', trained_classifier_model) >>> bento_service.save_to_dir('/bentoml_bundles')
-
name
()¶ - Returns
BentoService name
-
versioneer
()¶ Function used to generate a new version string when saving a new BentoService bundle. User can also override this function to get a customized version format
-
set_version
(version_str=None)¶ Set the version of this BentoService instance. Once the version is set explicitly via set_version, the self.versioneer method will no longer be invoked when saving this BentoService.
-
inference_apis
¶ Return a list of user defined API functions
- Returns
List of Inference API objects
- Return type
list(InferenceAPI)
-
artifacts
¶ Returns the ArtifactCollection instance specified with this BentoService class
- Returns
A dictionary of packed artifacts from the artifact name to the BentoServiceArtifact instance
- Return type
artifacts(ArtifactCollection)
-
pack
(name, *args, *kwargs)¶ BentoService#pack method is used for packing trained model instances with a BentoService instance and make it ready for BentoService#save.
- Parameters
name – name of the declared model artifact
args – args passing to the target model artifact to be packed
kwargs – kwargs passing to the target model artifact to be packed
- Returns
this BentoService instance
-
pack
(*args, **kwargs) Deprecated: Legacy BentoService#pack class method, no longer supported
-
save
(yatai_url=None, version=None, labels=None)¶ Save and register this BentoService via BentoML’s built-in model management system. BentoML by default keeps track of all the SavedBundle’s files and metadata in local file system under the $BENTOML_HOME(~/bentoml) directory. Users can also configure BentoML to save their BentoService to a shared Database and cloud object storage such as AWS S3.
- Parameters
yatai_url – optional - URL path to Yatai server
version – optional - save with version override
labels – optional - labels dictionary
- Returns
saved_path: file path to where the BentoService is saved
-
save_to_dir
(path, version=None)¶ Save this BentoService along with all its artifacts, source code and dependencies to target file path, assuming path exist and empty. If target path is not empty, this call may override existing files in the given path.
- Parameters
(str) (path) – Destination of where the bento service will be saved
version – optional - save with version override
-
api¶
-
bentoml.
api
(*args, input: Optional[bentoml.adapters.base_input.BaseInputAdapter] = None, output: Optional[bentoml.adapters.base_output.BaseOutputAdapter] = None, api_name: Optional[str] = None, api_doc: Optional[str] = None, mb_max_batch_size: int = 2000, mb_max_latency: int = 10000, batch=False, **kwargs)¶ A decorator exposed as bentoml.api for defining Inference API in a BentoService class.
- Parameters
input – InputAdapter instance of the inference API
output – OutputAdapter instance of the inference API
api_name – API name, default to the user-defined callback function’s function name
api_doc – user-facing documentation of the inference API. default to the user-defined callback function’s docstring
mb_max_batch_size – The maximum size of requests batch accepted by this inference API. This parameter governs the throughput/latency trade off, and avoids having large batches that exceed some resource constraint (e.g. GPU memory to hold the entire batch’s data). Default: 1000.
mb_max_latency – The latency goal of this inference API in milliseconds. Default: 10000.
Example usage:
>>> from bentoml import BentoService, api >>> from bentoml.adapters import JsonInput, DataframeInput >>> >>> class FraudDetectionAndIdentityService(BentoService): >>> >>> @api(input=JsonInput(), batch=True) >>> def fraud_detect(self, json_list): >>> # user-defined callback function that process inference requests >>> >>> @api(input=DataframeInput(input_json_orient='records'), batch=True) >>> def identity(self, df): >>> # user-defined callback function that process inference requests
env¶
-
bentoml.
env
(pip_dependencies: Optional[List[str]] = None, pip_packages: Optional[List[str]] = None, pip_index_url: Optional[str] = None, pip_trusted_host: Optional[str] = None, pip_extra_index_url: Optional[str] = None, auto_pip_dependencies: Optional[bool] = None, infer_pip_packages: bool = False, requirements_txt_file: Optional[str] = None, conda_channels: Optional[List[str]] = None, conda_overwrite_channels: bool = False, conda_dependencies: Optional[List[str]] = None, conda_env_yml_file: Optional[str] = None, setup_sh: Optional[str] = None, docker_base_image: Optional[str] = None, zipimport_archives: Optional[List[str]] = None)¶ Define environment and dependencies required for the BentoService being created
- Parameters
pip_packages: – list of pip_packages required: or with specified version {package_name}=={package_version}
by package name (specified) – or with specified version {package_name}=={package_version}
pip_dependencies – same as pip_packages but deprecated
pip_index_url – passing down to pip install –index-url option
pip_trusted_host – passing down to pip install –trusted-host option
pip_extra_index_url – passing down to pip install –extra-index-url option
infer_pip_packages – whether to automatically find all the required pip dependencies and pin their version
auto_pip_dependencies – same as infer_pip_packages but deprecated
requirements_txt_file – pip dependencies in the form of a requirements.txt file, this can be a relative path to the requirements.txt file or the content of the file
conda_channels – list of extra conda channels to be used
conda_overwrite_channels – Turn on to make conda_channels overwrite the list of channels instead of adding to it
conda_dependencies – list of conda dependencies required
conda_env_yml_file – use a pre-defined conda environment yml file
setup_sh – user defined setup bash script, it is executed in docker build time
docker_base_image – used for customizing the docker container image built with BentoML saved bundle. Base image must either have both bash and conda installed; or have bash, pip, python installed, in which case the user is required to ensure the python version matches the BentoService bundle
zipimport_archives – list of zipimport archives paths relative to the module path
artifacts¶
-
bentoml.
artifacts
(artifacts: List[bentoml.service.artifacts.BentoServiceArtifact])¶ Define artifacts required to be bundled with a BentoService
- Parameters
artifacts (list(bentoml.artifact.BentoServiceArtifact)) – A list of desired artifacts required by this BentoService
ver¶
-
bentoml.
ver
(major, minor)¶ Decorator for specifying the version of a custom BentoService.
- Parameters
major (int) – Major version number for Bento Service
minor (int) – Minor version number for Bento Service
BentoML uses semantic versioning for BentoService distribution:
MAJOR is incremented when you make breaking API changes
MINOR is incremented when you add new functionality without breaking the existing API or functionality
PATCH is incremented when you make backwards-compatible bug fixes
‘Patch’ is provided(or auto generated) when calling BentoService#save, while ‘Major’ and ‘Minor’ can be defined with ‘@ver’ decorator
>>> from bentoml import ver, artifacts >>> from bentoml.service.artifacts.common import PickleArtifact >>> >>> @ver(major=1, minor=4) >>> @artifacts([PickleArtifact('model')]) >>> class MyMLService(BentoService): >>> pass >>> >>> svc = MyMLService() >>> svc.pack("model", trained_classifier) >>> svc.set_version("2019-08.iteration20") >>> svc.save() >>> # The final produced BentoService bundle will have version: >>> # "1.4.2019-08.iteration20"
save¶
-
bentoml.
save
(bento_service, base_path=None, version=None, labels=None)¶ Save and register the given BentoService via BentoML’s built-in model management system. BentoML by default keeps track of all the SavedBundle’s files and metadata in local file system under the $BENTOML_HOME(~/bentoml) directory. Users can also configure BentoML to save their BentoService to a shared Database and cloud object storage such as AWS S3. :param bento_service: target BentoService instance to be saved :param base_path: optional - override repository base path :param version: optional - save with version override :param labels: optional - user defined labels :return: saved_path: file path to where the BentoService is saved
load¶
-
bentoml.
load
(bundle_path)¶ Load bento service from local file path or s3 path
- Parameters
bundle_path (str) – The path that contains saved BentoService bundle, supporting both local file path and s3 path
- Returns
a loaded BentoService instance
- Return type
bentoml.service.BentoService
save_to_dir¶
-
bentoml.
save_to_dir
(bento_service, path, version=None, silent=False)¶ Save given BentoService along with all its artifacts, source code and dependencies to target file path, assuming path exist and empty. If target path is not empty, this call may override existing files in the given path.
- Parameters
(bentoml.service.BentoService) (bento_service) – a Bento Service instance
(str) (version) – Destination of where the bento service will be saved. The destination can be local path or remote path. The remote path supports both AWS S3(‘s3://bucket/path’) and Google Cloud Storage(‘gs://bucket/path’).
(str) – Override the service version with given version string
(boolean) (silent) – whether to hide the log message showing target save path
load_from_dir¶
-
bentoml.
load_from_dir
(bundle_path)¶ Load bento service from local file path or s3 path
- Parameters
bundle_path (str) – The path that contains saved BentoService bundle, supporting both local file path and s3 path
- Returns
a loaded BentoService instance
- Return type
bentoml.service.BentoService