BentoService API


class bentoml.BentoService

BentoService packs a list of artifacts and exposes service APIs for BentoAPIServer and BentoCLI to execute. By subclassing BentoService, users can customize the artifacts and environments required for a ML service.

>>>  from bentoml import BentoService, env, api, artifacts, ver
>>>  from bentoml.handlers import DataframeHandler
>>>  from bentoml.artifact import SklearnModelArtifact
>>>  @ver(major=1, minor=4)
>>>  @artifacts([SklearnModelArtifact('clf')])
>>>  @env(pip_dependencies=["scikit-learn"])
>>>  class MyMLService(BentoService):
>>>     @api(DataframeHandler)
>>>     def predict(self, df):
>>>         return self.artifacts.clf.predict(df)
>>>  bento_service = MyMLService()
>>>  bento_service.pack('clf', trained_classifier_model)
>>>  bento_service.save_to_dir('/bentoml_bundles')

return BentoService name


Function used to generate a new version string when saving a new BentoService bundle. User can also override this function to get a customized version format


Manually override the version of this BentoService instance


Return a list of user defined API functions


List of user defined API functions

Return type



bentoml.api(handler_cls, *args, **kwargs)

Decorator for adding api to a BentoService

  • handler_cls (bentoml.handlers.BentoHandler) – The handler class for the API function.

  • api_name (str, optional) – API name to replace function name

  • api_doc (str, optional) – Docstring for API function

  • **kwargs – Additional keyword arguments for handler class. Please reference to what arguments are available for the particular handler


InvalidArgument – API name must contains only letters

>>> from bentoml import BentoService, api
>>> from bentoml.handlers import JsonHandler, DataframeHandler
>>> class FraudDetectionAndIdentityService(BentoService):
>>>     @api(JsonHandler)
>>>     def fraud_detect(self, parsed_json):
>>>         # do something
>>>     @api(DataframeHandler, input_json_orient='records')
>>>     def identity(self, df):
>>>         # do something


bentoml.env(setup_sh=None, pip_dependencies=None, auto_pip_dependencies=False, conda_channels=None, conda_dependencies=None)

Define environment and dependencies required for the BentoService being created

  • setup_sh (str) – bash script for initializing docker environment before loading the BentoService for inferencing

  • pip_dependencies (list(str)) – List of python PyPI package dependencies

  • auto_pip_dependencies (bool) – Flag to tell bentoml auto seek package dependencies for project or not

  • conda_channels (list(str)) – List of conda channels required for conda dependencies

  • conda_dependencies (list(str)) – List of conda dependencies required



Define artifacts required to be bundled with a BentoService


artifacts (list(bentoml.artifact.BentoServiceArtifact)) – A list of desired artifacts required by this BentoService


bentoml.ver(major, minor)

Decorator for specifying the version of a custom BentoService.

  • major (int) – Major version number for Bento Service

  • minor (int) – Minor version number for Bento Service

BentoML uses semantic versioning for BentoService distribution:

  • MAJOR is incremented when you make breaking API changes

  • MINOR is incremented when you add new functionality without breaking the existing API or functionality

  • PATCH is incremented when you make backwards-compatible bug fixes

‘Patch’ is provided(or auto generated) when calling BentoService#save, while ‘Major’ and ‘Minor’ can be defined with @ver’ decorator

>>>  @ver(major=1, minor=4)
>>>  @artifacts([PickleArtifact('model')])
>>>  class MyMLService(BentoService):
>>>     pass
>>>  svc = MyMLService()
>>>  svc.pack("model", trained_classifier)
>>>  svc.set_version("2019-08.iteration20")
>>>  # The final produced BentoService bundle will have version:
>>>  # "1.4.2019-08.iteration20"

save, base_path=None, version=None)


bentoml.save_to_dir(bento_service, path, version=None)

Save given BentoService along with all its artifacts, source code and dependencies to target file path, assuming path exist and empty. If target path is not empty, this call may override existing files in the given path.

  • bento_service (bentoml.service.BentoService) – a Bento Service instance

  • path (str) – Destination of where the bento service will be saved



Load bento service from local file path or s3 path


bundle_path (str) – The path that contains saved BentoService bundle, supporting both local file path and s3 path


a loaded BentoService instance

Return type