Yatai Client

A Yatai RPC Server is a stateful service that provides a complete BentoML model management and model serving/deployment workflow.

Two sets of APIs are provided:

  • BentoRepositoryAPIClient (via YataiClient.repository) manages saved BentoService bundle, making them available for serving in production environments.

  • DeploymentAPIClient (via YataiClient.deployment) deploys BentoServices to a variety of different cloud platforms, track deployment status, set up logging monitoring for your model serving workload.

Note

We want to provide a better documentation on using DeploymentAPIClient programmatically. For now refer to deployment_api.py or using the CLI commands bentoml deployment

bentoml.yatai.client.get_yatai_client(yatai_url: Optional[str] = None) → bentoml.yatai.client.YataiClient
Parameters

yatai_url (str) – Yatai Service URL address.

Returns

YataiClient, a python client to interact with Yatai gRPC server.

Example:

from bentoml.yatai.client import get_yatai_client

custom_url = 'https://remote.yatai:50050'
yatai_client = get_yatai_client(custom_url)
class bentoml.yatai.client.bento_repository_api.BentoRepositoryAPIClient(yatai_service)

Provided API to manage BentoService for local Yatai repository

containerize(bento: str, tag: str, build_args: Optional[Dict[str, str]] = None, push: bool = False) → str

Create a container image from a BentoService

Parameters

  • bento (str) – A BentoService identifier with NAME:VERSION format.

  • tag (str) – BentoService tag.

  • build_args (Dict[str, str], optional) – Build args to parse to docker build

  • push (bool, optional) – Whether to push built container to remote YataiService.

Returns

str representing the image tag of the containerized BentoService

Raises

BentoMLException – bento is missing or incorrect format.

Example:

svc = MyModelService()
svc.save()

from bentoml.yatai.client import get_yatai_client

yatai_client = get_yatai_client()
tag = yatai_client.repository.containerize(f'{svc.name}:{svc.version}')
delete(bento_tag: Optional[str] = None, labels: Optional[str] = None, bento_name: Optional[str] = None, bento_version: Optional[str] = None, prune: Optional[bool] = False, require_confirm: Optional[bool] = False) → None

Delete bentos that matches the specified criteria.

Parameters

  • bento_tag (str, optional) – BentoService tags

  • labels (str, optional) – BentoService labels

  • bento_name (str, optional) – given name of BentoService

  • bento_version (str, optional) – versions of given BentoService

  • prune (bool, optional) – Whether to delete all BentoService, default to False

  • require_confirm (bool, optional) – Requires to confirm delete, default to False

Raises

BentoMLException – bento_tag, bento_name and bento_version are parsed at the same time

Example:

from bentoml.yatai.client import get_yatai_client

yatai_client = get_yatai_client()
# Delete all bento services
yatai_client.repository.delete(prune=True)
# Delete bento service with name is `IrisClassifier` and version `0.1.0`
yatai_client.repository.delete(
    bento_name='IrisClassifier', bento_version='0.1.0'
)
# or use bento tag
yatai_client.repository.delete('IrisClassifier:v0.1.0')
# Delete all bento services with name 'MyService`
yatai_client.repository.delete(bento_name='MyService')
# Delete all bento services with labels match `ci=failed` and `cohort=20`
yatai_client.repository.delete(labels='ci=failed, cohort=20')
get(bento: str) → Bento
Parameters

bento (str) – A BentoService identifier in the format of NAME:VERSION

Returns

BentoService metadata from Yatai RPC server.

Raises

BentoMLException – bento is missing or have invalid format.

Example:

from bentoml.yatai.client import get_yatai_client
yatai_client = get_yatai_client()
bento_info = yatai_client.repository.get('my_service:version')
list(bento_name: str = None, offset: int = None, limit: int = None, order_by: str = None, ascending_order: bool = None, labels: str = None) → List[Bento]

List BentoServices that satisfy the specified criteria.

Parameters

  • bento_name (str) – BentoService name

  • offset (int) – offset of results

  • limit (int) – maximum number of returned results

  • labels (str) – sorted by given labels

  • order_by (str) – orders retrieved BentoService by created_at or name

  • ascending_order (bool) – direction of results order

Returns

lists of BentoService metadata.

Example:

from bentoml.yatai.client import get_yatai_client
yatai_client = get_yatai_client()
bentos_info_list = yatai_client.repository.list(labels='key=value,key2=value')
load(bento: str) → BentoService

Load BentoService from nametag identifier or from a bento bundle path.

Parameters

bento (str) – BentoService identifier or bundle path. Note that nametag identifier will have the following format: NAME:VERSION

Returns

BentoService

Example:

from bentoml.yatai.client import get_yatai_client
yatai_client = get_yatai_client()

# Load BentoService bases on bento tag.
bento_from_name = yatai_client.repository.load('service_name:version')
# Load BentoService from bento bundle path
bento_from_path = yatai_client.repository.load('/path/to/bento/bundle')
# Load BentoService from s3 storage
bento_from_reg = yatai_client.repository.load('s3://bucket/path/bundle.tar.gz')
pull(bento: str) → repository_pb2.BentoUri

Pull a BentoService from remote Yatai. The BentoService will then be saved and registered to local Yatai.

Parameters

bento (str) – a BentoService identifier in the form of NAME:VERSION

Returns

URI as gRPC stub for save location of BentoService.

Return type

reflection.GeneratedProtocolMessageType

Example:

from bentoml.yatai.client import get_yatai_client
client = get_yatai_client('127.0.0.1:50051')
saved_path = client.repository.pull('MyService:20210808_E38F3')
push(bento: str, with_labels: bool = True) → repository_pb2.BentoUri

Push a local BentoService to a remote yatai server.

Parameters

  • bento (str) – A BentoService identifier in the format of NAME:VERSION.

  • with_labels (bool, optional) – Whether to push BentoService with given label.

Returns: URI as gRPC stub for BentoService saved location.

Example:

from bentoml.yatai.client import get_yatai_client
svc = MyBentoService()
svc.save()

remote_yatai_client = get_yatai_client('http://remote.yatai.service:50050')
bento_name = f'{svc.name}:{svc.version}'
remote_saved_path= remote_yatai_client.repository.push(bento_name)
upload(bento_service: BentoService, version: str = None, labels: Dict = None) → BentoUri

Save and upload given BentoService to YataiService.

Parameters

  • bento_service (BentoService) – a BentoService instance

  • version (str, optional) – version of bento_service

  • labels (dict, optional) – BentoService metadata

Returns

BentoUri as gRPC stub for save location of BentoService.

Example:

from bentoml.yatai.client import get_yatai_client

svc = MyBentoService()
svc.save()

remote_yatai_client = get_yatai_client('https://remote.yatai.service:50050')
remote_path = remote_yatai_client.repository.upload(svc)