Client API#

You can create a BentoML client to interact with your BentoML Service.

bentoml.SyncHTTPClient#

class bentoml.SyncHTTPClient(url: str, *, token: str | None = None, timeout: float = 30)[source]#

A synchronous client for BentoML service.

Parameters:
  • url (str) – URL of the BentoML service.

  • token (str, optional) – Authentication token. Defaults to None.

  • timeout (float, optional) – Timeout for the client. Defaults to 30.

Example:

with SyncHTTPClient("http://localhost:3000") as client:
    resp = client.call("classify", input_series=[[1,2,3,4]])
    assert resp == [0]
    # Or using named method directly
    resp = client.classify(input_series=[[1,2,3,4]])
    assert resp == [0]
client_cls#

alias of Client

is_ready(timeout: int | None = None) bool[source]#

bentoml.AsyncHTTPClient#

class bentoml.AsyncHTTPClient(url: str, *, token: str | None = None, timeout: float = 30)[source]#

An asynchronous client for BentoML service.

Parameters:
  • url (str) – URL of the BentoML service.

  • token (str, optional) – Authentication token. Defaults to None.

  • timeout (float, optional) – Timeout for the client. Defaults to 30.

Example:

async with AsyncHTTPClient("http://localhost:3000") as client:
    resp = await client.call("classify", input_series=[[1,2,3,4]])
    assert resp == [0]
    # Or using named method directly
    resp = await client.classify(input_series=[[1,2,3,4]])
    assert resp == [0]

    # Streaming
    resp = client.stream(prompt="hello")
    async for data in resp:
        print(data)
client_cls#

alias of AsyncClient

async is_ready(timeout: int | None = None) bool[source]#