Call Deployment endpoints#

This document explains how to interact with a Deployment by calling its endpoint URL.

Obtain the endpoint URL#

Choose one of the following ways to obtain the endpoint URL.

Install jq, then run the following command.

bentoml deployment get <your_deployment_name> -o json | jq ."endpoint_url"

import bentoml

deployment_info = bentoml.deployment.get('your_deployment_name')
print(deployment_info.endpoint_url)

  1. Navigate to the Deployments page.

  2. Click the desired Deployment.

  3. On the details page, you can find the endpoint URL under the Deployment’s name.

    ../../_images/deployment-endpoint-url.png

Interact with the Deployment#

Choose one of the following ways to access your Deployment with the endpoint URL. The example below shows how to interact with the Summarization Deployment. You can find the corresponding Python code and CURL command for a specific Deployment on the Playground tab of its details page.

curl -X 'POST' \
    'http://<deployment_endpoint_url>/summarize' \
    -H 'accept: text/plain' \
    -H 'Content-Type: application/json' \
    -d '{
    "text": "Your long text to summarize"
    }'

Include the endpoint URL in your client as below. For more information, see Clients.

import bentoml

client = bentoml.SyncHTTPClient(url='<deployment_endpoint_url>')
result: str = client.summarize(text="Your long text to summarize")
print(result)

You can retrieve the information of a client by using get_client or get_async_client (set the token parameter if you enable Authorization), then call its available endpoint to make HTTP requests to your Deployment.

import bentoml

dep = bentoml.deployment.get(name="deploy-1")
# Get synchronous HTTP client for Deployment:
client = dep.get_client()
# Get asynchronous HTTP client for Deployment:
async_client = dep.get_async_client()
# Call the client's endpoint to interact
result = client.summarize(text="Your long text to summarize")

Access the Deployment endpoint URL directly. The Swagger UI dynamically generates documentation and a user interface based on OpenAPI Specifications.

../../_images/swagger-ui.png

Interact with protected endpoints#

If you enable Authorization for a Deployment when creating it, its endpoint URL will be protected. You need to create an API token with Protected Endpoint Access and then use this token to access it.