Deploying to Azure Functions

Azure Functions is an event driven, compute-on-demand cloud service offered by Microsoft. Its serverless execution model along with the ability to bring your own container allow users deploy a BentoML API server with full functionality that scales base on usage.

In this guide, it will deploy an IrisClassifier BentoService to Azure Functions, make a prediction request to the deployment endpoint, and then delete the deployment. It will also show how to start an remote YataiService with Azure credential for deploying to the Azure Functions.

Prerequisites:

Azure Functions deployment with BentoML

Run the example project from the Quickstart guide to create an IrisClassifier BentoService saved bundle for the deployment:

git clone git@github.com:bentoml/BentoML.git
python ./bentoml/guides/quick-start/main.py

Use BentoML CLI tool to get the information of the IrisClassifier BentoService created above

$ bentoml get IrisClassifier:latest

# Sample output
{
  "name": "IrisClassifier",
  "version": "20200618124555_310CE0",
  "uri": {
    "type": "LOCAL",
    "uri": "/Users/bozhaoyu/bentoml/repository/IrisClassifier/20200618124555_310CE0"
  },
  "bentoServiceMetadata": {
    "name": "IrisClassifier",
    "version": "20200618124555_310CE0",
    "createdAt": "2020-06-18T19:46:18.675900Z",
    "env": {
      "condaEnv": "name: bentoml-IrisClassifier\nchannels:\n- defaults\ndependencies:\n- python=3.7.3\n- pip\n",
      "pipDependencies": "scikit-learn\npandas\nbentoml==0.8.1",
      "pythonVersion": "3.7.3",
      "dockerBaseImage": "bentoml/model-server:0.8.1"
    },
    "artifacts": [
      {
        "name": "model",
        "artifactType": "SklearnModelArtifact"
      }
    ],
    "apis": [
      {
        "name": "predict",
        "inputType": "DataframeInput",
        "docs": "BentoService API",
        "inputConfig": {
          "orient": "records",
          "typ": "frame",
          "is_batch_input": true,
          "input_dtypes": null
        },
        "outputConfig": {
          "cors": "*"
        },
        "outputType": "DefaultOutput",
        "mbMaxLatency": 300,
        "mbMaxBatchSize": 2000
      }
    ]
  }
}

Deploy BentoService to Azure Functions

Deploy to Azure Functions requires a deployment name, BentoService in name:version format, and a valid Azure location. You can find list of Azure locations by calling command az account list-locations.

BentoML will create necessary Azure resources, and then build and deploy your BentoService as docker image to Azure Functions.

$ bentoml azure-functions deploy iris-classifier-demo -b IrisClassifier:20200622131825_5788D9 --location westus

# sample output
[2020-06-22 13:59:22,881] INFO - ApplyDeployment (iris-classifier-demo, namespace dev) succeeded
-Successfully created Azure Functions deployment iris-classifier-demo
{
  "namespace": "dev",
  "name": "iris-classifier-demo",
  "spec": {
    "bentoName": "IrisClassifier",
    "bentoVersion": "20200622131825_5788D9",
    "operator": "AZURE_FUNCTIONS",
    "azureFunctionsOperatorConfig": {
      "location": "westus",
      "premiumPlanSku": "EP1",
      "minInstances": 1,
      "maxBurst": 20,
      "functionAuthLevel": "anonymous"
    }
  },
  "state": {
    "state": "RUNNING",
    "infoJson": {
      "defaultHostName": "dev-iris-classifier-demo.azurewebsites.net",
      "enabledHostNames": [
        "dev-iris-classifier-demo.azurewebsites.net",
        "dev-iris-classifier-demo.scm.azurewebsites.net"
      ],
      "hostNames": [
        "dev-iris-classifier-demo.azurewebsites.net"
      ],
      "id": "/subscriptions/f01c41a2-72ba-480c-99a6-b3241fada0ac/resourceGroups/dev-iris-classifier-demo/providers/Microsoft.Web/sites/dev-iris-classifier-demo",
      "kind": "functionapp,linux,container",
      "lastModifiedTimeUtc": "2020-06-22T20:59:22.350000",
      "location": "West US",
      "name": "dev-iris-classifier-demo",
      "repositorySiteName": "dev-iris-classifier-demo",
      "reserved": true,
      "resourceGroup": "dev-iris-classifier-demo",
      "state": "Running",
      "type": "Microsoft.Web/sites",
      "usageState": "Normal"
    },
    "timestamp": "2020-06-22T20:59:30.428159Z"
  },
  "createdAt": "2020-06-22T20:53:26.607038Z",
  "lastUpdatedAt": "2020-06-22T20:53:26.607073Z"
}

Use bentoml azure-functions get command to retrieve the latest status information of the deployment.

$ bentoml azure-functions get iris-classifier-demo

# Sample output
{
  "namespace": "dev",
  "name": "iris-classifier-demo",
  "spec": {
    "bentoName": "IrisClassifier",
    "bentoVersion": "20200622131825_5788D9",
    "operator": "AZURE_FUNCTIONS",
    "azureFunctionsOperatorConfig": {
      "location": "westus",
      "premiumPlanSku": "EP1",
      "minInstances": 1,
      "maxBurst": 20,
      "functionAuthLevel": "anonymous"
    }
  },
  "state": {
    "state": "RUNNING",
    "infoJson": {
      "defaultHostName": "dev-iris-classifier-demo.azurewebsites.net",
      "enabledHostNames": [
        "dev-iris-classifier-demo.azurewebsites.net",
        "dev-iris-classifier-demo.scm.azurewebsites.net"
      ],
      "hostNames": [
        "dev-iris-classifier-demo.azurewebsites.net"
      ],
      "id": "/subscriptions/f01c41a2-72ba-480c-99a6-b3241fada0ac/resourceGroups/dev-iris-classifier-demo/providers/Microsoft.Web/sites/dev-iris-classifier-demo",
      "kind": "functionapp,linux,container",
      "lastModifiedTimeUtc": "2020-06-22T20:59:22.350000",
      "location": "West US",
      "name": "dev-iris-classifier-demo",
      "repositorySiteName": "dev-iris-classifier-demo",
      "reserved": true,
      "resourceGroup": "dev-iris-classifier-demo",
      "state": "Running",
      "type": "Microsoft.Web/sites",
      "usageState": "Normal"
    },
    "timestamp": "2020-06-22T21:04:59.779887Z"
  },
  "createdAt": "2020-06-22T20:53:26.607038Z",
  "lastUpdatedAt": "2020-06-22T20:53:26.607073Z"
}

During Azure Functions initialized stage, it takes a while to download the docker image. Please wait few minutes before visiting your deployment.

With the default authorization setting, your Azure Functions deployment is visible to all. Open your browser and visit the URL in hostNames. You should see the web UI provided by BentoML API server.

To test the prediction API in the Azure Functions deployment, you could use the web ui mentioned above or you could make a curl request to the endpoint.

$  curl -i --request POST --header "Content-Type: application/json" \
    --data '[[5.1, 3.5, 1.4, 0.2]]' \
    "https://dev-iris-classifier-demo.azurewebsites.net/predict"

# Sample output

HTTP/1.1 200 OK
Content-Length: 3
Content-Type: application/json
Server: Kestrel
Request-Context: appId=cid-v1:1f23e525-f1cd-471a-ae47-e313f784b99e
request_id: 525a5c94-41a8-4d9f-9259-0216d3ceb465
Date: Mon, 22 Jun 2020 21:19:40 GMT

[0]%

Use bentoml azure-functions list to display all active deployments to Azure Functions

$ bentoml azure-functions list

# Sample output
NAME                  NAMESPACE    PLATFORM         BENTO_SERVICE                         STATUS    AGE
iris-classifier-demo  dev          azure-functions  IrisClassifier:20200622131825_5788D9  running   26 minutes and 24.49 seconds

Update existing Azure Functions deployment

To update an active Azure Function deployment use bentoml azure-functions update command.

Remove Azure Functions deployment

$ bentoml azure-functions delete iris-classifier-demo

Deploy and manage Azure Functions deployment with remote YataiService

BentoML recommends to use remote YataiService for managing and deploying BentoService when you are working in a team. To deploy Azure Functions in remote YataiService, you need to provide the Azure credential for it.

After Sign in with Azure CLI in your local machine, you should be able to find the accessTokens.json in your Azure directory. Now start the BentoML YataiService docker image and mount that accessTokens.json file to the running container.

$ docker run -v ~/.azure/accessTokens.json:/home/.azure/accessTokens.json -p 50051:50051 -p 3000:3000 bentoml/yatai-service:latest

After the YataiService docker container is running, in another terminal window, set yatai service address with bentoml config set

$ bentoml config set yatai_service.url=127.0.0.1:50051

Deploy and manage Azure Functions deployment with Kubernetes

Create a Kubernetes secret base on the accessTokens.json

$ kubectl create secret generic azure-access-tokens --from-file=~/.azure/accessTokens.json

Confirm the secrete is created successfully by using kubectl describe command

$kubectl describe secret azure-access-tokens

Copy and paste the code below into a file named yatai-service.yaml

apiVersion: v1
kind: Service
metadata:
  labels:
    app: yatai-service
  name: iris-classifier
  name: yatai-service
spec:
  ports:
  - name: grpc
    port: 50051
    targetPort: 50051
  - name: web
    port: 3000
    targetPort: 3000
  selector:
    app: yatai-service
  type: LoadBalancer
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: yatai-service
  name: yatai-service
spec:
  selector:
    matchLabels:
      app: yatai-service
  template:
    metadata:
      labels:
        app: yatai-service
    spec:
      containers:
      - image: bentoml/yatai-service
        imagePullPolicy: IfNotPresent
        name: yatai-service
        ports:
        - containerPort: 50051
        - containerPort: 3000
        volumeMounts:
        - mountPath: "/home/.azure"
          name: azure-access-tokens
          readOnly: true
      volumes:
      - name: azure-access-tokens
        secret:
            secretName: azure-access-tokens

Run kubectl apply command to deploy Yatai service to the Kubernetes cluster

$ kubectl apply -f yatai-service.yaml