CatBoost#

CatBoost is a machine learning algorithm that uses gradient boosting on decision trees. It is available as an open source library. To learn more about CatBoost, visit their documentation.

BentoML provides native support for CatBoost, and this guide provides an overview of how to use BentoML with CatBoost.

Saving a trained CatBoost model#

In this example, we will train a new model using UCI’s breast cancer dataset.

import bentoml

import catboost as cbt

from sklearn.datasets import load_breast_cancer

cancer = load_breast_cancer()

X = cancer.data
y = cancer.target

model = cbt.CatBoostClassifier(
    iterations=2,
    depth=2,
    learning_rate=1,
    loss_function="Logloss",
    verbose=False,
)

# train the model
model.fit(X, y)

Use save_model to save the model instance to BentoML model store:

bento_model = bentoml.catboost.save_model("catboost_cancer_clf", model)

To verify that the saved learner can be loaded properly:

model = bentoml.catboost.load_model("catboost_cancer_clf:latest")

model.predict(cbt.Pool([[1.308e+01, 1.571e+01, 8.563e+01, 5.200e+02, 1.075e-01, 1.270e-01,
    4.568e-02, 3.110e-02, 1.967e-01, 6.811e-02, 1.852e-01, 7.477e-01,
    1.383e+00, 1.467e+01, 4.097e-03, 1.898e-02, 1.698e-02, 6.490e-03,
    1.678e-02, 2.425e-03, 1.450e+01, 2.049e+01, 9.609e+01, 6.305e+02,
    1.312e-01, 2.776e-01, 1.890e-01, 7.283e-02, 3.184e-01, 8.183e-02]]))

Building a Service using CatBoost#

See also

Building a Service: more information on creating a prediction service with BentoML.

import bentoml

import numpy as np

from bentoml.io import NumpyNdarray

runner = bentoml.catboost.get("catboost_cancer_clf:latest").to_runner()

svc = bentoml.Service("cancer_clf", runners=[runner])


@svc.api(input=NumpyNdarray(), output=NumpyNdarray())
async def classify_cancer(input: np.ndarray) -> np.ndarray:
   # returns sentiment score of a given text
   res = await runner.predict.async_run(input)
   return res

When constructing a bentofile.yaml, there are two ways to include CatBoost as a dependency, via python or conda:

python:
  packages:
    - catboost

conda:
  channels:
  - conda-forge
  dependencies:
  - catboost

Using Runners#

See also

See Using Runners doc for a general introduction to the Runner concept and its usage.

A CatBoost Runner can be created as follows:

runner = bentoml.catboost.get("model_name:model_version").to_runner()

runner.predict.run is generally a drop-in replacement for model.predict.

While a Pool can be passed to a CatBoost Runner, BentoML does not support adaptive batching for Pool objects.

To use adaptive batching feature from BentoML, we recommend our users to use either NumPy ndarray or Pandas DataFrame instead.

Note

Currently staged_predict callback is not yet supported with bentoml.catboost.

Using GPU#

CatBoost Runners will automatically use task_type=GPU if a GPU is detected.

This behavior can be disabled using the BentoML configuration file:

access:

runners:
   # resources can be configured at the top level
   resources:
      nvidia.com/gpu: 0
   # or per runner
   my_runner_name:
      resources:
          nvidia.com/gpu: 0

Adaptive batching#

See also

Adaptive Batching: a general introduction to adaptive batching in BentoML.

CatBoost’s model.predict supports taking batch input for inference. This is disabled by default, but can be enabled using the appropriate signature when saving your model.

Note

BentoML does not currently support adaptive batching for Pool input. In order to enable batching, use either a NumPy ndarray or a Pandas DataFrame instead.

bento_model = bentoml.catboost.save_model(
 "catboost_cancer_clf", model, signatures={"predict": {"batchable": True}}
 )