Detectron2

Users can now use Detectron2 with BentoML with the following API: load, save, and load_runner as follow:

import numpy as np
import torch
import pytest
import imageio
from detectron2 import model_zoo
from detectron2.data import transforms as T
from detectron2.config import get_cfg
from detectron2.modeling import build_model

import bentoml

def detectron_model_and_config() -> t.Tuple[torch.nn.Module, "CfgNode"]:
    model_url: str = "COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml"

    cfg: "CfgNode" = get_cfg()
    cfg.merge_from_file(model_zoo.get_config_file(model_url))
    # set threshold for this model
    cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.5
    cfg.MODEL.WEIGHTS = model_zoo.get_checkpoint_url(model_url)

    cloned = cfg.clone()
    cloned.MODEL.DEVICE = "cpu"  # running on CI
    model: torch.nn.Module = build_model(cloned)
    model.eval()

    return model, cfg

# `save` a given classifier and retrieve coresponding tag:
model, config = detectron_model_and_config()
tag_info = bentoml.detectron.save(
    "mask_rcnn_R_50_FPN_3x",
    model,
    model_config=config,
)

# retrieve metadata with `bentoml.models.get`:
metadata = bentoml.models.get(tag)

# `load` the model back in memory:
model = bentoml.detectron2.load("mask_rcnn_R_50_FPN_3x")

def prepare_image(
    original_image: "np.ndarray[t.Any, np.dtype[t.Any]]",
) -> "np.ndarray[t.Any, np.dtype[t.Any]]":
    """Mainly to test on COCO dataset"""
    _aug = T.ResizeShortestEdge([800, 800], 1333)

    image = _aug.get_transform(original_image).apply_image(original_image)
    return image.transpose(2, 0, 1)

# Run a given model under `Runner` abstraction with `load_runner`
runner = bentoml.detectron2.load_runner(tag)
image = torch.as_tensor(prepare_image(np.asarray(imageio.imread(IMAGE_URL))))
res = runner.run_batch(image)
runner.run_batch(imarr)

Note

You can find more examples for Detectron2 in our gallery repo.

bentoml.detectron.save(name, model, *, model_config=None, labels=None, custom_objects=None, metadata=None)

Save a model instance to BentoML modelstore.

Parameters
  • name (str) – Name for given model instance. This should pass Python identifier check.

  • model (torch.nn.Module) – Instance of detectron2 model to be saved.

  • model_config (detectron2.config.CfgNode, optional, default to None) – model config from detectron2.model_zoo.get_config_file()

  • labels (Dict[str, str], optional, default to None) – user-defined labels for managing models, e.g. team=nlp, stage=dev

  • custom_objects (Dict[str, Any]], optional, default to None) – user-defined additional python objects to be saved alongside the model, e.g. a tokenizer instance, preprocessor function, model configuration json

  • metadata (Dict[str, Any], optional, default to None) – Custom metadata for given model.

  • model_store (~bentoml._internal.models.ModelStore, default to BentoMLContainer.model_store) – BentoML modelstore, provided by DI Container.

Returns

A tag with a format name:version where name is the user-defined model’s name, and a generated version by BentoML.

Return type

Tag

Examples:

import bentoml

# import some common detectron2 utilities
import detectron2
from detectron2 import model_zoo
from detectron2.config import get_cfg
from detectron2.modeling import build_model

model_url: str = "COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml"
cfg: "CfgNode" = get_cfg()
cfg.merge_from_file(model_zoo.get_config_file(model_url))
# set threshold for this model
cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.5
cfg.MODEL.WEIGHTS = model_zoo.get_checkpoint_url(model_url)
cloned = cfg.clone()
cloned.MODEL.DEVICE = "cpu"
model: torch.nn.Module = build_model(cloned)

tag = bentoml.detectron.save(
    "my_detectron_model",
    model,
    model_config=cfg,
)

# load the model back:
loaded = bentoml.detectron.load("my_detectron_model:latest")
# or:
loaded = bentoml.detectron.load(tag)
bentoml.detectron.load(tag, device='cpu', model_store=<simple_di.providers.SingletonFactory object>)

Load a model from BentoML local modelstore with given tag.

Parameters
  • tag (Union[str, Tag]) – Tag of a saved model in BentoML local modelstore.

  • device (str, optional, default to cpu) – Device type to cast model. Default behaviour similar to torch.device("cuda") Options: “cuda” or “cpu”. If None is specified then return default config.MODEL.DEVICE

  • model_store (~bentoml._internal.models.ModelStore, default to BentoMLContainer.model_store) – BentoML modelstore, provided by DI Container.

Returns

an instance of torch.nn.Module

Return type

torch.nn.Module

Examples:

import bentoml
model = bentoml.detectron.load("my_detectron_model")
bentoml.detectron.load_runner(tag, predict_fn_name='__call__', *, name=None)

Runner represents a unit of serving logic that can be scaled horizontally to maximize throughput. bentoml.detectron.load_runner() implements a Runner class that wrap around a torch.nn.Module model, which optimize it for the BentoML runtime.

Parameters
  • tag (Union[str, Tag]) – Tag of a saved model in BentoML local modelstore.

  • predict_fn_name (str, default to __call__) – Options for inference functions. Default to __call__

  • model_store (~bentoml._internal.models.ModelStore, default to BentoMLContainer.model_store) – BentoML modelstore, provided by DI Container.

Returns

Runner instances for bentoml.detectron model

Return type

Runner

Examples:

import bentoml
import numpy as np

runner = bentoml.detectron.load_runner(tag)
runner.run_batch(np.array([[1,2,3,]]))