SpaCy¶

Users can now use SpaCy with BentoML with the following API: load, save, and load_runner as follow:

import bentoml
import spacy as cbt

from sklearn.datasets import load_breast_cancer

cancer = load_breast_cancer()

X = cancer.data
y = cancer.target

clf = cbt.SpaCyClassifier(
    iterations=2,
    depth=2,
    learning_rate=1,
    loss_function="Logloss",
    verbose=False,
)

# train the model
clf.fit(X, y)

# `save` a given classifier and retrieve coresponding tag:
tag = bentoml.spacy.save("cancer_clf", clf)

# retrieve metadata with `bentoml.models.get`:
metadata = bentoml.models.get(tag)

# `load` the model back in memory:
model = bentoml.spacy.load("cancer_clf:latest")

# Run a given model under `Runner` abstraction with `load_runner`
input_data = pd.from_csv("/path/to/csv")
runner = bentoml.spacy.load_runner("cancer_clf:latest")
runner.run(cbt.Pool(input_data))

Note

You can find more examples for SpaCy in our gallery repo.

bentoml.spacy.save(name, model, *, labels=None, custom_objects=None, metadata=None)¶

Save a model instance to BentoML modelstore.

Parameters

name (str) – Name for given model instance. This should pass Python identifier check.
model (spacy.language.Language) – Instance of model to be saved
labels (Dict[str, str], optional, default to None) – user-defined labels for managing models, e.g. team=nlp, stage=dev
custom_objects (Dict[str, Any]], optional, default to None) – user-defined additional python objects to be saved alongside the model, e.g. a tokenizer instance, preprocessor function, model configuration json
metadata (Dict[str, Any], optional, default to None) – Custom metadata for given model.
model_store (ModelStore, default to BentoMLContainer.model_store) – BentoML modelstore, provided by DI Container.

Returns

A tag with a format name:version where name is the user-defined model’s name, and a generated version by BentoML.

Return type

Tag

Examples:

import spacy
import bentoml.spacy
nlp = spacy.load("en_core_web_trf")

# custom training or layers here

bentoml.spacy.save("spacy_roberta", nlp)

bentoml.spacy.load(tag, model_store=<simple_di.providers.SingletonFactory object>, vocab=True, disable=(), exclude=(), config=None)¶

Load a model from BentoML local modelstore with given name.

Parameters

tag (Union[str, Tag]) – Tag of a saved model in BentoML local modelstore.
model_store (ModelStore, default to BentoMLContainer.model_store) – BentoML modelstore, provided by DI Container.
vocab (Union[spacy.vocab.Vocab, bool], optional, defaults to True) – Optional vocab to pass in on initialization. If True, a new Vocab object will be created.
disable (Sequence[str], optional) – Names of pipeline components to disable.
exclude (Sequence[str], optional) – Names of pipeline components to exclude. Excluded components won’t be loaded.
config (Union[Dict[str, Any], spacy.Config], optional) – Config overrides as nested dict or dict keyed by section values in dot notation.

Returns

an instance of spacy.Language from BentoML modelstore.

Return type

spacy.language.Language

Examples:

import bentoml

model = bentoml.spacy.load('custom_roberta')

bentoml.spacy.projects(save_name, tasks, name=None, repo_or_store=None, remotes_config=None, *, branch=None, sparse_checkout=False, verbose=True, metadata=None, model_store=<simple_di.providers.SingletonFactory object>)¶

Enables users to use spacy cli and integrate SpaCy Projects to BentoML.

Parameters

save_name (str) – Name for given model instance. This should pass Python identifier check.
tasks (str) – Given SpaCy CLI tasks. Currently only support pull and clone
repo_or_store (str, optional, defaults to None) – URL of Git repo or given S3 store containing project templates.
model (spacy.language.Language) – Instance of model to be saved
metadata (Dict[str, Any], optional, default to None) – Custom metadata for given model.
branch (str, optional, defaults to None) – The branch to clone from. If not specified, defaults to main branch
verbose (bool, optional, default to True) – Verbosely post all logs.
model_store (ModelStore, default to BentoMLContainer.model_store) – BentoML modelstore, provided by DI Container.

Returns

A tag with a format name:version where name is the user-defined model’s name, and a generated version by BentoML.

Return type

Tag

Warning

This is an EXPERIMENTAL API as it is subjected to change. We are also looking for feedback.

Examples:

import bentoml

clone_tag = bentoml.spacy.projects(
    "test_spacy_project",
    "clone",
    name="integrations/huggingface_hub",
    repo_or_store="https://github.com/aarnphm/bentoml-spacy-projects-integration-tests",
)
project_path = bentoml.spacy.load_project(clone_tag)


project_yml = {
    "remotes": {
        "default": "https://github.com/aarnphm/bentoml-spacy-projects-integration-tests/tree/v3/pipelines/tagger_parser_ud",
    }
}
pull_tag = bentoml.spacy.projects("test_pull", "pull", remotes_config=project_yml)
project_path = bentoml.spacy.load_project(pull_tag)

bentoml.spacy.load_runner(tag, *, gpu_device_id=None, backend_options=None, name=None, vocab=True, disable=(), exclude=(), config=None, as_tuples=False, batch_size=None, component_cfg=None)¶

Runner represents a unit of serving logic that can be scaled horizontally to maximize throughput. bentoml.spacy.load_runner() implements a Runner class that wrap around spacy.language.Language model, which optimize it for the BentoML runtime.

Parameters

tag (Union[str, Tag]) – Tag of a saved model in BentoML local modelstore..
gpu_device_id (int, optional, defaults to None) – GPU device ID.
backend_options (str, optional, defaults to None) – Backend options for Thinc. Either “pytorch” or “tensorflow”.
vocab (Union[spacy.vocab.Vocab, bool], optional, defaults to True) – Optional vocab to pass in on initialization. If True, a new Vocab object will be created.
disable (Sequence[str], optional) – Names of pipeline components to disable.
exclude (Sequence[str], optional) – Names of pipeline components to exclude. Excluded components won’t be loaded.
config (Union[Dict[str, Any], spacy.Config], optional) – Config overrides as nested dict or dict keyed by section values in dot notation.
as_tuples (bool, optional, defaults to False) – If set to True, inputs should be a sequence of (text, context) tuples. Output will then be a sequence of (doc, context) tuples.
batch_size (int, optional, defaults to None) – The number of texts to buffer.
component_cfg (Dict[str, :code:`Dict[str, Any]], optional, defaults to None) – An optional dictionary with extra keyword arguments for specific components.

Returns

Runner instances for the target bentoml.sklearn model

Return type

Runner

Examples:

import bentoml

runner = bentoml.sklearn.load_runner("my_model:latest")
runner.run([[1,2,3,4]])