BentoML
latest
Getting Started
Installation
Save Models
Define and Debug Services
Build and Deploy Bentos
Further Reading
Core Concepts
Service Definition
Composition
Runners
Services
APIs
Further Reading
API and IO Descriptors
Sync vs Async APIs
IO Descriptors
Built-in Types
Composite Types
Further Reading
Model and Bento Management
Managing Models Locally
Creating Models
Listing Models
Deleting Models
Managing Bentos Locally
Creating Bentos
Listing Bentos
Deleting Bentos
Managing Models and Bentos Remotely with Yatai
Pushing Models
Pulling Models
Pushing Bentos
Pulling Bentos
Further Reading
Containerize Bentos as Docker Images
Frameworks
CatBoost
Detectron2
EasyOCR
MXNet Gluon
H2O
MLFlow
ONNX
ONNX-mlir
LightGBM
PaddlePaddle
Picklable Model
PyCaret
PyTorch
PyTorch Lightning
Keras
Scikit-Learn
SpaCy
Statsmodels
Tensorflow
Tensorflow V1
Transformers
XGBoost
Advanced Guides
BentoServer
Runners
Adaptive Batching
Architecture
Running with Adaptive Batching
Standalone Mode
Distributed
Configuring Batching
Configuration
Docker Deployment
Serving on GPU
Prerequisite
NVIDIA Drivers
NVIDIA Container Toolkit
General workaround (Recommended)
Debian-based OS
Other OS
docker-compose
Framework Support for GPU Inference with Implementation
Preface
Docker Images Options
Tensorflow
Tensorflow Implementation
PyTorch
PyTorch Implementation
ONNX
ONNX Implementation
Building Bentos
Configuring files to include
Configuring files to exclude
Build your Bento
Bento Format
Service
Description
Labels
Additional Models
Python Packages
Python Options
Package Locking
Pip Wheels
Conda Options
Conda Fields
Docker Options
Docker Fields
Conclusion
Offline Batch Inference
Logging
OpenTelemetry Compatible
Exception Logging
Logging Configuration
Web Service Request Logging
Model Runner Request Logging
CI/CD workflow
Use custom ML framework
Performance Tracing
Custom HTTP endpoints
Mounting WSGI based web frameworks
Mounting ASGI based web frameworks
Serving Multiple Models
Multiple dependent models
Performance Benchmark
API Reference
Core APIs
BentoService
Bento Build
Runner
Tag
Model
Storage API
Bento Store
Model Store
Yatai Store API
API IO Descriptors
IODescriptor base
bentoml.io.JSON
bentoml.io.NumpyNdarray
bentoml.io.File
bentoml.io.Image
bentoml.io.Text
bentoml.io.PandasDataFrame
bentoml.io.PandasSeries
bentoml.io.Multipart
CLI Reference
bentoml
build
containerize
delete
export
get
import
list
models
delete
export
get
import
list
pull
push
pull
push
serve
yatai
login
BentoML
Docs
»
Advanced Guides
»
CI/CD workflow
Edit on GitHub
CI/CD workflow
ΒΆ
TODO
Read the Docs
v: latest
Versions
latest
v0.13.1
v0.13.0
v0.12.1
v0.12.0
0.13-lts
Downloads
On Read the Docs
Project Home
Builds
Free document hosting provided by
Read the Docs
.