Hide navigation sidebar

Hide table of contents sidebar

Toggle site navigation sidebar

Toggle table of contents sidebar

BentoML

Get started
Toggle navigation of Get started
Use cases
Toggle navigation of Use cases
- Large language models
  Toggle navigation of Large language models
  - vLLM inference
- Diffusion models
  Toggle navigation of Diffusion models
- Embeddings
  Toggle navigation of Embeddings
  - Sentence Transformer
  - CLIP embeddings
- Audio
  Toggle navigation of Audio
  - WhisperX: Speech recognition
  - XTTS: Text to speech
- BLIP: Image captioning
Guides
Toggle navigation of Guides
- Services
- Input and output types
- Deployment
- Containerization
- Workers
- Build options
- Model Store
- GPU inference
- Model composition
- Distributed Services
- Concurrency
- Testing
- Clients
- Adaptive batching
- Observability
  Toggle navigation of Observability
  - Monitoring and data collection
- ASGI integration
- Configurations
- Lifecycle hooks
API reference
Toggle navigation of API reference
- Bento and model APIs
- BentoML SDK
- BentoML CLI
- Client API
- Metrics API
- Framework APIs
  Toggle navigation of Framework APIs
  - Diffusers
  - ONNX
  - Scikit-Learn
  - Transformers
  - Flax
  - TensorFlow
  - TorchScript
  - XGBoost
  - Picklable Model
  - PyTorch
  - LightGBM
  - MLflow
  - CatBoost
  - fast.ai
  - EasyOCR
  - Keras
  - Ray
  - Detectron
- Batch inference
- Exceptions
- Container APIs
- Types
Examples

BentoCloud

Get started
Guides
Toggle navigation of Guides
Reference
Toggle navigation of Reference

v: latest

Versions: latest; v1.1.11

Downloads

On Read the Docs: Project Home; Builds

Toggle table of contents sidebar

Ray#

About this page

This is an API reference for deploying BentoML with Ray. Please refer to BentoRay guide for more information.

Copyright © 2022-2024, bentoml.com

Made with Furo