Advanced Guides

This part of the BentoML documentation is a work in progress. If you have any questions related to this, please join the BentoML Slack community and ask in the bentoml-users channel.

• Offline Batch Serving
• Monitoring with Prometheus
• Request Logging
• Understanding BentoML adaptive micro batching
• 1. The overall architecture of BentoML’s micro-batching server
  • 1.1 Why micro batching matters
  • 1.2 Architecture & Data Flow
  • 1.3 Parameters & Concepts of micro batching
  • 1.4 Sequence & How it works
  • 1.5 The main design decisions and trade-offs
• 2. parameter tuning best practices & recommendations
• 3. How to implement batch mode for custom input adapters
• 4. Comparison
  • 4.1 TensorFlow Serving
  • 4.2 Clipper
• Adding Custom Model Artifact
• Customizing InputAdapter
• Deploy yatai server behind NGINX
  • 1. make NGINX proxying HTTP/2 from h2 to h2c
  • 2. config yatai client to use h2
  • 3. (Optional) using self-signed certificates
• Using Helm to install YataiService
• 1. Configuration
  • 1.1 Persistent Storage
  • 1.2 Ingress
  • 1.3 TLS
• 2. Deploying
  • 2.1 Minikube (Local)
    • Dry Run
    • Custom Values
  • 2.2 Cloud Providers