Advanced Guides

This part of the BentoML documentation is a work in progress. If you have any questions related to this, please join the BentoML Slack community and ask in the bentoml-users channel.

• Configuration
  • Environment Variables
  • User Defined Configuration File
  • BentoML Defaults
• Logging
• Offline Batch Serving
  • StringInput
  • FileInput
  • MultiFileInput
• Monitoring with Prometheus
• Adaptive Micro Batching
  • 1. The overall architecture of BentoML’s micro-batching server
    • 1.1 Why micro batching matters
    • 1.2 Architecture & Data Flow
    • 1.3 Parameters & Concepts of micro batching
    • 1.4 Sequence & How it works
    • 1.5 The main design decisions and trade-offs
  • 2. parameter tuning best practices & recommendations
  • 3. How to implement batch mode for custom input adapters
  • 4. Comparisons
    • 4.1 TensorFlow Serving
    • 4.2 Clipper
• Adding Custom Model Artifact
  • Create custom Artifact
  • Define and save BentoService with the custom Artifact
  • Test with example data
• Customizing InputAdapter
  • Create custom InputAdapter
  • Define and save BentoService with the customized InputAdapter
  • Test with example data
• Deploy yatai server behind NGINX
  • 1. make NGINX proxying HTTP/2 from h2 to h2c
  • 2. config yatai client to use h2
  • 3. (Optional) using self-signed certificates
  • 4. (Optional) using TLS client certificates
• Performance Tracing
• Install Yatai on K8w with Helm
  • 1. Configuration
    • 1.1 Persistent Storage
    • 1.2 Ingress
    • 1.3 TLS
  • 2. Deploying
    • 2.1 Minikube (Local)
      • Dry Run
      • Custom Values
    • 2.2 Cloud Providers