Architecture

The versions of the Yatai and yatai-image-builder and yatai-deployment components need to be consistent in the minor version.

Yatai

• Dashboard interface for all components.

• Registry for managing models and bentos with version control.

• APIs for programmatic access of Yatai functionality, e.g. bento push and pull.

The yatai component is the foundation of all other components in Yatai. It offers a dashboard user interface for viewing and managing bentos and models, deployments, clusters, and events. It also provides APIs for programmatic access to various Yatai functionality.

The yatai component installs and runs in a Kubernetes cluster and requires a relational database service (e.g. PostgreSQL) and an object storage service (e.g. AWS S3) to function.

Relational Database Service

A relational database used to store metadata, such as models, bentos, and users, required by the yatai component. Any PostgreSQL compatible relational databases can be used with provided user name and password. A database named yatai will be created to store tables of various metadata. We recommend using one of the following based on availability.

• PostgreSQL

• AWS Fully Managed Relational Database (RDS)

• Google Cloud SQL

Object Storage Service

Bucket storage for storing model and bento objects. Any S3 compatible services can be used providing the bucket name and endpoint address. We recommend using one of the following based on availability.

• MinIO

• AWS S3

• Google Cloud Storage (GCS)

yatai-image-builder

• Builds OCI images for bentos.

• Generate Bento CR.

The yatai-image-builder component is an add-on on top of yatai for building OCI images for bentos.

yatai-image-builder runs in Kubernetes, it is the operator of BentoRequest CRD, it is responsible for reconcile BentoRequest CR and then build the image for Bento, after the image is built Bento CR is generated, yatai-deployment component will depend on Bento CR to deploy Bento in Kubernetes.

Docker Registry

The Docker registry stores the OCI images built from bentos. yatai-deployment fetches images from the OCI image registry during deployment.

yatai-deployment

• Deploy bentos in Kubernetes.

The yatai-deployment component is an add-on on top of yatai for deploying bentos to Kubernetes.

yatai-deployment runs in Kubernetes, it is the operator of BentoDeployment CRD, it is responsible for reconcile BentoDeployment CR and then deploying bentos to Kubernetes. It relies on Bento CR to get the OCI image and runners information, so it should install after the yatai-image-builder component installation.

cert-manager

cert-manager adds certificate and certificate issuers resource types in the Kubernetes cluster. These resources are required by the BentoDeployment CRD conversion webhook.

metrics-server

metrics-server collects resource metrics from kubelet and exposes them in Kubernetes apiserver horizontal pod autoscaling.

Ingress Controller

Ingress controller provides access and load balances external requests to the BentoDeployment . Yatai Deployment is agnostic about the Ingress Class or the ingress controller implementations.

Observability

Yatai uses a group of tools to source and visualize metrics that helps pod observability and narrow down root causes when issues occur.

Prometheus

Prometheus is a monitoring system and time series database. It collects metrics exposed by BentoDeployment such as request count, duration, and in-progress.

Grafana

Grafana is used for visualizing metrics and collecting logs. Grafana works with the Prometheus stack to turn pod metrics into time series graphs to provide analytic insight. Grafana offers the user a customizable dashboard with clean, organized graphs.

Promtail

Promtail is a Grafana agent that acquires logs, turns logs into streams, and pushes the streams to Loki through an HTTP API. Promtail can be deployed as a DaemonSet on every Kubernetes node to collect logs from the node.

Loki

Loki is a stack that is used for log aggregation. It receives logs and streams and indexes them for efficient querying. Along with its highly efficient memory usage and scalability, Loki is designed to integrate seamlessly with Grafana for effective log storage and retrieval.