Triton Inference Sever Sandbox

Triton Inference Server

• Understand the Overheads of Ensemble Models

PyTriton

• Quick Start

Setup

# for CPU devce
make init

# for GPU device
make init-gpu

Launch Monitoring Tools

# Launch Prometheus and Grafana
make up-monitor

# Stop monitoring services
make stop-monitor

# Stop and remove monitoring services
make down-monitor