Managed
Turbokern cloud
Runs on our GPU fleet. Fastest to start; you upload a model or point us at a registry.
Platform
Turbokern is one agent runtime, one real-GPU benchmark harness and one packaging pipeline, shared by six products. Every run feeds one dataset that makes the next run faster.
Architecture
The architecture follows the work a performance engineer does, in order, and records everything it does along the way.
| Layer | What it does | Built on | Product |
|---|---|---|---|
| Ingest | Takes the model, objective and hardware target; captures the baseline. | CUDA, your eval set | Platform |
| Profiler | Traces the workload and classifies bottlenecks: compute, memory, launch, serving. | Nsight Systems, Nsight Compute, CUPTI | Profyle |
| Optimization agent | Plans strategies and spawns candidate kernels, precisions and configs. | Agent runtime, learned policies | Kernl, Quantra, Servex |
| Predictor | Ranks candidates before they run so only the promising ones use GPU time. | RAPIDS, trajectory dataset | Simyx |
| Benchmark harness | Runs candidates on real GPUs; measures latency, throughput, accuracy and cost. | DGX, HGX, Jetson | Platform |
| Artifact builder | Emits the engine, a reproducible recipe and the run report. | TensorRT-LLM, Triton, NIM | Servex, Edgyn |
Data flywheel
Each benchmark is labelled data: this change, on this workload, on this GPU, did this. That optimization-trajectory dataset trains Simyx and the policies inside Kernl, Quantra and Servex.
Better predictions mean fewer candidates benchmarked for the same win, which lowers the cost of every run that follows.
Deployment
Managed
Runs on our GPU fleet. Fastest to start; you upload a model or point us at a registry.
Your cloud
The agent and harness run in your account on your GPUs. Only run metadata leaves.
On-prem
Everything on your hardware, supported through NVIDIA AI Enterprise. Planned for enterprise tier.
PlannedControl plane
Every run has an objective, guards and a compute budget. The agent cannot keep a change that breaks a guard and cannot spend beyond the budget.
Policies apply per model, per team or across the organization, and every decision is written to the run report.
budget: gpu_hours_per_run: 120 max_parallel_gpus: 8 guards: max_accuracy_drop: 0.005 numeric_tolerance: bf16-ulp:4 forbid: [int4] # this team only approval: deploy: manual # a person ships the engine
Security
Summary of controls. The full detail is on the security page.
Integrations
Turbokern reads from your registry and eval store and writes standard NVIDIA artifacts back. Nothing proprietary sits in your serving path.
Start runs, stream progress and fetch artifacts over a REST API. The CLI and the CI actions use the same API.
| Data | Format |
|---|---|
| Run report | JSON and HTML |
| Opportunity map | JSON |
| Recipe | YAML, reproducible |
| Engine | TensorRT-LLM plan, Triton model repo, NIM container |
# start a run from CI curl -X POST https://api.turbokern.com/v1/runs \ -H "Authorization: Bearer $TURBOKERN_TOKEN" \ -d @turbokern.yaml # response { "run_id": "run_07", "status": "profiling" }
Next step
Bring one workload to the pilot. We profile it first and tell you what we expect to find before any optimization runs.