Artifact
A deployable engine
A TensorRT-LLM engine, Triton model repository or NIM container, built for the exact GPU you named.
How it works
A Turbokern run looks like a careful performance engineer's week, compressed: baseline, profile, try many changes, keep the ones that survive measurement, ship the result.
The run
Each step writes to the run report, so you can see what was tried and why.
Run the model as you serve it today and record latency, throughput, memory and accuracy.
Profyle traces the workload and produces a ranked opportunity map.
Simyx scores every candidate the optimizers propose and passes on the most promising.
Kernl, Quantra and Servex try kernels, precisions and serving configs.
Each candidate runs on the real GPU against correctness and accuracy guards.
Winning changes are combined, re-measured together and packaged.
Who does what
| Task | Turbokern | Your team |
|---|---|---|
| Define the objective and guards | Suggests defaults | Decides |
| Provide the evaluation set | Validates it runs | Provides |
| Profile and find bottlenecks | Does it | Reviews the map |
| Generate and test candidates | Does it | Nothing required |
| Approve deployment | Packages the engine | Approves and ships |
| Re-run on model or GPU change | Triggers automatically | Sets the policy |
Configuration
The objective says what better means. Guards say what must never get worse. The budget says how much GPU time the search may use.
objective: throughput_tok_s: maximize latency_p99_ms: 120 # hard limit guards: max_accuracy_drop: 0.01 stages: [profyle, kernl, quantra, servex] budget: gpu_hours: 96
Kept or rejected
This is what the run report's candidate ledger looks like. Most candidates are rejected; that is the point of measuring.
Example run on one H100. Candidates, speedups and reasons are illustrative, not measured results.
What you get
Artifact
A TensorRT-LLM engine, Triton model repository or NIM container, built for the exact GPU you named.
Recipe
Every kept change with versions and parameters, so the engine can be rebuilt and diffed.
Report
Baseline, final numbers, every candidate tried, and why each was kept or rejected.
FAQ
For one LLM endpoint we plan for one to three days, mostly GPU time. Smaller vision or speech models are faster. The compute budget you set caps it.
The run ends with the best result inside your guards and a report explaining which bottleneck remains, for example memory bandwidth at that batch size.
Yes. Each kept change is listed separately in the recipe, and you can rebuild with any subset.
A new model version, a new GPU type, a TensorRT-LLM upgrade or a traffic shift beyond a threshold you set.
Next step
The pilot begins with Profyle on one workload, so you see the opportunity map before committing GPU time to optimization.