Platform

One loop for every model on every GPU

Turbokern is one agent runtime, one real-GPU benchmark harness and one packaging pipeline, shared by six products. Every run feeds one dataset that makes the next run faster.

Platform at a glancePilot scope
Products
6 modules, 1 loop
Targets
H100, H200, Blackwell, Jetson
Outputs
TensorRT-LLM, Triton, NIM
Deploy
Cloud, on-prem, air-gapped
Guards
Correctness and accuracy
Pilot scope. Blackwell and Jetson follow H100 and H200.

Architecture

Six layers, each one measurable

The architecture follows the work a performance engineer does, in order, and records everything it does along the way.

LayerWhat it doesBuilt onProduct
IngestTakes the model, objective and hardware target; captures the baseline.CUDA, your eval setPlatform
ProfilerTraces the workload and classifies bottlenecks: compute, memory, launch, serving.Nsight Systems, Nsight Compute, CUPTIProfyle
Optimization agentPlans strategies and spawns candidate kernels, precisions and configs.Agent runtime, learned policiesKernl, Quantra, Servex
PredictorRanks candidates before they run so only the promising ones use GPU time.RAPIDS, trajectory datasetSimyx
Benchmark harnessRuns candidates on real GPUs; measures latency, throughput, accuracy and cost.DGX, HGX, JetsonPlatform
Artifact builderEmits the engine, a reproducible recipe and the run report.TensorRT-LLM, Triton, NIMServex, Edgyn

Data flywheel

Every run makes the next one cheaper

Each benchmark is labelled data: this change, on this workload, on this GPU, did this. That optimization-trajectory dataset trains Simyx and the policies inside Kernl, Quantra and Servex.

Better predictions mean fewer candidates benchmarked for the same win, which lowers the cost of every run that follows.

  • Trajectory data is pooled only as anonymous performance features, never as weights or prompts.
  • Customers can opt out of pooling; their runs then only improve their own tenant.
Candidates benchmarked per kept changeIllustrative
Brute-force searchGuided by SimyxIllustrative
First run on a new GPU
~40
~40
After 100 runs
~40
~18
After 1,000 runs
~40
~8
Illustrative planning curve, not measured. Brute-force search shown as the grey bar.

Deployment

Run it where your models live

Managed

Turbokern cloud

Runs on our GPU fleet. Fastest to start; you upload a model or point us at a registry.

Your cloud

Customer VPC

The agent and harness run in your account on your GPUs. Only run metadata leaves.

On-prem

Air-gapped

Everything on your hardware, supported through NVIDIA AI Enterprise. Planned for enterprise tier.

Planned

Control plane

You set the rules the agent runs under

Every run has an objective, guards and a compute budget. The agent cannot keep a change that breaks a guard and cannot spend beyond the budget.

Policies apply per model, per team or across the organization, and every decision is written to the run report.

policy.yaml
budget:
  gpu_hours_per_run: 120
  max_parallel_gpus: 8
guards:
  max_accuracy_drop: 0.005
  numeric_tolerance: bf16-ulp:4
  forbid: [int4]   # this team only
approval:
  deploy: manual   # a person ships the engine

Security

Your models and data stay yours

Summary of controls. The full detail is on the security page.

Tenant isolationEach customer's runs, traces and artifacts are stored and processed separately.
No training on your weightsPooled learning uses performance features only; weights, prompts and eval data are never pooled.
Audit trailEvery candidate, measurement and decision is logged with the hardware and versions used.
Independent auditA SOC 2 Type II audit is planned before general availability. It has not started.

Integrations

Fits the stack you already run

Turbokern reads from your registry and eval store and writes standard NVIDIA artifacts back. Nothing proprietary sits in your serving path.

ModelsHugging Face, S3, GCSModel and checkpoint sources
FormatsPyTorch, ONNX, safetensorsAccepted inputs
EnginesTensorRT-LLM, TensorRTOptimized outputs
ServingTriton, NIMPackaged deployables
CIGitHub Actions, GitLab CIRun on every model change
TrackingMLflow, Weights & BiasesReport export
OrchestrationKubernetes, SlurmWhere the harness runs
EdgeJetPack, DeepStream, HoloscanEmbedded targets

API

Start runs, stream progress and fetch artifacts over a REST API. The CLI and the CI actions use the same API.

DataFormat
Run reportJSON and HTML
Opportunity mapJSON
RecipeYAML, reproducible
EngineTensorRT-LLM plan, Triton model repo, NIM container
api request
# start a run from CI
curl -X POST https://api.turbokern.com/v1/runs \
  -H "Authorization: Bearer $TURBOKERN_TOKEN" \
  -d @turbokern.yaml

# response
{ "run_id": "run_07", "status": "profiling" }

Next step

See the loop on your own model

Bring one workload to the pilot. We profile it first and tell you what we expect to find before any optimization runs.