Cumulus's YC profile says Ion delivers "30 to 50 percent more throughput than standard vLLM or SGLang", and the Launch HN post claimed 588 tok/s against 298 for a large hosted provider. Yet the Ion technical page, where an engineer goes to check, shows only absolute numbers (7,167 tok/s on one GH200; 588 tok/s on video with p50 latency up from 0.74 s to 1.46 s) and no comparison with any other engine. Cumulus's own measurement post explains why peak-throughput benchmarks mislead and promises a "next capacity test" with open-loop arrivals, but publishes no harness. The headline claim lives off-site, and the proof page has no baseline.
First action
Freeze two workloads (short chat on a 7B model, video on an 8B vision model) and the vLLM and SGLang versions to test against.
How to measure it
Ion technical page conversions (demo + API signup)
- Now
- –
- Target
- 3x the current monthly count
- By
- 90 days
Steps
Freeze the setup
Two workloads, model checkpoints, precision, deadlines and the exact vLLM and SGLang releases and flags.
CTO 1 day
Build the harness
Open-loop arrivals, swept request rates, correctness checks; failures and timeouts count as misses, as the measurement post defines.
Inference Engineer 3 days
Run all three engines
Same GH200, repeated boundary points, warm and cold runs.
Inference Engineer 3 days
Write it up honestly
Capacity at deadline, TTFT, p50 and p95, including where Ion loses.
CTO 2 days
Publish data and code
Table on /products/ion/technical, raw CSV and the harness in a public repo.
CTO 1 day
Carry the result to the front
One-line result in the homepage hero and on the Ion page, linked to the full method.
CEO 1 day
Add the two next steps
"Try it on the API" and "Benchmark your workload" with a demo form asking for model and deadline.
CEO 1 day
Share it where engineers are
A follow-up Show HN and posts on X and LinkedIn, with the harness link.
CEO 1 day
Why it works
Run the capacity test the measurement post promises and publish it on /products/ion/technical: the same model, the same GH200 and the same request mix through Ion, current vLLM and current SGLang (versions and flags documented), with an open-loop arrival sweep and capacity reported at a stated deadline and correctness bar. Show the table, the raw CSV and the harness, and include the cases where Ion loses (p50 latency on video). Then put the one-line result ("x% more on-time requests per GPU than vLLM at a 1-second deadline") in the homepage hero and on the Ion page, with two next steps: "Try it on the API" and "Benchmark your workload" (a demo booking that asks for the model and deadline).
Infrastructure buyers shortlist on numbers they can check, and a comparison is the first thing they look for. Bringing an existing claim onto the site where it can convert, measured with a method that is already published and stricter than the usual peak-throughput charts, answers the Launch HN critic who doubted Cumulus is on the cost and speed frontier. Showing the losses makes the wins believable.
Ready to use
ChecklistHead-to-head benchmark publishing list
[ ] Model, precision, context length and request mix stated [ ] Ion, vLLM and SGLang versions and flags listed [ ] Open-loop arrival sweep, not fixed concurrency [ ] Capacity at a named deadline and correctness bar [ ] TTFT, p50 and p95 for each engine [ ] Cases where Ion loses shown [ ] Raw CSV and harness linked [ ] One-line result on the homepage and Ion page [ ] "Try it on the API" and "Benchmark your workload" buttons [ ] YC profile wording matched to the measured result
Risks and how to handle them
The margin over current vLLM or SGLang is smaller than the YC profile says
Publish what you measure and update the profile; an honest, reproducible 20% beats an unverifiable 50%.
Rival maintainers dispute the configuration
Publish flags and invite pull requests to the harness; rerun with their suggested settings.
Evidence
- YC profileYC W26, founded 2025, team size 2, San Francisco; describes Cumulus as "The Fastest Multimodal Inference OS" and says Ion delivers "30 to 50 percent more throughput than standard vLLM or SGLang"; the launch post says customers save "50-70%" through usage-based pricing.Source
- Your websiteIon technical page: 7,167 output tokens/s for Qwen2.5-7B-Instruct on one GH200 at 128 concurrent requests, and 588 tokens/s for Qwen3-VL-8B on real video clips with p50 latency of 1.46 s (up from 0.74 s); no comparison with vLLM, SGLang or any other engine or provider appears on the page.Source
- Launch HN"Launch HN: IonRouter (YC W26) – High-throughput, low-cost inference" (72 points) claimed 588 tok/s against 298 for a large hosted provider on a multimodal workload, while conceding worse p50 latency. Commenters asked for the models/pricing page to be "linked from the top", for zero data retention, for a clear way to host a fine-tune (the founders cited roughly $8-10 per GPU hour with manual onboarding) and for availability on OpenRouter; one wrote it "seems like this is trailing the pareto frontier in cost and speed".Source
- Your websiteBlog, "Measuring inference throughput under latency constraints" (May 14, 2026): defines capacity as "the highest offered request rate that delivers a stated share of correct responses", uses open-loop arrivals, criticises the vLLM serving benchmark's default infinite request rate, and says the "next capacity test" will use the method; no harness, repository or script is published.Source