Llmspeed
AI
local llm benchmarking
SHOW HN2026
331points
86comments
#113of 4,366 in 2026
Top 3%of 2026 by points
How HN took it
CuriousMostly questions and neutral comments
5 positive, 28 neutral or asking, 2 mixed, 15 critical, out of 50 comments about the project. Thread score -0.20.
Performance: 3 for, 11 againstExecution: 0 for, 2 againstIdea: 1 for, 1 against
The thread, comment by comment
Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.
positive neutral or mixed critical about another commenter or something else the maker
What HN said
What people liked
Useful for judging whether expensive hardware makes a difference
From 2 comments
This is great. It solves two of my issues: ``` 1. It is difficult to figure out if dropping $10k+ on hardware will make a meaningful difference for running local models. 2.…
tom_alexander on HNI don't know what basis these numbers are estimated on, but my M2 Ultra 128GB runs almost twice as fast as what's reported for the M3 Ultra 512GB. I wished there was such a tool…
pil0u on HN
Good idea and well designed
From 2 comments
It would be cool if I could just search for a GPU instead of scrolling through the whole list. Overall, great website!
riobits on HNCool examples, this is interesting to test. I've noticed that OpenAI and Meta play a lot with animations and fades to mask the delays.
kinduff on HN
Objections
Speed estimates are off compared with real setups
From 4 comments
It says 46 tok/sec for Qwen3.6-27b 4 bit on RTX5090, but I get 60-70 tok/sec on a small context like that (5 bit weights with 8 bit KV cache), although it does slow down to 40…
kgeist on HNA random number I checked feels kinda optimistic, or there's something wrong with my setup; Gemma 26B-A4B on a 16GB 5060Ti is reported to get 21-88tok/s with Q4_K_M, meanwhile I…
adrian17 on HNJust chiming in that the idea is cool, but the speeds are way wrong
lisplist on HNI don't know what basis these numbers are estimated on, but my M2 Ultra 128GB runs almost twice as fast as what's reported for the M3 Ultra 512GB. I wished there was such a tool…
pil0u on HN
Ignores llama.cpp tuning, speculative decoding and context size
From 2 comments
It says 46 tok/sec for Qwen3.6-27b 4 bit on RTX5090, but I get 60-70 tok/sec on a small context like that (5 bit weights with 8 bit KV cache), although it does slow down to 40…
kgeist on HNIt would be great if numbers weren't off, would include optimised versions (ie. speculative decoding), context size, include allowed parallel inference (single agents use…
mirekrusin on HN
Model and quant choices are limited and default to tiny quants
From 2 comments
This tool has very tiny quants available for each model, and defaults to the tiny quants, so I'm not so sure you... tried it?
layla5alive on HNModel list is a bit limited, and whole harness affects the speed, so the libs used in harness and their versions might be useful to show for each individual model setup.
bozdemir on HN
Requests
Multi-GPU configurations
From 3 comments
I would like to see in such simulators: - multi GPU configuration (hetero and homo) - PCIe gen chooser (3.0/4.0/5.0) - is nvlink enabled (2 cards, 2 x 2 cards, 3 x 2 cards) - is…
gitowiec on HNPlease modify to include multiples of each as well. Most serious people aren't going to use a single PCIe card if they can help it.
sleight42 on HNDoesn't provide for hybrid setups RTX5070 + RTX5080 for example.
Perenti on HN
Add harness details to results
From 1 comment
I’ll look into adding harnesses. The benchmark sources usually don’t contain harness so the problem is limited data. What models would you like to see?
albinstman on HN
More models and hardware such as DGX Spark laptops
From 2 comments
A ton of RTX Spark (supposedly similar to DGX Spark) laptops just came out. We really need this for those. Asus ProArt P14, Surface Ultra, Yoga Pro 9n, etc. I am hoping the…
ilaksh on HNMissing: Dual Strix Halo, Dual Nvidia Spark
Tepix on HN
Questions
How accurate are the simulations?
From 1 comment
How accurate are the simulations?
prantosmss on HN
How does it handle context length and prefill?
From 2 comments
How about context length? Isn't it still just a few thousand tokens on local models?
Geee on HNThis is a great idea but I'm far less concerned about output token speed and way more about the ability to parallelize prefill - for agentic coding that's pretty much way more…
friendly_chap on HN
Alternatives named
Sparkbench for platform-specific leaderboards
From 1 comment
There are also platform specific sites which have leaderboards for different models and setups. Having an OEM Spark, this one has been very helpful https://sparkbench.dev/
verdverm on HN
Renting GPUs on vast.ai to test hardware
From 2 comments
just go rent one for a few minutes and try it out ... it'll cost you well under $0.50 if you're just trying to get a feel for if it's right for you. There's a lot of things like…
kristopolous on HNDepending on what hardware you're considering, you might rent a vast.ai box with similar specs for a month or two to get a feel for it.
montroser on HN