llm-circuit-finder
AI
llm layer duplication
SHOW HN2026
Legend
llm-circuit-finder
Duplicate 3 layers in a 24B LLM, logical deduction .22→.76. No training
github.com#llm layer duplication#reasoning circuits#transformer layers#lm-eval benchmarks
How HN took it
5 positive, 27 neutral or asking, 0 mixed, 9 critical, out of 41 comments about the project. Thread score -0.12.
Idea: 4 for, 5 againstPerformance: 0 for, 4 against
The thread, comment by comment
Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.
positive neutral or mixed critical about another commenter or something else the maker
What HN said
What people liked
Interesting, cool result on the logical deduction benchmark
From 2 comments
This is very cool
edg5000 on HNThere's still a lot of low hanging fruit left IMO. Good find and rather funny to think about as you can have someone simply clone the various layers multiple times and instead of…
nowittyusername on HN
Fits with pruning findings that middle layers carry reasoning
From 1 comment
this lines up with what pruning papers have been finding, the middle layers carry most of the reasoning weight and you can often drop the outer ones without much loss. cool to…
zhangchen on HN
Objections
Prior art: David Ng's RYS method and earlier layer duplication work
From 3 comments
> I replicated David Ng's RYS method [...] found something I didn't expect. > Transformers appear to have discrete "reasoning circuits" — contiguous blocks of 3-4 layers that act…
simgt on HNWow, every single word in the original post and on that README.md is pure LLM. How sad. In any case, this has been done at least since the very first public releases of Llama by…
Karuma on HNThe method used here by David Ng, was discussed a few days ago at https://news.ycombinator.com/item?id=47322887
kristianp on HN
Gains may be benchmark noise or come with losses on other tasks
From 3 comments
But if it got worse on other tests, it doesn't do much good, right?
XCSme on HNWhat, just randomly choose some "layer" and duplicate it and give some arbitrary reasoning went from 0.2 -> 0.7, i don't know man. You need to use real benchmarks.
m3kw9 on HNI’d love to believe this is real, but I’m pretty sure you will lose performance on a “fair” mix of tasks, even after fine tuning. I know multiple teams have explored recurrent…
rao-v on HN
Small test sample (50 per task) and speed-limited validation
From 1 comment
I published the results for devstral... results folder of the github https://github.com/alainnothere/llm-circuit-finder/tree/main/results I'm using the following configuration…
xlayn on HN
Post and README appear LLM-written
From 1 comment
Wow, every single word in the original post and on that README.md is pure LLM. How sad. In any case, this has been done at least since the very first public releases of Llama by…
Karuma on HN
Requests
Publish per-task benchmark results
From 1 comment
Would you be able to publish the individual benchmarks for Qwen2.5-Coder-32B? GSM8K specifically would be useful to look at.
colejhudson on HN
Test on newer Qwen models
From 1 comment
Why test on Qwen 2.5 when Qwen 3 has been out for about a year, and Qwen 3.5 for a month? My problem with this is ironically entirely vibes based, that for some reason, LLMs love…
christianqchung on HN
Try skipping layers instead of duplicating them
From 1 comment
That's interesting, could you point me to some source on these findings? It seems to me that the difference between "iterative improvement" as you put it and "close to the…
4bpp on HN
Questions
Why the logical deduction gain happens and whether it generalises
From 2 comments
Assuming the benchmarks are sound (rather than capturing a fluke), the provided explanation still does not pass the smell test. As far as I can tell, there is nothing about the…
4bpp on HNthe "reasoning circuits" framing is doing a lot of work here. tidy story, fits the result, but I'm not sure it explains anything - you duplicated layers and got better scores,…
the_harpia_io on HN
How to run these models on AMD GPUs
From 1 comment
How do you run these models on AMD GPUs?
gukoff on HN
Alternatives named
David Ng's RYS method
From 1 comment
> I replicated David Ng's RYS method [...] found something I didn't expect. > Transformers appear to have discrete "reasoning circuits" — contiguous blocks of 3-4 layers that act…
simgt on HN
Depth Up-Scaling (Solar 10.7B)
From 1 comment
Reminds me of Solar 10.7B, which was a very good model for its size ~2 year ago and the "Depth Up-Scaling" technique behind it. Although, that involved continued training after…
woadwarrior01 on HN
Layer pruning
From 1 comment
you can also have removed layers of models and keep the same score in benchmarks [1]. i feel that sometimes a lot of the layers might just be redundant and are not fully needed…
snats on HN