RYS (layer duplication in LLMs)
AI
model research
SHOW HN2026
Legend
RYS (layer duplication in LLMs)
How I topped the HuggingFace open LLM leaderboard on two gaming GPUs
dnhkng.github.io#llm layer duplication#open llm leaderboard#transformer layers#qwen2-72b
How HN took it
34 positive, 56 neutral or asking, 0 mixed, 1 critical, out of 91 comments about the project. Thread score +0.41.
Idea: 17 for, 1 againstDocs: 3 for, 0 againstPerformance: 1 for, 0 against
The thread, comment by comment
Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.
positive neutral or mixed critical about another commenter or something else the maker
What HN said
What people liked
Clear, engaging writeup of technical work
From 2 comments
I'm surprised the point/comment ratio is this skewed. There's so much meat in the post to chew on. I like your writing. This was one of those blogs where I can tell you spent a…
momojo on HNAmazing write up and i wish more people showed the process for discovery which is often even more interesting than the result itself Still the result is really interesting being…
imranq on HN
Replication of results on a different setup
From 2 comments
dude thats sick! i tried it out and it works. theres a couple layers in there that are part of the voidy block that doesnt do much for the selected answer, so i narrowed it down…
hashmap on HNGlad to see someone replicate the results already :)
dnhkng on HN
Objections
Single-layer duplication does not help and larger blocks degrade results
From 1 comment
Author here. I found that duplicating a specific block of 7 middle layers in Qwen2-72B, without modifying any weights, improved performance across all Open LLM Leaderboard…
dnhkng on HN
Requests
Release of code and analysis scripts
From 2 comments
At some point I will clean up and share the dynamic layer modification code for oobabooga Text-Generation-WebuUI. You can enter the setting, and apply new re-layering…
dnhkng on HNI build my own analysis tools. I'm just finishing up running the current generation of LLMs (MiniMax M2.5 and the Qwen3.5 family), and then I will put it all on Github. It less…
dnhkng on HN
Try repeating mid-layer blocks more than once
From 1 comment
Did you try repeating the same mid-layer block more than once? If the gain comes from giving the model another pass over its internal representation, I'd expect some sort of…
BloodAndCode on HN
Questions
Why did no one notice that out-of-order layers still work
From 2 comments
I'm surprised the point/comment ratio is this skewed. There's so much meat in the post to chew on. I like your writing. This was one of those blogs where I can tell you spent a…
momojo on HNThe astounding thing about Goliath wasn’t that is was a huge leap in performance, it was that the damn thing functioned at all. To this day, I still don’t understand why this…
mysteria on HN
Could a single layer be trained alone
From 1 comment
I had a dumb question. Does this mean one could train just one layer?
6510 on HN
Alternatives named
SOLAR depth up-scaling (Kim et al., 2023)
From 1 comment
Amazing write up and i wish more people showed the process for discovery which is often even more interesting than the result itself Still the result is really interesting being…
imranq on HN
Recurrent depth latent reasoning (Geiping et al., 2025)
From 1 comment
Amazing write up and i wish more people showed the process for discovery which is often even more interesting than the result itself Still the result is really interesting being…
imranq on HN