[{"data":1,"prerenderedAt":593},["ShallowReactive",2],{"project-47322887":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":28,"sentiment":29,"stance_mix":30,"scored_comments":35,"aspects":36,"strip":42,"post_text":45,"what":46,"said":80,"maker_in_thread":149,"shape":160,"grid":164,"attempts":555,"same_week":556,"maker_shows":38,"maker_hidden":585,"reader_version":586,"card":587},"47322887","RYS (layer duplication in LLMs)","How I topped the HuggingFace open LLM leaderboard on two gaming GPUs","title","Show HN: How I topped the HuggingFace open LLM leaderboard on two gaming GPUs","https:\u002F\u002Fdnhkng.github.io\u002Fposts\u002Frys\u002F","dnhkng.github.io","dnhkng",1773148735,2026,495,126,53,4366,2,"legend","ai","model research","other",[24,25,26,27],"llm layer duplication","open llm leaderboard","transformer layers","qwen2-72b","loved",0.406,{"positive":31,"neutral":32,"mixed":33,"critical":34},34,56,0,1,91,{"docs":37,"idea":39,"performance":41},{"positive":38,"critical":33},3,{"positive":40,"critical":34},17,{"positive":34,"critical":33},[43,43,38,44,18,38,38,38,43,43,38,43,43,44,43,38,38,44,43,38,43,44,43,33,44,43,43,43,38,44,44,43,44,43,44,38,38,38,43,43],5,4,null,{"problem":47,"audience":45,"pricing":45,"motivation":57,"time_to_build":45,"stack":66,"openness":77,"stage":78,"team_size":79},{"value":48,"source":49,"evidence":50},"Improving LLM benchmark performance without fine-tuning or new training","maker",[51,54],{"id":52,"by":11,"text":53},"47326242","Here's an extract, the core TL;DR for a feel of the article. \"And now for the weirdness: There was never the case where any Transformer layer would have seen the output from a…",{"id":55,"by":11,"text":56},"47322904","Author here. I found that duplicating a specific block of 7 middle layers in Qwen2-72B, without modifying any weights, improved performance across all Open LLM Leaderboard…",{"value":58,"source":49,"evidence":59},"Hobbyist curiosity about LLM internal structure, limited hardware forced unusual approaches",[60,63],{"id":61,"by":11,"text":62},"47332270","Cheers. I will go back though my other old projects (optogenetics, hacking Crispr\u002FCAS9 etc), and put them on my blog. On your questions: 1) A few other papers have been mentioned…",{"id":64,"by":11,"text":65},"47332342","But blogging is fun! I do wish one of the big labs would sponsor with a rack of HGX Rubin NVL8's. I have lots of ideas to test, and I have probably hit the spending limit with…",[67,71],{"value":68,"source":49,"evidence":69},"Qwen2-72B",[70],{"id":55,"by":11,"text":56},{"value":72,"source":49,"evidence":73},"Text-Generation-WebUI",[74],{"id":75,"by":11,"text":76},"47327730","At some point I will clean up and share the dynamic layer modification code for oobabooga Text-Generation-WebuUI. You can enter the setting, and apply new re-layering…","unknown","launched","solo",{"praise":81,"objections":104,"requests":109,"questions":124,"alternatives":140},[82,94],{"value":83,"source":84,"evidence":85},"Clear, engaging writeup of technical work","commenter",[86,90],{"id":87,"by":88,"text":89},"47329722","momojo","I'm surprised the point\u002Fcomment ratio is this skewed. There's so much meat in the post to chew on. I like your writing. This was one of those blogs where I can tell you spent a…",{"id":91,"by":92,"text":93},"47328151","imranq","Amazing write up and i wish more people showed the process for discovery which is often even more interesting than the result itself Still the result is really interesting being…",{"value":95,"source":84,"evidence":96},"Replication of results on a different setup",[97,101],{"id":98,"by":99,"text":100},"47360629","hashmap","dude thats sick! i tried it out and it works. theres a couple layers in there that are part of the voidy block that doesnt do much for the selected answer, so i narrowed it down…",{"id":102,"by":11,"text":103},"47365376","Glad to see someone replicate the results already :)",[105],{"value":106,"source":49,"evidence":107},"Single-layer duplication does not help and larger blocks degrade results",[108],{"id":55,"by":11,"text":56},[110,117],{"value":111,"source":84,"evidence":112},"Release of code and analysis scripts",[113,114],{"id":75,"by":11,"text":76},{"id":115,"by":11,"text":116},"47324862","I build my own analysis tools. I'm just finishing up running the current generation of LLMs (MiniMax M2.5 and the Qwen3.5 family), and then I will put it all on Github. It less…",{"value":118,"source":84,"evidence":119},"Try repeating mid-layer blocks more than once",[120],{"id":121,"by":122,"text":123},"47327655","BloodAndCode","Did you try repeating the same mid-layer block more than once? If the gain comes from giving the model another pass over its internal representation, I'd expect some sort of…",[125,133],{"value":126,"source":84,"evidence":127},"Why did no one notice that out-of-order layers still work",[128,129],{"id":87,"by":88,"text":89},{"id":130,"by":131,"text":132},"47327628","mysteria","The astounding thing about Goliath wasn’t that is was a huge leap in performance, it was that the damn thing functioned at all. To this day, I still don’t understand why this…",{"value":134,"source":84,"evidence":135},"Could a single layer be trained alone",[136],{"id":137,"by":138,"text":139},"47405332","6510","I had a dumb question. Does this mean one could train just one layer?",[141,145],{"value":142,"source":84,"evidence":143},"SOLAR depth up-scaling (Kim et al., 2023)",[144],{"id":91,"by":92,"text":93},{"value":146,"source":84,"evidence":147},"Recurrent depth latent reasoning (Geiping et al., 2025)",[148],{"id":91,"by":92,"text":93},{"comments":150,"answered":151,"key":152},25,13,[153,154,157],{"id":61,"text":62},{"id":155,"text":156},"47332308","Hi, thanks for the praise! On the other papers, models like SOLAR or training a model that uses a single layers are probably going to hit a wall, based on the heatmaps I found.…",{"id":158,"text":159},"47328051","It's still non-trivial, as multi-digit numbers can be constructed a huge combination of valid tokens. The code in the blog helps derive useful metrics from partial answers.",{"commenters":161,"max_depth":162,"maker_replies":150,"first_comment_mins":34,"lifespan_hours":163},89,6,321,[165,167,169,173,174,175,176,178,181,184,187,190,194,197,200,204,206,208,211,215,217,220,223,227,230,233,236,238,241,243,248,252,254,256,260,264,268,272,276,279,281,283,285,289,293,297,300,304,306,310,312,314,316,318,322,324,328,332,336,338,342,346,348,350,354,356,359,363,365,369,373,375,379,380,383,387,390,392,394,396,398,403,407,411,415,419,423,425,428,432,436,440,442,445,447,449,452,455,457,460,462,465,468,471,473,476,479,481,484,487,490,493,496,499,502,505,508,511,514,516,518,521,523,525,529,531,534,536,538,542,544,548,552],[87,33,43,166,88,33],"maker\u002Fpositive\u002Fdocs",[61,33,33,168,11,34],"maker\u002Fneutral\u002Fidea",[170,33,38,171,172,33],"47333463","project\u002Fneutral\u002Fidea","trhway",[91,34,43,166,92,33],[155,34,33,171,11,34],[130,18,38,171,131,33],[158,18,33,177,11,34],"project\u002Fneutral\u002Fexecution",[179,18,38,171,180,33],"47328303","janalsncm",[182,18,38,171,183,33],"47331952","wolttam",[185,18,38,171,186,33],"47344921","evangambit",[188,18,38,171,189,33],"47495823","ibestvina",[191,38,43,192,193,33],"47327567","project\u002Fpositive\u002Fidea","iamjackg",[195,38,33,196,11,34],"47328056","maker\u002Fpositive\u002Fnone",[198,44,38,171,199,33],"47327132","Lerc",[201,44,34,202,203,33],"47327447","project\u002Fcritical\u002Fidea","le-mark",[205,44,38,171,180,33],"47328187",[207,44,38,171,199,33],"47330751",[209,43,38,171,210,33],"47325418","Havoc",[212,43,33,213,214,33],"47325960","commenter\u002Fneutral\u002Fnone","dinobones",[216,43,33,171,11,34],"47326330",[218,43,38,171,219,33],"47326369","dormento",[221,43,33,213,222,33],"47326900","dtj1123",[224,43,33,225,226,33],"47327461","other\u002Fneutral\u002Fnone","idiotsecant",[228,43,38,171,229,33],"47334754","adcoleman6",[231,43,33,225,232,33],"47328742","fweimer",[234,43,38,171,235,33],"47330960","gwern",[237,43,33,171,11,34],"47332333",[239,162,43,192,240,33],"47325223","hmokiguess",[242,162,33,196,11,34],"47325415",[244,245,43,246,247,33],"47323520",7,"maker\u002Fpositive\u002Fidea","tgw43279w",[249,250,38,171,251,33],"47327136",8,"hex4def6",[75,250,33,253,11,34],"maker\u002Fneutral\u002Fnone",[255,250,33,213,229,33],"47334740",[257,258,38,171,259,33],"47332411",9,"phire",[261,258,33,262,263,33],"47442604","other\u002Fneutral\u002Fidea","FuckButtons",[265,266,33,225,267,33],"47325130",10,"GaggiX",[269,270,43,196,271,33],"47332228",11,"supriyo-biswas",[273,274,43,192,275,33],"47351859",12,"fennecfoxy",[277,151,33,225,278,33],"47324419","WithinReason",[280,151,43,192,247,33],"47324540",[282,151,33,171,11,34],"47324817",[284,151,38,171,278,33],"47324864",[286,287,43,196,288,33],"47324897",14,"cootsnuck",[290,291,43,192,292,33],"47369249",15,"BrownSol",[294,295,38,171,296,33],"47369420",16,"vjsrinivas",[298,40,38,171,299,33],"47331935","twotwotwo",[301,302,43,192,303,33],"47335514",18,"koenschipper",[52,305,33,171,11,34],19,[307,308,38,171,309,33],"47325918",20,"Aditya_Garg",[311,308,38,171,180,33],"47326311",[313,308,38,171,235,33],"47331001",[315,308,38,171,180,33],"47340068",[317,308,38,171,235,33],"47340277",[319,320,38,171,321,33],"47333432",21,"hackerchy",[323,320,38,171,275,33],"47352274",[325,326,38,171,327,33],"47330397",22,"dgoet",[329,330,38,171,331,33],"47374987",23,"v9v",[333,334,43,192,335,33],"47330340",24,"dubbel",[337,150,38,171,180,33],"47328400",[339,340,43,192,341,33],"47339184",26,"JimSanchez",[343,344,43,196,345,33],"47323385",27,"blourvim",[347,344,33,171,11,34],"47324967",[349,344,38,171,275,33],"47352402",[351,352,38,171,353,33],"47325103",28,"goodmythical",[355,352,33,171,11,34],"47325186",[357,352,38,171,358,33],"47325473","sva_",[360,361,38,171,362,33],"47325794",29,"kovek",[364,361,33,171,11,34],"47326512",[366,367,43,192,368,33],"47324845",30,"tjwei",[370,371,43,196,372,33],"47326309",31,"dongecko",[137,374,38,171,138,33],32,[376,377,38,171,378,33],"47331208",33,"siliconc0w",[64,377,33,225,11,34],[381,31,43,196,382,33],"47329414","Xuzzo",[384,385,38,171,386,33],"47327251",35,"d0100",[98,388,43,389,99,33],36,"project\u002Fpositive\u002Fperformance",[102,388,33,391,11,34],"project\u002Fpositive\u002Fexecution",[393,388,33,225,99,33],"47365978",[395,388,33,225,11,34],"47366673",[397,388,33,225,99,33],"47372005",[399,400,38,401,402,33],"47325428",37,"project\u002Fneutral\u002Fperformance","lordmathis",[404,405,38,171,406,33],"47328511",38,"kristianp",[408,409,43,196,410,33],"47331375",39,"Handsome2734",[412,413,43,192,414,33],"47325341",40,"rob_c",[416,417,33,225,418,33],"47331000",41,"FergusArgyll",[420,421,38,171,422,33],"47324695",42,"seeknotfind",[424,421,33,171,11,34],"47324781",[426,421,38,171,427,33],"47327867","cosarara",[429,430,43,196,431,33],"47331543",43,"vicentwu",[433,434,43,196,435,33],"47337115",44,"sxhilmuffin",[437,438,43,196,439,33],"47332569",45,"opendeck",[55,441,33,253,11,34],46,[443,441,43,192,444,33],"47324257","rapatel0",[446,441,33,171,11,34],"47325289",[448,441,38,171,444,33],"47327766",[450,441,38,171,451,33],"47328081","gavinray",[453,441,38,171,454,33],"47325894","skerit",[456,441,43,192,444,33],"47329182",[458,441,38,171,459,33],"47324329","naasking",[461,441,33,171,11,34],"47324922",[463,441,38,171,464,33],"47327788","radarsat1",[466,441,43,196,467,33],"47324606","jauntywundrkind",[469,441,38,253,470,33],"47324724","digdugdirk",[115,441,33,472,11,34],"project\u002Fneutral\u002Fnone",[474,441,43,192,475,33],"47325175","Balinares",[477,441,33,225,478,33],"47326102","gitpusher",[480,441,38,171,219,33],"47326336",[482,441,43,192,483,33],"47327457","pennomi",[485,441,38,171,486,33],"47330862","ay",[488,441,33,225,489,33],"47331202","CuriouslyC",[491,441,33,225,492,33],"47436627","kgeist",[494,441,43,196,495,33],"47325808","afpx",[497,441,43,166,498,33],"47325835","user_7832",[500,441,38,171,501,33],"47327282","phn",[503,441,38,171,504,33],"47331479","nixon_why69",[506,441,43,196,507,33],"47327723","3abiton",[509,441,38,253,510,33],"47329138","himmi-01",[512,441,38,171,513,33],"47329671","lifis",[121,515,38,171,122,33],47,[517,515,33,171,11,34],"47327702",[519,515,38,168,520,33],"47327803","efromvt",[522,515,38,171,275,33],"47352344",[524,515,33,168,11,34],"47365405",[526,527,38,171,528,33],"47325126",48,"patchnull",[530,527,33,168,11,34],"47325176",[532,527,38,171,533,33],"47325903","doctorpangloss",[535,527,33,171,11,34],"47327767",[537,527,43,192,528,33],"47338258",[539,540,43,196,541,33],"47325577",49,"priowise",[543,540,33,213,498,33],"47325842",[545,546,33,225,547,33],"47351602",50,"ozgurozkan",[549,550,33,225,551,33],"47330674",51,"phacker007",[553,550,43,192,554,33],"47331561","bblb",[],[557,562,566,571,575,580],{"id":558,"name":559,"points":560,"category":561},"47336100","Channel Surfer",604,"consumer_social",{"id":563,"name":564,"points":565,"category":20},"47263036","Jido",323,{"id":567,"name":568,"points":569,"category":570},"47337607","Site Spy",320,"productivity",{"id":572,"name":573,"points":574,"category":22},"47300102","Shadowbroker",312,{"id":576,"name":577,"points":578,"category":579},"47275236","Moongate",287,"games",{"id":581,"name":582,"points":583,"category":584},"47280654","realtuner",259,"dev_tools",false,"v2",{"id":588,"name":5,"line":6,"ai":585,"cat":20,"year":13,"points":14,"comments":15,"rec":28,"s":589,"top":18,"kw":590,"maker":11,"domain":10,"tier":19,"attempts":33,"strip":591,"url":9,"mix":592,"praise":83,"objection":106,"sub":21},47322887,0.41,[24,25,26,27],"5534233355355453345354504555344545433355",[31,32,33,34],1794036695755]