[{"data":1,"prerenderedAt":398},["ShallowReactive",2],{"project-44112326":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":9,"maker":10,"posted_at":11,"year":12,"points":13,"comments":14,"rank_in_year":15,"shown_in_year":16,"top":17,"tier":18,"category":19,"subcategory":20,"format":21,"keywords":22,"reception":27,"sentiment":28,"stance_mix":29,"scored_comments":33,"aspects":34,"strip":39,"post_text":42,"what":43,"said":75,"maker_in_thread":164,"shape":174,"grid":178,"attempts":359,"same_week":360,"maker_shows":38,"maker_hidden":389,"reader_version":390,"card":391},"44112326","AutoThink","Boosts local LLM performance with adaptive reasoning","title","Show HN: AutoThink – Boosts local LLM performance with adaptive reasoning",null,"codelion",1748399951,2025,397,68,118,3837,4,"legend","ai","reasoning optimization","library",[23,24,25,26],"local llm","adaptive reasoning","steering vectors","token budget","mixed",0.294,{"positive":30,"neutral":31,"mixed":32,"critical":17},10,8,1,23,{"idea":35,"performance":37},{"positive":36,"critical":32},9,{"positive":32,"critical":38},3,[40,40,41,40,40,40,40,40,40,41,40,41,40,38,32,38,40,40,38,41,41,38,41,41,40,40,40,32,32,38,40,40,40,40,38,40,40,38,41,32],0,5,"I built AutoThink, a technique that makes local LLMs reason more efficiently by adaptively allocating computational resources based on query complexity.\n\nThe core idea: instead of giving every query the same \"thinking time,\" classify queries as HIGH or LOW complexity and allocate thinking tokens accordingly. Complex reasoning gets 70-90% of tokens, simple queries get 20-40%.\n\nI also implemented steering vectors derived from Pivotal Token Search (originally from Microsoft's Phi-4 paper) that guide the model's reasoning patterns during generation. These vectors encourage behaviors like numerical",{"problem":44,"audience":9,"pricing":9,"motivation":51,"time_to_build":9,"stack":55,"openness":73,"stage":74,"team_size":74},{"value":45,"source":46,"evidence":47},"Reasoning models spend the same thinking budget on simple and complex queries","maker",[48],{"id":49,"by":10,"text":50},"44112333","The motivation for AutoThink came from watching how current reasoning models waste computation - they spend the same amount of \"thinking time\" on \"what's 2+2?\" as they do on…",{"value":52,"source":46,"evidence":53},"Reasoning models waste computation on easy questions",[54],{"id":49,"by":10,"text":50},[56,62,67],{"value":57,"source":46,"evidence":58},"PyTorch",[59],{"id":60,"by":10,"text":61},"44156481","Optillm works with llama.cpp but this approach is implemented as a decoding strategy in PyTorch so at the moment you will need to use the local inference server in optillm to use…",{"value":63,"source":46,"evidence":64},"DeepSeek-R1-Distill-Qwen-1.5B",[65],{"id":66},"post",{"value":68,"source":46,"evidence":69},"DistilBERT classifier",[70],{"id":71,"by":10,"text":72},"44114469","We use an adaptive classifier to learn how many tokens the model takes to respond correctly on a known dataset. I used the https:\u002F\u002Fhuggingface.co\u002Fadaptive-classifier\u002Fllm-router…","open_source","unknown",{"praise":76,"objections":92,"requests":113,"questions":132,"alternatives":145},[77,85],{"value":78,"source":79,"evidence":80},"Obvious optimisation, good to see it written up","commenter",[81],{"id":82,"by":83,"text":84},"44112490","nssnsjsjsjs","This is an obvious optimisation. Surprised this isn't been done already. Good job writing it up and showing how it can be done.",{"value":86,"source":79,"evidence":87},"Interesting work toward competitive open reasoning models",[88],{"id":89,"by":90,"text":91},"44115373","CharlesW","> I think the same goes for o3. Definitely, in my experience. Elsewhere in the thread, OP says that open models\u002Fsystems don't do this, in which case this seems like important…",[93,103],{"value":94,"source":79,"evidence":95},"Classifying difficulty by token count can misjudge hard questions and trade correctness for speed",[96,100],{"id":97,"by":98,"text":99},"44113157","bufferoverflow","But how do you classify a question as high vs low complexity? Some seemingly simple questions can turn out to be very very complex. For example, integer solution to ``` x³ + y³ +…",{"id":101,"by":98,"text":102},"44113218","So it can be very very wrong. You're trading correctness for speed.",{"value":104,"source":79,"evidence":105},"Benchmark setup differs from official results (GPQA-Diamond 33.8 reported)",[106,110],{"id":107,"by":108,"text":109},"44115367","lostmsu","Official https:\u002F\u002Fhuggingface.co\u002Fdeepseek-ai\u002FDeepSeek-R1-Distill-Qwen-1.5B says GPQA-Diamond is 33.8",{"id":111,"by":108,"text":112},"44116580","GPQA-Diamond is 200 questions. Any GPU since 2019 with 12GB of VRAM should be able to run tens if not hundreds of queries for a 1.5B model in parallel.",[114,125],{"value":115,"source":79,"evidence":116},"Explain how the complexity classifier works and generalizes",[117,121],{"id":118,"by":119,"text":120},"44114458","Dowwie","Hey, this is really interesting. What are the features you used to measure the reasoning complexity? In other words, how does one evaluate a query during classification?",{"id":122,"by":123,"text":124},"44114153","VagabundoP","If you ask it to rethink the problem again because you've found a flaw, does it bump up the complexity and actually think about it. Like a person might give you a quick answer to…",{"value":126,"source":79,"evidence":127},"Support for llama.cpp",[128],{"id":129,"by":130,"text":131},"44154695","shirman","Hi, it does not work with llama.cpp right?",[133,138],{"value":134,"source":79,"evidence":135},"How is query complexity measured during classification?",[136,137],{"id":118,"by":119,"text":120},{"id":97,"by":98,"text":99},{"value":139,"source":79,"evidence":140},"Does it help local users who want maximum accuracy?",[141],{"id":142,"by":143,"text":144},"44113283","vintermann","If host models for others, then sure, I'm happy to save some computation time for really simple queries. Sure the cost is that the model will be effectively dismissive of…",[146,157],{"value":147,"source":79,"evidence":148},"Gemini 2.5 Pro and o3 already vary thinking time",[149,153],{"id":150,"by":151,"text":152},"44112558","behnamoh","> they spend the same amount of \"thinking time\" on \"what's 2+2?\" as they do on complex mathematical proofs. Not anymore. Have you seen Gemini 2.5 Pro? Ask it simple questions and…",{"id":154,"by":155,"text":156},"44113384","MagicMoonlight","You didn’t invent this. Models like o3 already do it, that’s why the amount of thinking time varies.",{"value":158,"source":79,"evidence":159},"Earlier autothink POC using LLM-assigned complexity score",[160],{"id":161,"by":162,"text":163},"44115504","NiloCK","I, too, built a POC autothink shortly after the Claude 3.7 release that included the `extended thinking` toggle. It's literally also called autothink:…",{"comments":165,"answered":41,"key":166},11,[167,168,171],{"id":49,"text":50},{"id":169,"text":170},"44112589","Yes, we started with the idea of trying to replicate similar control on thinking processes for open reasoning models. They also announced the Deep Think approach at IO which goes…",{"id":172,"text":173},"44116048","This is an interesting idea, I hadn't thought of it. It is worth experimenting I am not aware of anyone else trying it yet.",{"commenters":175,"max_depth":36,"maker_replies":165,"first_comment_mins":176,"lifespan_hours":177},50,2,199,[179,181,183,185,188,192,195,198,202,205,208,210,212,215,218,221,222,225,228,231,235,237,240,241,243,245,248,249,252,255,258,261,264,265,266,269,272,275,278,282,285,288,291,294,297,300,301,303,304,306,308,309,313,316,318,321,323,325,327,331,334,337,340,342,344,349,353,356],[49,40,40,180,10,32],"maker\u002Fneutral\u002Fnone",[150,40,40,182,151,40],"other\u002Fneutral\u002Fidea",[169,40,40,184,10,32],"other\u002Fneutral\u002Fnone",[186,40,40,182,187,40],"44112996","sigmoid10",[189,40,41,190,191,40],"44113111","project\u002Fpositive\u002Fidea","shing3232",[193,40,40,182,194,40],"44113438","victorbjorklund",[196,40,40,182,197,40],"44114272","normie3000",[199,40,40,200,201,40],"44114820","commenter\u002Fneutral\u002Fnone","diggan",[203,40,40,200,204,40],"44116148","GTP",[206,40,40,200,207,40],"44120266","TeMPOraL",[209,40,40,200,204,40],"44190047",[211,40,40,182,194,40],"44130723",[213,40,40,182,214,40],"44114479","CjHuber",[216,40,40,182,217,40],"44114717","thegeomaster",[219,40,40,182,220,40],"44115702","vladf",[89,40,41,190,90,40],[223,40,40,182,224,40],"44115893","olddustytrail",[226,40,40,182,227,40],"44115984","mclau157",[229,40,41,190,230,40],"44112844","Abishek_Muthian",[232,40,38,233,234,40],"44116020","project\u002Fneutral\u002Fidea","Lerc",[172,40,40,236,10,32],"commenter\u002Fneutral\u002Fidea",[238,40,40,182,239,40],"44117281","waffletower",[97,32,38,233,98,40],[242,32,40,233,10,32],"44113194",[101,32,32,244,98,40],"project\u002Fcritical\u002Fperformance",[246,32,40,247,10,32],"44113262","project\u002Fneutral\u002Fperformance",[122,32,38,233,123,40],[250,32,40,251,10,32],"44114213","project\u002Fneutral\u002Fnone",[253,32,40,200,254,40],"44113544","baobabKoodaa",[256,32,40,200,257,40],"44115834","wat10000",[259,32,40,184,260,40],"44120769","xigency",[262,32,38,233,263,40],"44121186","MrManatee",[161,176,41,190,162,40],[82,38,41,190,83,40],[267,17,41,190,268,40],"44114284","mentalgear",[270,17,40,200,271,40],"44115033","chrisweekly",[273,41,38,233,274,40],"44117358","CMay",[142,276,38,277,143,40],6,"project\u002Fmixed\u002Fidea",[279,280,41,190,281,40],"44117128",7,"GENIXUS",[283,31,41,190,284,40],"44112688","shah_akshat",[286,36,40,182,287,40],"44120047","SamScout",[289,30,40,184,290,40],"44114265","casenmgreen",[292,30,40,184,293,40],"44114760","falcor84",[295,30,40,184,296,40],"44114856","dgb23",[298,30,40,184,299,40],"44114889","dymk",[107,165,32,244,108,40],[302,165,40,247,10,32],"44115950",[111,165,32,244,108,40],[305,165,40,247,10,32],"44116750",[118,307,38,233,119,40],12,[71,307,40,233,10,32],[310,311,40,182,312,40],"44113551",13,"shwouchk",[314,311,40,184,315,40],"44114444","NitpickLawyer",[317,311,40,184,312,40],"44121489",[319,311,40,184,320,40],"44117286","throwaway314155",[322,311,40,200,312,40],"44121506",[324,311,40,200,320,40],"44130650",[326,311,40,200,312,40],"44167324",[328,329,38,233,330,40],"44112574",14,"transfire",[332,329,40,184,333,40],"44112587","pkoird",[335,329,40,184,336,40],"44112993","knuppar",[338,329,40,339,10,32],"44112593","maker\u002Fneutral\u002Fidea",[129,341,38,247,130,40],15,[60,341,40,343,10,32],"maker\u002Fneutral\u002Fexecution",[345,346,41,347,348,40],"44112728",16,"project\u002Fpositive\u002Fperformance","danielhanchen",[350,351,41,190,352,40],"44112897",17,"keeganpoppen",[154,354,32,355,155,40],18,"project\u002Fcritical\u002Fidea",[357,354,40,236,358,40],"44113428","rohansood15",[],[361,366,370,375,380,384],{"id":362,"name":363,"points":364,"category":365},"44118023","Desktop Docs",597,"productivity",{"id":367,"name":368,"points":369,"category":19},"44110584","simonwillison.net",529,{"id":371,"name":372,"points":373,"category":374},"44103839","Lazy Tetris",438,"games",{"id":376,"name":377,"points":378,"category":379},"44067409","Defuddle",418,"languages",{"id":381,"name":382,"points":383,"category":374},"44101175","Porting Terraria and Celeste to WebAssembly",338,{"id":385,"name":386,"points":387,"category":388},"44080803","rotary_dial_kmod",337,"hardware",false,"v2",{"id":392,"name":5,"line":6,"ai":389,"cat":19,"year":12,"points":13,"comments":14,"rec":27,"s":393,"top":17,"kw":394,"maker":10,"domain":395,"tier":18,"attempts":40,"strip":396,"url":-1,"mix":397,"praise":78,"objection":94,"sub":20},44112326,0.29,[23,24,25,26],"","0050000005050313003553550001130000300351",[30,31,32,17],1794036675734]