[{"data":1,"prerenderedAt":217},["ShallowReactive",2],{"project-44156467":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":9,"maker":10,"posted_at":11,"year":12,"points":13,"comments":14,"rank_in_year":15,"shown_in_year":16,"top":17,"tier":18,"category":19,"subcategory":20,"format":21,"keywords":22,"reception":26,"sentiment":27,"stance_mix":28,"scored_comments":32,"aspects":33,"strip":38,"post_text":39,"what":40,"said":68,"maker_in_thread":136,"shape":146,"grid":149,"attempts":179,"same_week":180,"maker_shows":30,"maker_hidden":209,"reader_version":210,"card":211},"44156467","System Prompt Learning","LLMs Learn Problem-Solving from Experience","title","Show HN: System Prompt Learning – LLMs Learn Problem-Solving from Experience",null,"codelion",1748849342,2025,48,13,1283,3837,34,"shown","ai","prompt optimization","library",[23,24,25,20],"system prompt learning","llm strategies","optillm","curious",-0.25,{"positive":29,"neutral":30,"mixed":29,"critical":31},0,3,2,5,{"execution":34,"design":36,"idea":37},{"positive":29,"critical":35},1,{"positive":29,"critical":29},{"positive":29,"critical":35},[29,35,29,29,29,29,30,29,30,35,29,30,29],"I built a system that lets LLMs automatically learn and improve problem-solving strategies over time, inspired by Andrej Karpathy's idea of a \"third paradigm\" for LLM learning.\n\nThe basic idea: instead of using static system prompts, the LLM builds up a database of strategies that actually work for different problem types. When you give it a new problem, it selects the most relevant strategies, applies them, then evaluates how well they worked and refines them.\n\nFor example, after seeing enough word problems, it learned this strategy:\n\n1) Read carefully and identify unknowns,\n\n2) Define variab",{"problem":41,"audience":47,"pricing":9,"motivation":52,"time_to_build":9,"stack":56,"openness":66,"stage":67,"team_size":67},{"value":42,"source":43,"evidence":44},"Static system prompts miss problem-solving strategies that improve LLM performance","maker",[45],{"id":46},"post",{"value":48,"source":49,"evidence":50},"Developers using OpenAI-compatible LLM APIs with basic prompts","inferred",[51],{"id":46},{"value":53,"source":43,"evidence":54},"Noticed production AI uses detailed system prompts that most developers don't use",[55],{"id":46},[57,63],{"value":58,"source":49,"evidence":59},"Python",[60],{"id":61,"by":10,"text":62},"44158091","You can run in two modes, by default you run in the inference mode without learning. So, the changes you made will be used. If you switch to learning mode then the strategies are…",{"value":25,"source":43,"evidence":64},[65],{"id":46},"open_source","unknown",{"praise":69,"objections":82,"requests":103,"questions":115,"alternatives":127},[70],{"value":71,"source":72,"evidence":73},"Related case-based reasoning work and similar projects are worth looking at","commenter",[74,78],{"id":75,"by":76,"text":77},"44159898","imaltont","You should take a look at something called Case-based reasoning. Seems to perfectly fit into the road you are currently walking, as you basically just rediscovered the CBR-cycle.",{"id":79,"by":80,"text":81},"44157401","tanchaowen84","This is a really cool idea! I recently came across another project on GitHub: https:\u002F\u002Fgithub.com\u002Ftensorzero\u002Ftensorzero that explores a similar direction. You might find it…",[83,90,97],{"value":84,"source":72,"evidence":85},"Benchmark baseline unclear; generic strategies might work as plain prompt additions",[86],{"id":87,"by":88,"text":89},"44157490","yunusabd","That's an interesting space to explore! I'm wondering about the baseline in the benchmarks. Which prompts did you use for those? I'm asking because some of the resulting prompts…",{"value":91,"source":72,"evidence":92},"Unbounded strategy growth is a scaling concern",[93],{"id":94,"by":95,"text":96},"44157927","Falimonda","How do you forsee a system like this efficiently managing and relying on a set of strategies whose size can become unbounded?",{"value":98,"source":72,"evidence":99},"Claimed re-reading strategy may not match the paper's method",[100],{"id":101,"by":88,"text":102},"44157663","Okay, but it looks like in the paper, they are actually adding the question twice in the prompt, not just instructing the model to read it twice. Or am I missing something?",[104,111],{"value":105,"source":72,"evidence":106},"Show input\u002Foutput examples",[107],{"id":108,"by":109,"text":110},"44156821","ramonga","I would like to see some interesting input\u002Foutput pairs. Do you have any?",{"value":112,"source":72,"evidence":113},"Measure which prompt parts contribute most to gains",[114],{"id":87,"by":88,"text":89},[116,123],{"value":117,"source":72,"evidence":118},"What happens when a user manually edits a strategy in learning mode",[119],{"id":120,"by":121,"text":122},"44158057","dedicate","If I jump in and, say, manually 'tweak' one of those JSON strategies because I think I have a better idea, what happens next? Does the LLM just roll with my brilliant human…",{"value":124,"source":72,"evidence":125},"Which baseline prompts were used in the benchmarks",[126],{"id":87,"by":88,"text":89},[128,132],{"value":129,"source":72,"evidence":130},"TensorZero",[131],{"id":79,"by":80,"text":81},{"value":133,"source":72,"evidence":134},"Case-based reasoning",[135],{"id":75,"by":76,"text":77},{"comments":32,"answered":137,"key":138},4,[139,142,145],{"id":140,"text":141},"44156476","Thanks for checking this out! A few additional details that didn't fit in the main post: The system maintains two separate limits: a storage limit (max 10 strategies per problem…",{"id":143,"text":144},"44157556","Re-reading the problem apparently works well - https:\u002F\u002Farxiv.org\u002Fabs\u002F2309.06275 Here the system seems to have discovered this strategy by itself. The prompts are generic because…",{"id":61,"text":62},{"commenters":147,"max_depth":30,"maker_replies":32,"first_comment_mins":31,"lifespan_hours":148},8,201,[150,152,154,155,157,159,162,164,165,167,170,173,176],[140,29,29,151,10,35],"project\u002Fneutral\u002Fexecution",[87,35,35,153,88,29],"project\u002Fcritical\u002Fexecution",[143,35,29,151,10,35],[101,35,29,156,88,29],"commenter\u002Fneutral\u002Fnone",[79,31,29,158,80,29],"other\u002Fneutral\u002Fnone",[160,30,29,158,161,29],"44238868","pratikk10",[120,137,30,163,121,29],"project\u002Fneutral\u002Fdesign",[61,137,29,163,10,35],[75,32,30,166,76,29],"project\u002Fneutral\u002Fidea",[94,168,35,169,95,29],6,"project\u002Fcritical\u002Fidea",[171,168,29,172,10,35],"44157977","maker\u002Fneutral\u002Fdesign",[108,174,30,175,109,29],7,"project\u002Fneutral\u002Fnone",[177,174,29,178,10,35],"44156895","project\u002Fneutral\u002Fdocs",[],[181,186,191,196,200,204],{"id":182,"name":183,"points":184,"category":185},"44157177","Kan.bn",509,"productivity",{"id":187,"name":188,"points":189,"category":190},"44126612","Command Line Handbook",454,"learning_science",{"id":192,"name":193,"points":194,"category":195},"44127653","Onlook",408,"dev_tools",{"id":197,"name":198,"points":199,"category":195},"44123098","Typed-FFmpeg",342,{"id":201,"name":202,"points":203,"category":195},"44159758","Vanta",265,{"id":205,"name":206,"points":207,"category":208},"44147803","Patio",255,"web_saas",false,"v2",{"id":212,"name":5,"line":6,"ai":209,"cat":19,"year":12,"points":13,"comments":14,"rec":26,"s":27,"top":17,"kw":213,"maker":10,"domain":214,"tier":18,"attempts":29,"strip":215,"url":-1,"mix":216,"praise":71,"objection":84,"sub":20},44156467,[23,24,25,20],"","0100003031030",[29,30,29,31],1793991122260]