[{"data":1,"prerenderedAt":209},["ShallowReactive",2],{"project-45442177":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":28,"sentiment":29,"stance_mix":30,"scored_comments":35,"aspects":36,"strip":40,"post_text":42,"what":43,"said":83,"maker_in_thread":140,"shape":148,"grid":150,"attempts":171,"same_week":172,"maker_shows":35,"maker_hidden":201,"reader_version":202,"card":203},"45442177","Butter","A muscle memory cache for LLMs","title","Show HN: Butter, a muscle memory cache for LLMs","https:\u002F\u002Fdocs.butter.dev","docs.butter.dev","edunteman",1759346998,2025,23,11,1886,3837,50,"shown","ai","llm caching proxy","api",[24,25,26,27],"llm cache","openai proxy","deterministic replay","agent automation","mixed",0.386,{"positive":31,"neutral":32,"mixed":33,"critical":34},4,3,0,1,8,{"idea":37,"docs":38,"execution":39},{"positive":32,"critical":33},{"positive":33,"critical":34},{"positive":34,"critical":33},[41,41,33,32,33,34,41,41,32,33,32],5,"Hi HN, Erik here. Today we launch Butter, an OpenAI-compatible API proxy that caches LLM generations and serves them deterministically on revisit.\n\nSince April, we’ve been working on this concept of “muscle memory,” or deterministic replay, for agent systems performing automations. You may recall our first post in May, launching a python package called Muscle Mem: https:\u002F\u002Fnews.ycombinator.com\u002Fitem?id=43988381\n\nSince then, the product has evolved entirely, now taking the form of an LLM Proxy. For a deep dive into this process, check out: https:\u002F\u002Fblog.butter.dev\u002Fmuscle-mem-as-a-proxy\n\nThe proxy’",{"problem":44,"audience":53,"pricing":64,"motivation":69,"time_to_build":74,"stack":75,"openness":80,"stage":81,"team_size":82},{"value":45,"source":46,"evidence":47},"Repeated LLM generations for structurally similar agent requests cost tokens and time","inferred",[48,50],{"id":49},"post",{"id":51,"by":11,"text":52},"45444613","I've got a blog on this from the launch of Muscle Mem, which should paint a better picture https:\u002F\u002Ferikdunteman.com\u002Fblog\u002Fmuscle-mem Computer use agents (as an RPA alternative) is…",{"value":54,"source":46,"evidence":55},"Developers building LLM agents and automations",[56,60],{"id":57,"by":58,"text":59},"45442674","tsvoboda","looks pretty cool! How would you integrate this into production agent stacks like langchain, autogpt, even closed loop robotics?",{"id":61,"by":62,"text":63},"45442897","raymondtana","For AutoGPT, there is the option to set a llamafile endpoint, which follows the Chat Completions API. So, theoretically, you should be able to use that to point to Butter's LLM…",{"value":65,"source":66,"evidence":67},"Currently open-access and free to use","maker",[68],{"id":49},{"value":70,"source":66,"evidence":71},"Agent workloads repeat known paths, so replaying cached generations can save work",[72,73],{"id":49},{"id":51,"by":11,"text":52},null,[76],{"value":77,"source":66,"evidence":78},"OpenAI-compatible API",[79],{"id":49},"free","launched","unknown",{"praise":84,"objections":104,"requests":115,"questions":126,"alternatives":131},[85,97],{"value":86,"source":87,"evidence":88},"Simple proxy design makes integration easy via a URL swap","commenter",[89,93],{"id":90,"by":91,"text":92},"45444045","MorganGallant","I've known Erik for a while now — simply incredible founder. Doing this as a simple API proxy makes this practically effortless to integrate into existing systems, just a simple…",{"id":94,"by":95,"text":96},"45442511","ketan_around","Exciting to see a product like this launch! There are obviously a host of ‘memory’ solutions out there that try to integrate in fancy ways to cache knowledge \u002F save tokens, but I…",{"value":98,"source":87,"evidence":99},"Idea of cache reuse for repetitive LLM inference is promising",[100],{"id":101,"by":102,"text":103},"45443406","samraaj","logged back in to HN to comment on this. looks really sick - i've been saying for a while that a surprising amount of LLM inference really comes down to repetition down a known…",[105],{"value":106,"source":87,"evidence":107},"Value proposition and what gets cached is unclear",[108,112],{"id":109,"by":110,"text":111},"45445096","bigwheels","Still not clear - the tool calls come from the model, so what is being cached by Muscle Memory? Also: ``` After my time building computer-use agents, I’m convinced that the…",{"id":113,"by":110,"text":114},"45444461","Are you able to walk through a specific use case or example case in detail? I'm not yet totally grokking what Butter is going to do exactly.",[116,120],{"value":117,"source":87,"evidence":118},"Concrete walkthrough of a use case",[119],{"id":113,"by":110,"text":114},{"value":121,"source":87,"evidence":122},"Inject prompt cache tokens to save cost rather than bypass generations",[123],{"id":124,"by":11,"text":125},"45442693","An interesting alternative product to offer is injecting prompt cache tokens into requests where they could be helpful; not bypassing generations but at least low hanging fruit…",[127],{"value":128,"source":87,"evidence":129},"How to integrate with LangChain, AutoGPT and robotics stacks",[130],{"id":57,"by":58,"text":59},[132,136],{"value":133,"source":46,"evidence":134},"Prompt caching features offered by LLM providers",[135],{"id":124,"by":11,"text":125},{"value":137,"source":87,"evidence":138},"Memory solutions for LLM knowledge caching",[139],{"id":94,"by":95,"text":96},{"comments":32,"answered":141,"key":142},2,[143,144,145],{"id":124,"text":125},{"id":51,"text":52},{"id":146,"text":147},"45442770","Thanks! For langchain you can repoint your base_url in the client. Autogpt I'm not as familiar with. Closed loop robotics using LLMs may be a stretch for now, especially since…",{"commenters":35,"max_depth":32,"maker_replies":32,"first_comment_mins":149,"lifespan_hours":41},21,[151,153,155,157,159,160,162,163,167,168,169],[90,33,41,152,91,33],"maker\u002Fpositive\u002Fidea",[94,34,41,154,95,33],"project\u002Fpositive\u002Fidea",[124,34,33,156,11,34],"project\u002Fneutral\u002Fidea",[113,141,32,158,110,33],"project\u002Fneutral\u002Fdocs",[51,141,33,156,11,34],[109,141,34,161,110,33],"project\u002Fcritical\u002Fdocs",[101,32,41,154,102,33],[164,31,41,165,166,33],"45444290","project\u002Fpositive\u002Fexecution","zyadelgohary1",[57,41,32,156,58,33],[146,41,33,156,11,34],[61,41,32,170,62,33],"project\u002Fneutral\u002Fexecution",[],[173,178,183,188,192,196],{"id":174,"name":175,"points":176,"category":177},"45438346","Autism Simulator",779,"learning_science",{"id":179,"name":180,"points":181,"category":182},"45404667","Toolbrew",269,"dev_tools",{"id":184,"name":185,"points":186,"category":187},"45430294","Glide",207,"consumer_social",{"id":189,"name":190,"points":191,"category":177},"45384617","linear algebra",190,{"id":193,"name":194,"points":195,"category":182},"45427697","Sculptor",176,{"id":197,"name":198,"points":199,"category":200},"45431115","Cobalt",174,"creative",false,"v2",{"id":204,"name":5,"line":6,"ai":201,"cat":20,"year":13,"points":14,"comments":15,"rec":28,"s":205,"top":18,"kw":206,"maker":11,"domain":10,"tier":19,"attempts":33,"strip":207,"url":9,"mix":208,"praise":86,"objection":106,"sub":21},45442177,0.39,[24,25,26,27],"55030155303",[31,32,33,34],1793991131147]