[{"data":1,"prerenderedAt":229},["ShallowReactive",2],{"project-49293523":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":29,"sentiment":30,"stance_mix":31,"scored_comments":35,"aspects":36,"strip":43,"post_text":45,"what":46,"said":61,"maker_in_thread":130,"shape":139,"grid":143,"attempts":189,"same_week":194,"maker_shows":34,"maker_hidden":221,"reader_version":222,"card":223},"49293523","Lumabri","Run Moe Models on a P2P Swarm with Colibri","title","Show HN: Lumabri – Run Moe Models on a P2P Swarm with Colibri","https:\u002F\u002Fgithub.com\u002FJustVugg\u002Flumabri","github.com","vforno",1786668424,2026,48,19,1246,4366,29,"shown","ai","distributed llm inference","other",[24,25,26,27,28],"mixture of experts","p2p inference","distributed llm","local llm","expert offloading","debated",0.205,{"positive":32,"neutral":32,"mixed":33,"critical":34},4,1,3,12,{"docs":37,"idea":39,"privacy":41,"execution":42},{"positive":38,"critical":38},0,{"positive":40,"critical":38},2,{"positive":33,"critical":38},{"positive":38,"critical":34},[34,44,34,38,34,38,44,34,38,34,38,33,33,33,38,44,38,44,38],5,null,{"problem":47,"audience":45,"pricing":45,"motivation":45,"time_to_build":45,"stack":59,"openness":60,"stage":60,"team_size":60},{"value":48,"source":49,"evidence":50},"Running large Mixture-of-Experts models on machines with limited memory by spreading experts across a swarm","inferred",[51,55],{"id":52,"by":53,"text":54},"49299258","gebdev","I’ve been thinking about this same idea recently, so I’m glad it exists now! The biggest benefit I see is to enable RAM constrained GPUs to perform inference of large parameter…",{"id":56,"by":57,"text":58},"49296993","brainless","I am sorry I did not understand all of it. But, would this allow running large MoE LLMs on a local network with experts spread out over multiple cheaper GPUs (or even CPUs)? This…",[],"unknown",{"praise":62,"objections":83,"requests":103,"questions":115,"alternatives":122},[63,76],{"value":64,"source":65,"evidence":66},"Interesting idea for distributed local inference","commenter",[67,71,72],{"id":68,"by":69,"text":70},"49297634","SCHiM","This looks really interesting, and if I understand what this does properly: it was high time someone built this! Without diving into an experiment myself, it would be amazing if…",{"id":52,"by":53,"text":54},{"id":73,"by":74,"text":75},"49298041","bicepjai","Great work. Love the idea. I have been thinking along the same way, but for training. For inference, the waiting time might be turn off for users",{"value":77,"source":65,"evidence":78},"Useful for running models on a LAN with spare hardware",[79,80],{"id":56,"by":57,"text":58},{"id":81,"by":11,"text":82},"49297108","That’s one of the strongest use-cases. On a local network (office, lab, home cluster) the RTT is a few milliseconds instead of 20-50 ms, so the expert-offloading becomes much…",[84,99],{"value":85,"source":65,"evidence":86},"Floating-point results may not be deterministic across hardware, weakening verification",[87,91,95],{"id":88,"by":89,"text":90},"49297307","stymaar","> Expert execution is deterministic (pure matmul). Isn't that only true in theory but wrong in practice due to floating points?",{"id":92,"by":93,"text":94},"49299915","nikunjbjj","The harder case isn't same-hardware determinism; it's that lumabri wants CPU and GPU peers in the same swarm. Accumulation order differs across hardware, not just across runs.",{"id":96,"by":97,"text":98},"49308909","snovv_crash","There are fixed-point models too which can be deterministic. But for floating point you are very instruction-set dependent, never mind floating point operations not being…",{"value":100,"source":65,"evidence":101},"Latency over the internet may be high",[102],{"id":52,"by":53,"text":54},[104,108],{"value":105,"source":65,"evidence":106},"Add performance stats and experiment logs",[107],{"id":68,"by":69,"text":70},{"value":109,"source":65,"evidence":110},"Link explicit Hugging Face model repos instead of vague model names",[111],{"id":112,"by":113,"text":114},"49301077","cyanydeez","so looking at the start docs, you should integrating the same llamacpp setup where they just bake in huggingface support. second, don't use vague names of models; point to the…",[116],{"value":117,"source":65,"evidence":118},"How is temperature handled in verification",[119],{"id":120,"by":97,"text":121},"49296735","Cool idea. How do you handle temperature in the verification?",[123],{"value":124,"source":65,"evidence":125},"meshllm",[126],{"id":127,"by":128,"text":129},"49299115","krautsauer","On the surface it seems similar to https:\u002F\u002Fmeshllm.cloud\u002F, but now I notice that I don't understand either.",{"comments":44,"answered":34,"key":131},[132,133,136],{"id":81,"text":82},{"id":134,"text":135},"49297119","Because everything stays local and latency is tiny, even modest always-on devices can contribute. A few Raspberry Pi 5s, old mini-PCs, or stronger IoT-style boards can each hold…",{"id":137,"text":138},"49296946","Ho thanks for the comment. Verification does not depend on temperature. Expert execution is deterministic (pure matmul). LUMABRI_VERIFY=N re-runs N% of the calls on a second…",{"commenters":140,"max_depth":32,"maker_replies":44,"first_comment_mins":141,"lifespan_hours":142},14,470,32,[144,146,148,150,152,155,156,160,162,165,168,170,172,173,174,178,180,183,187],[68,38,34,145,69,38],"project\u002Fneutral\u002Fdocs",[52,33,44,147,53,38],"project\u002Fpositive\u002Fidea",[56,40,34,149,57,38],"project\u002Fneutral\u002Fidea",[81,40,38,151,11,33],"maker\u002Fneutral\u002Fidea",[153,34,34,149,154,38],"49297005","s2l",[134,34,38,151,11,33],[157,32,44,158,159,38],"49296469","project\u002Fpositive\u002Fprivacy","ptsneves",[112,44,34,161,113,38],"project\u002Fmixed\u002Fdocs",[127,163,38,164,128,38],6,"other\u002Fneutral\u002Fnone",[120,166,34,167,97,38],7,"project\u002Fneutral\u002Fexecution",[137,166,38,169,11,33],"maker\u002Fneutral\u002Fexecution",[88,166,33,171,89,38],"project\u002Fcritical\u002Fexecution",[92,166,33,171,93,38],[96,166,33,171,97,38],[175,176,38,164,177,38],"49296717",8,"Scaled",[73,179,44,147,74,38],9,[181,179,38,182,11,33],"49298182","maker\u002Fpositive\u002Fnone",[184,185,44,182,186,38],"49296085",10,"RIshabh235",[188,185,38,182,11,33],"49296449",[190,193],{"id":191,"t":192,"points":179,"comments":185},"49236781",1786314264,{"id":4,"t":12,"points":14,"comments":15},[195,200,205,209,213,217],{"id":196,"name":197,"points":198,"category":199},"49327643","Desktopcolors.com",166,"learning_science",{"id":201,"name":202,"points":203,"category":204},"49355607","Lucasartsifier",161,"dev_tools",{"id":206,"name":207,"points":208,"category":20},"49307700","ThoughtDAG",136,{"id":210,"name":211,"points":212,"category":20},"49354664","modelmap",113,{"id":214,"name":215,"points":216,"category":204},"49364296","terminal-code",108,{"id":218,"name":219,"points":220,"category":204},"49301215","LuaCAD",106,false,"v2",{"id":224,"name":5,"line":6,"ai":221,"cat":20,"year":13,"points":14,"comments":15,"rec":29,"s":225,"top":18,"kw":226,"maker":11,"domain":10,"tier":19,"attempts":40,"strip":227,"url":9,"mix":228,"praise":64,"objection":85,"sub":21},49293523,0.21,[24,25,26,27,28],"3530305303011105050",[32,32,33,34],1794036707782]