[{"data":1,"prerenderedAt":914},["ShallowReactive",2],{"project-48842459":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":29,"sentiment":30,"stance_mix":31,"scored_comments":36,"aspects":37,"strip":48,"post_text":51,"what":52,"said":68,"maker_in_thread":185,"shape":197,"grid":202,"attempts":876,"same_week":877,"maker_shows":49,"maker_hidden":906,"reader_version":907,"card":908},"48842459","colibri","Getting GLM 5.2 running on my slow computer","title","Show HN: Getting GLM 5.2 running on my slow computer","https:\u002F\u002Fgithub.com\u002FJustVugg\u002Fcolibri","github.com","vforno",1783584304,2026,937,240,12,4366,1,"legend","ai","local llm inference","cli",[24,25,26,27,28],"local llm","glm 5.2","ssd offloading","mixture of experts","inference engine","curious",0.184,{"positive":32,"neutral":33,"mixed":34,"critical":35},32,81,4,10,127,{"performance":38,"privacy":41,"pricing":43,"idea":44,"docs":45,"execution":46},{"positive":39,"critical":40},6,8,{"positive":18,"critical":42},0,{"positive":42,"critical":18},{"positive":16,"critical":42},{"positive":42,"critical":42},{"positive":47,"critical":42},2,[42,42,49,50,49,50,34,34,42,42,49,42,47,34,50,34,34,42,34,18,34,49,50,34,34,49,49,50,49,50,49,49,34,34,34,49,49,34,49,34],3,5,null,{"problem":53,"audience":51,"pricing":51,"motivation":51,"time_to_build":51,"stack":65,"openness":66,"stage":67,"team_size":67},{"value":54,"source":55,"evidence":56},"Running very large models on consumer hardware with limited RAM","inferred",[57,61],{"id":58,"by":59,"text":60},"48852349","walrus01","My main question is whether when put into practical use, this can be measured in tokens\u002Fsecond, or more like 1 token per minute... I have seen locally hosted LLM that are as slow…",{"id":62,"by":63,"text":64},"48853226","tarpitt","I am curious if it's possible to adjust this to use more RAM, as i've got a machine with 64GB RAM and 24GB VRAM. Or perhaps I could run Gemma\u002FQwen on the GPU and have GLM-5.2…",[],"open_source","unknown",{"praise":69,"objections":96,"requests":124,"questions":144,"alternatives":156},[70,86],{"value":71,"source":72,"evidence":73},"Impressive technical achievement running a huge model on modest hardware","commenter",[74,78,82],{"id":75,"by":76,"text":77},"48852912","stavros","This is great, well done! I love seeing people run things where they weren't meant to be run.",{"id":79,"by":80,"text":81},"48857292","AussieWog93","Well under 1tps, but the fact it runs at all on a 16gb system is incredibly impressive!",{"id":83,"by":84,"text":85},"48854509","barent","Yeah this idea makes instant sense. Very well done, this deserves a github star on concept alone.",{"value":87,"source":72,"evidence":88},"Fits the hacker spirit and an interesting idea",[89,93],{"id":90,"by":91,"text":92},"48852189","miohtama","This is the hacker spirit",{"id":94,"by":84,"text":95},"48854542","The best ideas are the ones that seem obvious. This is one of those ideas.",[97,109],{"value":98,"source":72,"evidence":99},"Speed around 0.05 to 1 token per second is too slow for practical use",[100,101,105],{"id":58,"by":59,"text":60},{"id":102,"by":103,"text":104},"48855796","3abiton","This is technically impressive, but is it usable in practice?",{"id":106,"by":107,"text":108},"48859132","zftnb666","\"Slow computer\" + running a 200B model = \"I'll just wait a week.\"",{"value":110,"source":72,"evidence":111},"Frequent SSD reads may accelerate wear on drives",[112,116,120],{"id":113,"by":114,"text":115},"48853358","voidmain0001","The page has an SSD wear warning [0] I use desktop PCs that I build from components so I can replace the SSD, but what do users with soldered SSD do? Just avoid these…",{"id":117,"by":118,"text":119},"48852314","Pragmata","Would this cause issues with SSD lifespan?",{"id":121,"by":122,"text":123},"48858364","doodlesarefun","is this bad for my SSD?",[125,137],{"value":126,"source":72,"evidence":127},"Support for multiple disks or RAID to speed up reads",[128,129,133],{"id":62,"by":63,"text":64},{"id":130,"by":131,"text":132},"48853798","xtracto","I wonder how would a RAID0 array of either disks or even nvme improve the performance of this.",{"id":134,"by":135,"text":136},"48857724","webprofusion","This is great stuff, I feel like fast disk (SSD) is a somewhat solvable problem, if you have many disks with the same content and a fast controller (?).",{"value":138,"source":72,"evidence":139},"Energy efficiency benchmarks",[140],{"id":141,"by":142,"text":143},"48857672","maxignol","I’m truly impressed by your work ! I don’t know if this is planned for near future, but how about adding energy efficiency benchmarks ? Because running locally is a great…",[145,149],{"value":146,"source":72,"evidence":147},"Can it use more RAM or a GPU alongside SSD streaming",[148],{"id":62,"by":63,"text":64},{"value":150,"source":72,"evidence":151},"Could it run on Ubuntu as a coding agent",[152],{"id":153,"by":154,"text":155},"48892551","kelvinxue","I am curious if it's possible to use coli on ubuntu as coding agent, as i've got a machine with 32GB RAM and 2T SSD.",[157,164,171,178],{"value":158,"source":72,"evidence":159},"llama.cpp with mmap",[160],{"id":161,"by":162,"text":163},"48855363","ac29","llama.cpp supports a wide variety of 4-bit and smaller quants and mmap's models by default, so you dont need to be able to hold the weights in memory (the OS will handle bringing…",{"value":165,"source":72,"evidence":166},"flash-moe",[167],{"id":168,"by":169,"text":170},"48853379","bahmboo","Another recent project that runs a huge model on a 48gb Mac is https:\u002F\u002Fgithub.com\u002Fdanveloper\u002Fflash-moe - it gets over 5 tokens\u002Fsec on an M3 Max compared to this projects very…",{"value":172,"source":72,"evidence":173},"fastllm",[174],{"id":175,"by":176,"text":177},"48854205","nogajun","Is this similar to fastllm? https:\u002F\u002Fgithub.com\u002Fztxz16\u002Ffastllm",{"value":179,"source":72,"evidence":180},"ds4",[181],{"id":182,"by":183,"text":184},"48857597","rullopat","Looking at the progress with DS4 project from Antirez and Colibri, are we probably going in the direction of having a stripe RAID of SSDs instead of RAM to get decent performance…",{"comments":186,"answered":40,"key":187},22,[188,191,194],{"id":189,"text":190},"48852359","In the readme you can see benchmark which everyone with different hardware is running Colibrì, and I have to say I've seen great times! I'm always doing more to improve!",{"id":192,"text":193},"48852471","Maybe we can see some integration!",{"id":195,"text":196},"48854199","Thanks for kind words!",{"commenters":198,"max_depth":199,"maker_replies":186,"first_comment_mins":200,"lifespan_hours":201},157,11,765,120,[203,207,210,213,216,219,222,225,228,231,235,237,239,242,245,248,251,254,256,258,261,263,267,268,272,276,280,284,288,291,293,295,298,300,302,305,308,311,315,319,322,324,326,329,333,336,339,342,346,349,352,354,357,359,361,363,365,368,371,374,377,379,382,385,387,390,393,396,398,400,402,405,407,410,412,414,417,420,423,426,429,432,435,438,442,444,446,449,452,455,458,460,463,467,470,472,474,477,480,483,485,487,489,492,494,497,500,503,506,508,509,513,515,518,520,521,523,526,528,531,534,538,541,543,546,547,549,550,552,554,557,560,563,566,569,572,574,576,578,581,584,586,588,592,596,598,602,604,608,612,615,619,623,626,628,631,634,637,639,641,644,647,650,652,654,658,660,662,664,668,672,675,678,681,685,687,691,693,695,697,701,703,707,710,711,713,716,719,722,724,728,731,735,739,743,745,747,749,751,755,758,762,764,768,770,773,777,781,783,787,791,795,797,801,805,808,810,812,814,816,818,820,822,824,828,833,837,840,842,844,846,848,852,854,857,861,863,867,871,873],[204,42,42,205,206,42],"48857790","other\u002Fneutral\u002Fnone","lopatin",[208,42,42,205,209,42],"48857892","LoganDark",[211,42,42,205,212,42],"48857898","FeepingCreature",[214,42,42,205,215,42],"48857979","lukaslalinsky",[217,42,42,205,218,42],"48860175","jannyfer",[220,42,42,205,221,42],"48859665","abe94",[223,42,42,205,224,42],"48859718","badatnames",[226,42,42,205,227,42],"48864593","lukas9",[229,42,42,205,230,42],"48869134","fibonachos",[232,42,42,233,234,42],"48870593","commenter\u002Fneutral\u002Fnone","taneq",[236,42,42,205,230,42],"48872206",[238,42,42,205,59,42],"48859963",[240,42,42,205,241,42],"48868965","brettpro",[243,42,42,205,244,42],"48871560","kzrdude",[246,42,42,205,247,42],"48872757","arximboldi",[249,42,42,205,250,42],"48860625","culi",[252,42,42,205,253,42],"48862242","rhdunn",[58,18,49,255,59,42],"project\u002Fmixed\u002Fperformance",[189,18,42,257,11,18],"maker\u002Fpositive\u002Fperformance",[259,18,49,260,59,42],"48852434","project\u002Fneutral\u002Fperformance",[192,18,42,262,11,18],"maker\u002Fneutral\u002Fnone",[264,18,50,265,266,42],"48854140","maker\u002Fpositive\u002Fnone","fuzzfactor",[195,18,42,262,11,18],[269,18,42,270,271,42],"48855149","commenter\u002Fpositive\u002Fnone","ProofHouse",[273,18,18,274,275,42],"48853761","project\u002Fcritical\u002Fnone","charcircuit",[277,18,50,278,279,42],"48854531","project\u002Fpositive\u002Fnone","joerawr",[281,18,50,282,283,42],"48870602","project\u002Fpositive\u002Fprivacy","cromka",[285,18,18,286,287,42],"48853795","project\u002Fcritical\u002Fpricing","bigiain",[289,18,42,290,59,42],"48853971","other\u002Fneutral\u002Fpricing",[292,18,42,290,266,42],"48854113",[294,18,42,290,287,42],"48855249",[296,18,42,205,297,42],"48855683","fwipsy",[299,18,42,205,59,42],"48855756",[301,18,50,257,266,42],"48854080",[303,18,42,290,304,42],"48854633","3836293648",[306,18,42,290,307,42],"48856360","kmarc",[309,18,42,290,310,42],"48857899","swiftcoder",[312,18,50,313,314,42],"48853807","project\u002Fpositive\u002Fidea","codazoda",[316,18,49,317,318,42],"48854273","project\u002Fneutral\u002Fidea","alexhans",[320,18,49,317,321,42],"48854752","antondd",[323,18,49,317,59,42],"48854914",[325,18,49,317,76,42],"48857108",[327,18,50,313,328,42],"48854800","digitaltrees",[330,18,49,331,332,42],"48856109","project\u002Fneutral\u002Fnone","ai_fry_ur_brain",[334,18,49,317,335,42],"48856408","0xc133",[337,18,49,317,338,42],"48856716","mft_",[340,18,42,205,341,42],"48861293","wincy",[343,18,49,344,345,42],"48854314","project\u002Fneutral\u002Fpricing","JumpCrisscross",[347,18,42,205,348,42],"48854514","paytonjjones",[350,18,42,205,351,42],"48854612","anon373839",[353,18,42,205,304,42],"48854628",[355,18,42,205,356,42],"48854727","kgeist",[358,18,42,205,304,42],"48871996",[360,18,42,205,356,42],"48888908",[362,18,42,205,304,42],"48900206",[364,18,42,205,351,42],"48856424",[366,18,42,205,367,42],"48854777","tharkun__",[369,18,42,205,370,42],"48856604","mrandish",[372,18,42,233,373,42],"48855156","dyauspitr",[375,18,42,233,376,42],"48855832","kennywinker",[378,18,42,233,351,42],"48856537",[380,18,42,233,381,42],"48877181","alfiedotwtf",[383,18,42,205,384,42],"48855535","stingraycharles",[386,18,49,260,59,42],"48854901",[388,18,49,255,389,42],"48855594","Abishek_Muthian",[391,18,42,392,59,42],"48855745","commenter\u002Fneutral\u002Fprivacy",[394,18,42,392,395,42],"48857453","sandworm101",[397,18,42,392,310,42],"48857826",[399,18,42,290,59,42],"48858151",[401,18,42,290,310,42],"48858216",[403,18,42,290,404,42],"48859210","somenameforme",[406,18,42,290,310,42],"48860034",[408,18,42,392,409,42],"48859138","monegator",[411,18,42,392,310,42],"48860003",[413,18,49,260,59,42],"48856067",[415,18,49,260,416,42],"48856613","puzzlingcaptcha",[418,18,18,419,389,42],"48856967","project\u002Fcritical\u002Fperformance",[421,18,49,260,422,42],"48858596","adrian_b",[424,18,49,260,425,42],"48857071","mcv",[427,18,49,260,428,42],"48857650","dv_dt",[430,18,49,344,431,42],"48872374","inigyou",[433,18,49,317,434,42],"48857452","netdevphoenix",[436,18,42,233,437,42],"48870085","hypfer",[439,18,50,440,441,42],"48856753","project\u002Fpositive\u002Fperformance","zozbot234",[443,18,50,440,422,42],"48858508",[445,18,18,419,59,42],"48866466",[447,18,42,448,422,42],"48869423","commenter\u002Fneutral\u002Fperformance",[450,47,42,205,451,42],"48853412","Archit3ch",[453,47,50,313,454,42],"48853530","sebastianconcpt",[456,47,50,278,457,42],"48853766","fragmede",[459,47,50,278,131,42],"48853770",[461,47,18,419,462,42],"48872913","ukd1",[464,47,49,465,466,42],"48861100","maker\u002Fneutral\u002Fdocs","arm32",[468,49,42,205,469,42],"48852772","Cieric",[471,49,42,265,11,18],"48853100",[473,49,50,278,469,42],"48854204",[475,49,50,313,476,42],"48854236","dicroce",[478,49,49,479,469,42],"48861054","maker\u002Fneutral\u002Fperformance",[481,49,49,260,482,42],"48857911","Roxxik",[484,49,49,479,469,42],"48861037",[486,49,49,260,482,42],"48875355",[488,49,42,448,422,42],"48869465",[490,49,42,205,491,42],"48863131","BigTTYGothGF",[493,49,42,205,469,42],"48867540",[495,49,42,205,496,42],"48873601","embedding-shape",[498,34,42,205,499,42],"48856274","harrouet",[501,50,42,205,502,42],"48852787","kodablah",[504,50,42,205,505,42],"48853342","espetro",[507,50,42,205,502,42],"48853648",[161,39,49,260,162,42],[510,511,50,278,512,42],"48852425",7,"shrinks99",[514,511,42,265,11,18],"48852430",[516,511,49,262,517,42],"48853811","agcat",[519,511,49,344,512,42],"48899359",[182,40,42,205,183,42],[113,522,18,419,114,42],9,[524,522,49,255,525,42],"48853395","hoppp",[527,522,42,262,11,18],"48853410",[529,522,42,205,530,42],"48853407","Cider9986",[532,522,42,205,533,42],"48853619","Gigachad",[535,522,49,536,537,42],"48853426","project\u002Fneutral\u002Fdocs","VortexLain",[539,522,49,536,540,42],"48853662","killerstorm",[542,522,42,262,11,18],"48853724",[544,522,49,536,545,42],"48854250","cjbprime",[90,35,50,313,91,42],[548,35,42,265,11,18],"48852208",[62,199,49,260,63,42],[551,199,42,262,11,18],"48853246",[553,199,42,233,63,42],"48853270",[555,199,49,536,556,42],"48853272","valicord",[558,199,42,233,559,42],"48853316","onlyrealcuzzo",[561,199,42,233,562,42],"48853360","gcr",[564,199,42,233,565,42],"48853481","throwa356262",[567,199,49,536,568,42],"48853341","TacticalCoder",[570,199,42,233,571,42],"48853362","fallingbananna",[573,199,42,262,11,18],"48853293",[575,199,50,440,63,42],"48853419",[577,199,42,265,11,18],"48853496",[579,199,42,233,580,42],"48855967","halcyonblue",[582,199,49,260,583,42],"48862081","throwdbaaway",[585,16,49,317,131,42],"48853786",[587,16,42,262,11,18],"48853792",[589,590,49,344,591,42],"48856284",13,"tw1984",[593,594,50,313,595,42],"48867002",14,"pobonin",[94,597,50,313,84,42],15,[599,600,49,317,601,42],"48852583",16,"bobim",[603,600,42,317,11,18],"48852626",[605,606,49,260,607,42],"48854073",17,"qiqitori",[609,610,42,205,611,42],"48853830",18,"mmastrac",[613,610,42,205,614,42],"48855837","saghm",[616,617,49,317,618,42],"48855589",19,"pianopatrick",[620,621,49,317,622,42],"48854605",20,"SubiculumCode",[624,621,49,317,625,42],"48855094","p-e-w",[627,621,42,233,622,42],"48855230",[629,621,49,260,630,42],"48855421","Infernal",[632,621,42,233,633,42],"48855795","kristianp",[635,621,49,317,636,42],"48857248","xyzsparetimexyz",[638,621,42,233,212,42],"48857914",[640,621,42,233,636,42],"48858174",[642,621,49,317,643,42],"48858397","jeffybefffy519",[645,646,49,260,482,42],"48857125",21,[648,186,49,317,649,42],"48852325","mariopt",[651,186,42,262,11,18],"48852348",[653,186,49,260,297,42],"48867889",[655,656,50,313,657,42],"48870775",23,"ykonstant",[659,656,49,260,244,42],"48873369",[661,656,50,440,244,42],"48897039",[663,656,42,270,657,42],"48903655",[665,666,49,260,667,42],"48857687",24,"AymanB",[669,670,42,205,671,42],"48853854",25,"sakesun",[673,670,49,331,674,42],"48855693","killingtime74",[676,670,42,205,677,42],"48857470","imhoguy",[679,670,42,205,680,42],"48858153","efficax",[682,683,49,344,684,42],"48853567",26,"tannertech",[686,683,42,290,614,42],"48855857",[688,689,50,313,690,42],"48852449",27,"khalic",[692,689,42,265,11,18],"48852481",[79,694,50,440,80,42],28,[141,696,49,260,142,42],29,[698,699,49,536,700,42],"48860093",30,"classified",[702,699,42,536,11,18],"48872075",[704,705,49,260,706,42],"48873109",31,"egberts1",[708,705,18,419,709,42],"48874227","tensegrist",[134,32,50,313,135,42],[175,712,49,331,176,42],33,[714,712,49,317,715,42],"48854269","vikmals",[717,712,49,317,718,42],"48854424","aliljet",[153,720,49,721,154,42],34,"project\u002Fneutral\u002Fexecution",[723,720,49,260,154,42],"48892564",[725,726,49,260,727,42],"48854587",35,"jnaina",[729,726,42,233,730,42],"48855028","simpaticoder",[732,733,49,260,734,42],"48854171",36,"flockonus",[736,737,49,262,738,42],"48874330",37,"andsoitis",[740,741,49,479,742,42],"48852402",38,"xfalcox",[744,741,42,260,11,18],"48852479",[746,741,49,260,59,42],"48852548",[75,748,50,265,76,42],39,[83,750,50,313,84,42],40,[752,753,49,331,754,42],"48857970",41,"_joel",[756,753,42,233,757,42],"48858298","jeffhuys",[759,760,50,313,761,42],"48855355",42,"jhalloran",[130,763,49,260,131,42],43,[765,766,49,262,767,42],"48852234",44,"nerder92",[769,766,42,265,11,18],"48852275",[771,766,42,205,772,42],"48856116","kamranjon",[774,775,18,419,776,42],"48880854",45,"crest",[778,779,50,265,780,42],"48853141",46,"Datagenerator",[782,779,42,265,11,18],"48853154",[784,785,49,260,786,42],"48902133",47,"anoop_kumar",[788,789,50,265,790,42],"48861267",48,"timedude",[792,793,50,794,244,42],"48852363",49,"maker\u002Fpositive\u002Fexecution",[102,796,49,260,103,42],50,[798,799,49,317,800,42],"48858632",51,"valentynkit",[802,803,49,317,804,42],"48856621",52,"vehbiemiroglu",[806,803,18,419,807,42],"48858561","gizajob",[117,809,49,260,118,42],53,[811,809,42,260,11,18],"48852335",[813,809,49,721,118,42],"48852806",[815,809,42,721,11,18],"48853274",[817,809,49,260,451,42],"48853483",[819,809,49,536,742,42],"48852393",[168,821,49,260,169,42],54,[823,821,42,205,244,42],"48873394",[825,826,49,721,827,42],"48898705",55,"kunxue",[829,830,50,831,832,42],"48857828",56,"project\u002Fpositive\u002Fexecution","RahulRajelli",[834,835,49,344,836,42],"48861426",57,"ballislife30",[838,839,49,255,680,42],"48858401",58,[841,839,49,260,807,42],"48858441",[843,839,42,233,680,42],"48858475",[845,839,42,260,11,18],"48872088",[121,847,49,260,122,42],59,[849,850,50,265,851,42],"48858757",60,"bobkb",[106,853,18,419,107,42],61,[855,853,49,260,856,42],"48859170","tarruda",[858,859,42,205,860,42],"48853382",62,"khimaros",[862,859,49,317,451,42],"48853601",[864,865,50,278,866,42],"48857250",63,"jdw64",[868,869,49,260,870,42],"48854377",64,"smetannik",[872,869,42,233,304,42],"48854655",[874,869,42,448,875,42],"48857282","ihateolives",[],[878,883,887,892,897,902],{"id":879,"name":880,"points":881,"category":882},"48845049","18 Words",1160,"games",{"id":884,"name":885,"points":886,"category":882},"48896286","Super Dario",397,{"id":888,"name":889,"points":890,"category":891},"48875377","Ant",330,"languages",{"id":893,"name":894,"points":895,"category":896},"48883305","Juggler",280,"dev_tools",{"id":898,"name":899,"points":900,"category":901},"48926939","Firefox in WebAssembly",273,"creative",{"id":903,"name":904,"points":905,"category":896},"48892859","Clawk",226,false,"v2",{"id":909,"name":5,"line":6,"ai":906,"cat":20,"year":13,"points":14,"comments":15,"rec":29,"s":910,"top":18,"kw":911,"maker":11,"domain":10,"tier":19,"attempts":42,"strip":912,"url":9,"mix":913,"praise":71,"objection":98,"sub":21},48842459,0.18,[24,25,26,27,28],"0035354400302454404143544335353344433434",[32,33,34,35],1794036704552]