[{"data":1,"prerenderedAt":249},["ShallowReactive",2],{"project-38360776":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":20,"keywords":22,"reception":27,"sentiment":28,"stance_mix":29,"scored_comments":34,"aspects":35,"strip":40,"post_text":41,"what":42,"said":65,"maker_in_thread":138,"shape":143,"grid":146,"attempts":211,"same_week":212,"maker_shows":173,"maker_hidden":241,"reader_version":242,"card":243},"38360776","Stella Nera","Maddness Hardware Accelerator","title","Show HN: Stella Nera – Maddness Hardware Accelerator","https:\u002F\u002Fgithub.com\u002Fjoennlae\u002Fhalutmatmul","github.com","joennlae",1700553983,2023,154,26,396,3109,13,"notable","hardware","ai accelerator chip",[23,24,25,26],"maddness","approximate matrix multiplication","hardware accelerator","edge inference","curious",0.276,{"positive":30,"neutral":31,"mixed":32,"critical":33},5,6,0,1,12,{"idea":36,"performance":38,"docs":39},{"positive":37,"critical":32},3,{"positive":33,"critical":33},{"positive":32,"critical":32},[32,30,37,30,30,33,32,32,37,32,37,30,37,37,32,37,32,30,32,32,32,32,32,32,32,32],null,{"problem":43,"audience":50,"pricing":41,"motivation":58,"time_to_build":41,"stack":62,"openness":63,"stage":64,"team_size":63},{"value":44,"source":45,"evidence":46},"Makes neural network matrix multiplication cheaper and more energy-efficient on edge inference chips","maker",[47],{"id":48,"by":11,"text":49},"38361651","Author here: Let me try to give an overview as I saw some questions repeating itself. * This accelerator is for an Edge\u002FInference case, so there is no training on this chip. * We…",{"value":51,"source":52,"evidence":53},"Edge and inference hardware designers, ETH Zurich researchers","inferred",[54,55],{"id":48,"by":11,"text":49},{"id":56,"by":11,"text":57},"38361454","Author here: Thank you for the feedback :-) A lot of the work regarding the comparison with „simple“ approximate matrix multiplication has been done in the preceding paper:…",{"value":59,"source":45,"evidence":60},"Show that Maddness is feasible with good hardware codesign",[61],{"id":48,"by":11,"text":49},[],"unknown","prototype",{"praise":66,"objections":82,"requests":102,"questions":103,"alternatives":129},[67],{"value":68,"source":69,"evidence":70},"Exciting energy-efficiency results and potential for LLMs on devices","commenter",[71,75,78],{"id":72,"by":73,"text":74},"38361155","fxtentacle","I am surprised that they do not mention comparing against quantized matrix multiplication because their \"encoding\" appears to be something like a quantization step with unevenly…",{"id":76,"by":73,"text":77},"38361254","That's the beauty of their method: If you can replace a 8192x8192 matrix multiplication with a 8192x256 decision tree and then a 256x8192 look up table, your memory requirements…",{"id":79,"by":80,"text":81},"38361417","baq","Mind blown. Sounds almost too good to be true except the human brain runs on 20W and this brings us to the same ballpark. This was hard scifi a year ago! Can an approach like…",[83,89,95],{"value":84,"source":45,"evidence":85},"Comparison against TPUv3 is unfair since TPU is a datacenter chip",[86],{"id":87,"by":11,"text":88},"38361549","Thank you for the feedback :-) We have to be careful with the comparisons we make. The TPUv3 is a training and datacenter chip and not an Edge\u002FInference chip. They optimise for a…",{"value":90,"source":69,"evidence":91},"RAM access latency would limit gains on regular CPUs",[92],{"id":93,"by":73,"text":94},"38361532","I'd expect that on a regular CPU, the RAM access latency will destroy any performance improvements. This work is much better suited for FPGAs or ASICs.",{"value":96,"source":69,"evidence":97},"0.5-bit quantization effectiveness questioned versus 4-bit schemes",[98],{"id":99,"by":100,"text":101},"38363942","devit","Based on the first figure in the paper, it seems that this scheme effectively turns 8 input values into a 4-bit number, thus giving an effective 0.5-bit quantization. Considering…",[],[104,111,122],{"value":105,"source":69,"evidence":106},"Where and how the technology can be used and its overall impact",[107],{"id":108,"by":109,"text":110},"38361419","jansan","Ok, I get excited by seeing the numbers, but can someone please explain in a single sentence where this can be used and how big the overall impact would be?",{"value":112,"source":69,"evidence":113},"Whether LoRA-style adaptation is possible given hardcoded matmul",[114,118],{"id":115,"by":116,"text":117},"38361473","ilaksh","This is a dumb question but I guess this means that you can't make something like a LoRA in software, right? Because the network is physically hardcoded?",{"id":119,"by":120,"text":121},"38363336","ddalex","No, the network is not hardcoded, you can build any network you want. Only the expensive step, the matmul, is hardcoded.",{"value":123,"source":69,"evidence":124},"Why the name Stella Nera",[125],{"id":126,"by":127,"text":128},"38361558","pmontra","As you are the author: why the name Stella Nera \u002F Black Star?",[130,134],{"value":131,"source":69,"evidence":132},"Quantized matrix multiplication",[133],{"id":72,"by":73,"text":74},{"value":135,"source":45,"evidence":136},"Binary and ternary neural networks (BNN\u002FTNN) and pruning",[137],{"id":48,"by":11,"text":49},{"comments":37,"answered":32,"key":139},[140,141,142],{"id":48,"text":49},{"id":87,"text":88},{"id":56,"text":57},{"commenters":144,"max_depth":31,"maker_replies":37,"first_comment_mins":145,"lifespan_hours":18},16,46,[147,149,151,155,157,158,160,162,163,165,168,171,176,178,180,182,183,186,189,192,195,197,200,202,205,207],[48,32,32,148,11,33],"maker\u002Fneutral\u002Fidea",[72,33,30,150,73,32],"project\u002Fpositive\u002Fidea",[152,33,37,153,154,32],"38361221","project\u002Fneutral\u002Fperformance","joelthelion",[76,33,30,156,73,32],"project\u002Fpositive\u002Fperformance",[79,33,30,150,80,32],[93,33,33,159,73,32],"project\u002Fcritical\u002Fperformance",[87,33,32,161,11,33],"commenter\u002Fneutral\u002Fperformance",[56,33,32,148,11,33],[126,33,37,164,127,32],"maker\u002Fneutral\u002Fnone",[166,33,32,167,73,32],"38361637","other\u002Fneutral\u002Fnone",[169,33,37,170,73,32],"38361612","project\u002Fneutral\u002Fdocs",[172,173,30,174,175,32],"38361116",2,"project\u002Fpositive\u002Fnone","ggambetta",[108,37,37,177,109,32],"project\u002Fneutral\u002Fidea",[115,179,37,177,116,32],4,[119,179,32,181,120,32],"commenter\u002Fneutral\u002Fidea",[99,30,37,153,100,32],[184,30,32,161,185,32],"38370091","jhj",[187,31,30,150,188,32],"38361156","larodi",[190,31,32,191,73,32],"38361186","other\u002Fneutral\u002Fperformance",[193,31,32,161,194,32],"38361488","maizeq",[196,31,32,191,73,32],"38361718",[198,31,32,191,199,32],"38363282","zith",[201,31,32,191,73,32],"38363850",[203,31,32,167,204,32],"38361740","LeanderK",[206,31,32,167,73,32],"38361765",[208,31,32,209,210,32],"38361393","other\u002Fcritical\u002Fperformance","dvh",[],[213,218,223,227,232,237],{"id":214,"name":215,"points":216,"category":217},"38327017","Ad Accelerator",713,"consumer_social",{"id":219,"name":220,"points":221,"category":222},"38316936","UneeBee",459,"learning_science",{"id":224,"name":225,"points":226,"category":222},"38365638","Pickcode",420,{"id":228,"name":229,"points":230,"category":231},"38285228","screenshot-to-code",358,"dev_tools",{"id":233,"name":234,"points":235,"category":236},"38368570","Neum AI",155,"ai",{"id":238,"name":239,"points":240,"category":217},"38295535","David Adler personal landing page",152,false,"v2",{"id":244,"name":5,"line":6,"ai":241,"cat":20,"year":13,"points":14,"comments":15,"rec":27,"s":245,"top":18,"kw":246,"maker":11,"domain":10,"tier":19,"attempts":32,"strip":247,"url":9,"mix":248,"praise":68,"objection":84,"sub":21},38360776,0.28,[23,24,25,26],"05355100303533030500000000",[30,31,32,33],1794036644963]