[{"data":1,"prerenderedAt":261},["ShallowReactive",2],{"project-45876742":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":29,"sentiment":30,"stance_mix":31,"scored_comments":36,"aspects":37,"strip":42,"post_text":44,"what":45,"said":65,"maker_in_thread":165,"shape":171,"grid":176,"attempts":219,"same_week":224,"maker_shows":32,"maker_hidden":253,"reader_version":254,"card":255},"45876742","Tiny Diffusion","A character-level text diffusion model from scratch","title","Show HN: Tiny Diffusion – A character-level text diffusion model from scratch","https:\u002F\u002Fgithub.com\u002Fnathan-barry\u002Ftiny-diffusion","github.com","nathan-barry",1762787635,2025,172,20,404,3837,11,"notable","ai","text diffusion model","other",[24,25,26,27,28],"text diffusion","diffusion language model","character-level model","tiny shakespeare","nanochat","curious",0.185,{"positive":32,"neutral":33,"mixed":34,"critical":35},3,9,0,1,13,{"design":38,"idea":39,"execution":40,"performance":41},{"positive":35,"critical":34},{"positive":34,"critical":34},{"positive":35,"critical":35},{"positive":34,"critical":34},[43,32,32,32,34,32,32,32,32,43,34,34,34,34,35,34,34,32,43,32],5,"This is a character-level language diffusion model for text generation.\n\nThe model is a modified version of Nanochat's GPT implementation and is trained on Tiny Shakespeare!\n\nIt is only 10.7 million parameters, so you can try it out locally.",{"problem":46,"audience":46,"pricing":46,"motivation":46,"time_to_build":46,"stack":47,"openness":62,"stage":63,"team_size":64},null,[48,56],{"value":49,"source":50,"evidence":51},"Python","inferred",[52],{"id":53,"by":54,"text":55},"45931624","simonw","This is really neat. I noticed the diffusion-process.py demo was using matplotlib in a window, but I figured it would be cute if it used a terminal UI instead - so I had Claude…",{"value":57,"source":58,"evidence":59},"Nanochat GPT implementation as base","maker",[60],{"id":61},"post","open_source","launched","unknown",{"praise":66,"objections":83,"requests":91,"questions":107,"alternatives":147},[67,79],{"value":68,"source":69,"evidence":70},"Fun project with easy-to-understand code and nice results","commenter",[71,75],{"id":72,"by":73,"text":74},"45933051","embedding-shape","Fun project, easy to understand and nice looking results, everything one could ask for! I played around with it locally, did some optimizations of low hanging fruits without…",{"id":76,"by":77,"text":78},"45949193","aa_y_ush","this is so so awesome.",{"value":80,"source":69,"evidence":81},"Neat idea, curses terminal UI variant",[82],{"id":53,"by":54,"text":55},[84],{"value":85,"source":58,"evidence":86},"No license was attached to the project initially",[87,88],{"id":72,"by":73,"text":74},{"id":89,"by":11,"text":90},"45933086","Hey, I’ll add the MIT licenses later today!",[92,96],{"value":93,"source":69,"evidence":94},"Add a license",[95],{"id":72,"by":73,"text":74},{"value":97,"source":69,"evidence":98},"Build an inference engine for diffusion LLMs",[99,103],{"id":100,"by":101,"text":102},"45934310","gdiamos","One year later and there is still no inference engine for diffusion LLMs Students looking for a project to break into AI - please!",{"id":104,"by":105,"text":106},"45939342","tough","training inspired on nanochat for diffusion models: https:\u002F\u002Fgithub.com\u002FZHZisZZ\u002Fdllm now someone needs to make it work with vllm or something",[108,122,133,140],{"value":109,"source":69,"evidence":110},"How masked token length is handled when output length differs",[111,115,119],{"id":112,"by":113,"text":114},"45931080","yugretcx","Why do these text diffusion demos always look like the number of allowed tokens is fixed for a specific unfilled region? Is this the case? Ie. if the region only has four…",{"id":116,"by":117,"text":118},"45931649","rand0mwalk","Tokens start as a special [MASK] token. Then as the diffusion process runs they are \"unmasked\" i.e. sampled. So yes, you define a sequence of [MASK] tokens with some length ahead…",{"id":120,"by":11,"text":121},"45931660","Yes, this is the case. During training, the model will get a sequence of text (ex, 512 tokens long) with a percentage of them masked out (with a special \u003CMASK> token). It learns…",{"value":123,"source":69,"evidence":124},"Why ReLU^2 activation was chosen",[125,129],{"id":126,"by":127,"text":128},"45932087","Majromax","The basic MLP block in this model uses a ReLU^2 activation function (x \u003C- ReLU(x)^2). That seems to be copied from the nanochat project, and it's not present in nanoGPT. Is there…",{"id":130,"by":131,"text":132},"45936218","kouteiheika","When it comes to compute cost the choice of activation function makes little difference nowadays (and it can often be fused with whatever operation comes before it, which makes…",{"value":134,"source":69,"evidence":135},"Why game of life sampling is used",[136],{"id":137,"by":138,"text":139},"45933799","tell_me_whai","Looks fun, thanks for sharing. I see you're implementing game of life sampling, what's the reasoning for using this logic?",{"value":141,"source":69,"evidence":142},"Can it run on mobile",[143],{"id":144,"by":145,"text":146},"45937406","doppelgunner","This is impressive. Can it run on mobile?",[148,154,161],{"value":149,"source":69,"evidence":150},"NVIDIA Fast-dLLM",[151],{"id":152,"by":11,"text":153},"45934454","Actually NVIDIA made one earlier this year, check out their Fast-dLLM paper",{"value":155,"source":69,"evidence":156},"char-mdlm",[157],{"id":158,"by":159,"text":160},"45932832","volodia","There is also this one that was released in October: https:\u002F\u002Fgithub.com\u002Fkuleshov\u002Fchar-mdlm",{"value":162,"source":69,"evidence":163},"dllm",[164],{"id":104,"by":105,"text":106},{"comments":32,"answered":166,"key":167},2,[168,169,170],{"id":120,"text":121},{"id":89,"text":90},{"id":152,"text":153},{"commenters":172,"max_depth":173,"maker_replies":32,"first_comment_mins":174,"lifespan_hours":175},16,4,6019,152,[177,179,183,184,185,186,188,191,192,195,197,199,201,203,205,208,210,212,214,217],[53,34,43,178,54,34],"project\u002Fpositive\u002Fdesign",[180,35,32,181,182,34],"45932451","project\u002Fneutral\u002Fidea","mlmonkey",[112,166,32,181,113,34],[116,166,32,181,117,34],[120,166,34,181,11,35],[126,32,32,187,127,34],"project\u002Fneutral\u002Fdesign",[189,32,32,187,190,34],"45934415","throwaway2027",[130,32,32,187,131,34],[193,32,32,187,194,34],"45936612","macleginn",[72,173,43,196,73,34],"project\u002Fpositive\u002Fexecution",[89,173,34,198,11,35],"maker\u002Fneutral\u002Fnone",[100,43,34,200,101,34],"other\u002Fcritical\u002Fnone",[152,43,34,202,11,35],"commenter\u002Fneutral\u002Fnone",[204,43,34,202,101,34],"45935380",[206,43,35,207,101,34],"45935723","project\u002Fcritical\u002Fexecution",[104,43,34,209,105,34],"other\u002Fneutral\u002Fnone",[158,211,34,209,159,34],6,[137,213,32,181,138,34],7,[76,215,43,216,77,34],8,"project\u002Fpositive\u002Fnone",[144,33,32,218,145,34],"project\u002Fneutral\u002Fperformance",[220,221],{"id":4,"t":12,"points":14,"comments":15},{"id":222,"t":223,"points":166,"comments":34},"46456200",1767289860,[225,230,235,240,245,249],{"id":226,"name":227,"points":228,"category":229},"45900425","I built a synth for my daughter",1278,"creative",{"id":231,"name":232,"points":233,"category":234},"45838592","What Is Hacker News Working On?",229,"learning_science",{"id":236,"name":237,"points":238,"category":239},"45891066","Unflip",184,"games",{"id":241,"name":242,"points":243,"category":244},"45833811","qqqa",165,"dev_tools",{"id":246,"name":247,"points":248,"category":239},"45887709","Gametje",112,{"id":250,"name":251,"points":252,"category":244},"45870925","DroidDock",78,false,"v2",{"id":256,"name":5,"line":6,"ai":253,"cat":20,"year":13,"points":14,"comments":15,"rec":29,"s":257,"top":18,"kw":258,"maker":11,"domain":10,"tier":19,"attempts":166,"strip":259,"url":9,"mix":260,"praise":68,"objection":85,"sub":21},45876742,0.19,[24,25,26,27,28],"53330333350000100353",[32,33,34,35],1793991133941]